DataBricks Specialist

Architecture & Development: Design end-to-end data pipelines, ETL/ELT processes, and lakehouse storage patterns using Apache Spark, Delta Lake, and Python or SQL.

• Performance Tuning: Optimize cluster sizing, query execution, shuffles, partitions, and storage layouts for large-scale data workflows. [1]
• AI & Machine Learning Integration: Implement MLOps workflows, manage model lifecycles, or deploy generative AI and vector search solutions using tools like MLflow and Unity Catalog. [1]
• Collaboration & Consulting: Partner with enterprise stakeholders, data scientists, and software engineers to translate business requirements into technical solutions. [1]
• Administration & Governance: Oversee security configurations, access controls, workspace administration, and CI/CD deployment pipelines

Software Development, Python Programming/Scripting Language, Apache Spark, Scala Programming Language, SQL (Structured Query Language), Ecosystems, Cloud Computing, Microsoft Windows Azure, GCP (Good Clinical Practices), Amazon Web Services (AWS), · Experience: 5 to 10+ years in data engineering, software development, or solution architecture, with hands-on Databricks production experience. · Technical Stack: Strong proficiency in Python, SQL, or Scala; deep knowledge of Apache Spark runtime internals and Delta Lake. · Cloud Ecosystems: Working knowledge of major cloud platforms like Microsoft Azure, AWS, or GCP. · Certifications: Preferred credentials include Databricks Certified Data Engineer or Solutions Architect credentials. []