Databricks Solution Architect

• Platform Architecture & Design: Lead the design and implementation of enterprise-scale, secure, and high-performance data architectures on the Databricks Lakehouse Platform, leveraging Delta Lake, Unity Catalog, serverless compute, and advanced features like Photon engine.

• Hands-On Implementation & Leadership: Provide expert hands-on guidance and lead development of complex data engineering pipelines using Apache Spark, PySpark, SQL, and Scala, with a focus on code quality, optimization, and best practices.
• Performance Tuning & Optimization: Identify, diagnose, and resolve complex performance issues in large-scale data processing, query optimization, and cluster configurations.
• Client Advisory & Strategy: Serve as the trusted senior technical advisor to client leadership, engineering, and data science teams, delivering best practices in data governance, security, MLOps, and operational excellence.
• Knowledge Transfer & Team Enablement: Design and deliver advanced workshops, training sessions, and ongoing mentorship to accelerate client team proficiency and self-sufficiency.
• Feature Adoption & Innovation: Champion the adoption of new Databricks features and tools (e.g., Photon, AI/BI, Mosaic AI) and integrate them effectively into client environments.
• Cloud Integration: Ensure seamless, optimized, and secure integration with major cloud platforms (AWS, Azure, Google Cloud Platform) including storage, networking, IAM, and security services.

• 8+ years of experience in data engineering, big data architecture, or software engineering with a focus on data platforms.
• 5+ years of deep, hands-on experience designing, implementing, and optimizing solutions on the Databricks platform.
• Expert-level proficiency in Apache Spark ecosystem (PySpark, Spark SQL, Scala) and large-scale data processing.
• Strong experience with cloud infrastructure (AWS, Azure, Google Cloud Platform), including storage solutions (S3, ADLS), networking, and security models.
• Deep expertise in Databricks Lakehouse architecture, Delta Lake, Unity Catalog, and best practices for ETL/ELT, data governance, and reliability.
• Proficiency in Python, Scala, advanced SQL, and modern DevOps practices including Docker, Kubernetes, and CI/CD.

Professional & Communication Skills:

• Exceptional client-facing and presentation skills, able to communicate complex concepts to both technical and C-level audiences.
• Proven leadership in client engagements, ability to drive consensus and influence decision-making across stakeholder groups.
• Strong problem-solving, analytical skills, and a proactive, consultative mindset.

Preferred Qualifications:

• Databricks Certified Professional or higher certifications.
• Relevant cloud certifications (e.g., AWS Certified Big Data, Azure Data Engineer, Google Professional Data Engineer).
• Experience with MLOps, generative AI, and advanced analytics on Databricks.
• Prior experience in senior consulting, solution architecture, or resident roles