Databricks Solutions Architect

We are seeking an experienced Databricks Solutions Architect to design, develop, and implement scalable, secure, and high-performance data solutions using the Databricks Lakehouse Platform. The ideal candidate will have strong expertise in data architecture, cloud data engineering, Apache Spark, PySpark, SQL, Delta Lake, and modern data platform design., * Design and implement end-to-end data architectures using Databricks and the Lakehouse platform.

• Develop scalable data engineering solutions using PySpark, Apache Spark, SQL, and Delta Lake.
• Implement Medallion Architecture, including Bronze, Silver, and Gold data layers.
• Design efficient batch, near-real-time, and streaming data processing pipelines.
• Lead data migration, legacy platform modernization, and cloud data transformation initiatives.
• Optimize Spark jobs, Delta tables, SQL queries, and data pipelines for performance and cost efficiency.
• Implement Change Data Capture (CDC) and incremental data processing strategies.
• Establish data governance, security, access controls, and data quality standards using appropriate Databricks capabilities.
• Design and implement CI/CD pipelines using Git, GitHub, Jenkins, and Databricks Asset Bundles.
• Collaborate with data engineers, business stakeholders, cloud architects, and analytics teams.
• Define architectural standards, technical documentation, best practices, and reusable design patterns.
• Mentor engineering teams and provide technical leadership throughout solution delivery.

• 10 15 years of experience in data engineering, data architecture, or enterprise data platform development.
• Strong hands-on experience with Databricks, Apache Spark, PySpark, SQL, and Delta Lake.
• Proven experience designing end-to-end data solutions and scalable data architectures.
• Strong understanding of Lakehouse architecture and Medallion Architecture.
• Experience with ETL/ELT pipelines, data integration, data migration, and legacy modernization.
• Hands-on experience with batch processing, near-real-time processing, streaming, and CDC.
• Experience with cloud data platforms such as Microsoft Azure, AWS, or Google Cloud Platform.
• Experience with Unity Catalog and enterprise data governance.
• Experience with infrastructure as code and automated deployment practices.
• Knowledge of data modeling, data warehousing, partitioning, and performance optimization.
• Familiarity with CI/CD practices, GitHub, Jenkins, and Databricks Asset Bundles.
• Understanding of data governance, security, data quality, and access management.
• Strong analytical, problem-solving, communication, and stakeholder-management skills.
• Knowledge of Data Mesh and modern distributed data architecture patterns.
• Exposure to orchestration tools such as Azure Data Factory, Apache Airflow, or equivalent platforms.

Qualification: Bachelor’s degree or equivalent combination of education and experience.

Artmac Soft is a technology consulting and service-oriented IT company dedicated to providing innovative technology solutions and services to customers.