Azure Data Bricks Architect

Lead the architectural design and deployment of comprehensive data solutions on the Databricks platform. Develop and enhance scalable data pipelines processing both structured and unstructured data using Apache Spark within Databricks. Establish and maintain best practices for data engineering and system architecture. Collaborate closely with data scientists, analysts, and business stakeholders to gather requirements and convert them into scalable, effective solutions. Design and oversee the development of CI/CD pipelines and automation workflows for Databricks projects. Ensure all solutions meet data governance, security, and compliance standards. Provide technical guidance and mentorship to the data engineering team. Assess and integrate new tools and technologies to improve platform performance and efficiency. Work with cloud environments such as Azure, AWS, or GCP integrated with Databricks.

Minimum 8 years of hands-on experience with Databricks technologies, including Apache Spark, Delta Lake, MLflow, and DBSQL. Extensive experience building large-scale ETL/ELT data pipelines. Strong proficiency in data modeling, data warehousing concepts, and distributed computing frameworks. Experience working with cloud platforms-preferably Azure, but AWS or GCP is also acceptable. Demonstrated ability to implement and enforce data engineering best practices and architectural standards. Skilled in Python, SQL, and Spark-based data processing. Solid understanding of DevOps methodologies related to Databricks, including CI/CD and infrastructure as code., Knowledge of BI tools such as Power BI or Synapse Analytics. Certifications like Databricks Certified Data Engineer or Microsoft Certified: Azure Solutions Architect are advantageous. Role Descriptions: Lead the architectural design and deployment of comprehensive data solutions on the Databricks platform.Develop and enhance scalable data pipelines processing both structured and unstructured data using Apache Spark within Databricks.Establish and maintain best practices for data engineering and system architecture.Collaborate closely with data scientists| analysts| and business stakeholders to gather requirements and convert them into scalable| effective solutions.Design and oversee the development of CI/CD pipelines and automation workflows for Databricks projects.Ensure all solutions meet data governance| security| and compliance standards.Provide technical guidance and mentorship to the data engineering team.Assess and integrate new tools and technologies to improve platform performance and efficiency.Work with cloud environments such as Azure| AWS| or GCP integrated with Databricks.Required Skills & Experience:Minimum 8 years of hands-on experience with Databricks technologies| including Apache Spark| Delta Lake| MLflow| and DBSQL.Extensive experience building large-scale ETL/ELT data pipelines.Strong proficiency in data modeling| data warehousing concepts| and distributed computing frameworks.Experience working with cloud platformspreferably Azure| but AWS or GCP is also acceptable.Demonstrated ability to implement and enforce data engineering best practices and architectural standards.Skilled in Python| SQL| and Spark-based data processing.Solid understanding of DevOps methodologies related to Databricks| including CI/CD and infrastructure as code.Preferred Qualifications:Knowledge of BI tools such as Power BI or Synapse Analytics.Certifications like Databricks Certified Data Engineer or Microsoft Certified: Azure Solutions Architect are advantageous. Essential Skills: Lead the architectural design and deployment of comprehensive data solutions on the Databricks platform.Develop and enhance scalable data pipelines processing both structured and unstructured data using Apache Spark within Databricks.Establish and maintain best practices for data engineering and system architecture.Collaborate closely with data scientists| analysts| and business stakeholders to gather requirements and convert them into scalable| effective solutions.Design and oversee the development of CI/CD pipelines and automation workflows for Databricks projects.Ensure all solutions meet data governance| security| and compliance standards.Provide technical guidance and mentorship to the data engineering team.Assess and integrate new tools and technologies to improve platform performance and efficiency.Work with cloud environments such as Azure| AWS| or GCP integrated with Databricks.Required Skills & Experience:Minimum 8 years of hands-on experience with Databricks technologies| including Apache Spark| Delta Lake| MLflow| and DBSQL.Extensive experience building large-scale ETL/ELT data pipelines.Strong proficiency in data modeling| data warehousing concepts| and distributed computing frameworks.Experience working with cloud platformspreferably Azure| but AWS or GCP is also acceptable.Demonstrated ability to implement and enforce data engineering best practices and architectural standards.Skilled in Python| SQL| and Spark-based data processing.Solid understanding of DevOps methodologies related to Databricks| including CI/CD and infrastructure as code.Preferred Qualifications:Knowledge of BI tools such as Power BI or Synapse Analytics.Certifications like Databricks Certified Data Engineer or Microsoft Certified: Azure Solutions Architect are advantageous. Desirable Skills: Keyword: Skills: Digital : Databricks Experience Required: 8-10