Databricks Azure Consultant

The Nam

Houston, TX, United States

3 days ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Temporary contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

2 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Application Programming Interfaces (APIs) Agile Methodology Artificial Intelligence Amazon Web Services Microsoft Azure Big Data Cloud Storage Databases Continuous Integration Data Architecture Information Engineering Extract Transform Load (ETL)

+23 more

Data Warehousing DevOps Apache Hive Machine Learning Performance Tuning Software Deployment SQL Databases Data Streaming Google Cloud Cloud Platform System Azure Data Factory Apache Spark Git Data Lakes Pyspark Information Technology AWS Glue Apache Kafka Machine Learning Operations Video Streaming Azure Synapse Analytics Data Pipelines Databricks

Job description

We are seeking an experienced Databricks Consultant to design, develop, and implement scalable data solutions on the Databricks platform., * Design and develop data pipelines using Databricks, Apache Spark, and PySpark.
Build and maintain ETL/ELT processes for large-scale data ingestion and transformation.
Implement data lake and lakehouse architectures on Databricks.
Integrate data from multiple sources, including databases, APIs, and cloud storage.
Optimize Spark jobs and Databricks workloads for performance and cost efficiency.
Collaborate with business users, architects, and developers to understand requirements and deliver solutions.
Develop data models and support reporting, analytics, and AI/ML initiatives.
Implement data quality, governance, security, and monitoring standards.
Support production deployments, troubleshooting, and performance tuning.
Prepare technical documentation and provide knowledge transfer to project teams.

Requirements

The ideal candidate will have strong expertise in data engineering, cloud platforms, Spark, and modern data architectures, with the ability to work closely with business and technical stakeholders., * Strong hands-on experience with Databricks.
Expertise in Apache Spark, PySpark, and SQL.
Experience with cloud platforms such as Azure, AWS, or Google Cloud Platform.
Knowledge of Delta Lake, Unity Catalog, and Databricks Workflows.
Experience with data warehousing and data lake concepts.
Strong understanding of ETL/ELT frameworks and data modeling.
Familiarity with CI/CD, Git, and DevOps practices.
Excellent problem-solving and communication skills.

Preferred Skills

Experience with Azure Data Factory (ADF), Azure Synapse, or AWS Glue.
Knowledge of Machine Learning and MLOps on Databricks.
Experience with streaming technologies such as Kafka and Structured Streaming.
Databricks or Cloud Platform certifications., * Bachelor’s degree in Computer Science, Information Technology, Engineering, or related field.
5+ years of experience in Data Engineering/Big Data.
2+ years of hands-on Databricks implementation experience.

Nice to Have

Databricks Certified Data Engineer Associate/Professional.
Experience in Utilities, Energy, Banking, Healthcare, or Retail domains.
Experience with Agile delivery methodologies.
Primary Technologies: Databricks, PySpark, Spark SQL, Delta Lake, Azure/AWS/Google Cloud Platform, ADF, Kafka, Git, CI/CD.

Experience: 7+ Years