Senior Azure Databricks Data Engineer

• Design and implement scalable data and application architectures using Python and Azure ML.
• Develop complex ETL/ELT pipelines using Python, Spark, and Databricks.
• Process and transform large-scale datasets from multiple sources.
• Optimize Python/Spark jobs for performance and cost efficiency.
• Perform partitioning, caching, cluster tuning, and bottleneck troubleshooting.
• Work with data scientists, analysts, and business stakeholders to convert business requirements into technical solutions.
• Implement data security, governance, compliance, and access controls.
• Develop and maintain CI/CD pipelines for deployment across development and production environments.
• Build production-grade monitoring, logging, and alerting solutions.
• Maintain code quality and follow software development best practices.
• Support project planning and delivery across multiple Agile pods.

Nice-to-Have

• Java / Spring
• Strong knowledge of various Azure services
• Experience using AI tools to improve developer productivity.

Submission format please complete in full: First Middle Last Name Bill rate: OR W2 Pay Rate: Visa Status: Location: Availability to interview: Availability to start: Vacations: LinkedIn Profile: Phone Number: Email Address: Last 4 of SSN: MM/DD of birth: If on a visa, attach copy. Proof of ID - please attach Previous Client Experience (need manager name, group/LOB, if they were FTE or contractor, and dates)

Looking for a strong Senior Data Engineer with deep experience in Python, Databricks, Spark, Azure Cloud, Azure ML, ETL/ELT, and application development. The candidate should be comfortable working across multiple Agile pods and helping with project planning, delivery, and technical execution. Must-Have Skills

• 7+ years of experience in Python, ETL, Data Engineering, and SQL
• 5+ years of experience with Apache Spark and Databricks
• Strong Python development and application development experience
• Strong experience with Azure Cloud
• Experience with Azure ML
• Advanced knowledge of Python, Scala and/or SQL
• Strong ETL/ELT development and large-scale data processing
• Experience with Databricks Unity Catalog, data governance, security, access controls, encryption, and data lineage
• Strong CI/CD and DevOps experience
• Experience with Azure DevOps, GitHub Actions, or Jenkins
• 5+ years of experience with Logging, Monitoring, and Alerting
• Strong understanding of SDLC
• Excellent communication and stakeholder-management skills

This assignment is 12 months with opportunity to convert. Candidate must be working onsite Tuesdays - Thursdays; virtual on Mondays and Fridays. There will be 2 round interviews; 1. Round of virtual for 45 min. 2. In person interview at SV7 if selected in Round 1. In person interview is Mandatory with no exceptions and Local candidates are preferred. There will be a coding assignment during interview.