Databricks Engineering

Design enterprise ingestion frameworks using Databricks Native Data Loader, Lakeflow Connect, and other Databricks-native capabilities.

Build and optimize batch and streaming pipelines integrating Oracle, MySQL, SQL Server, APIs, and other enterprise sources.

Develop robust ELT and ETL workflows with Spark, Delta Lake, and cloud-native technologies, covering extraction, transformation, validation, delivery, and monitoring.

Transform raw and unstructured data into reliable datasets that satisfy functional and non-functional business requirements.

Lead architecture decisions across ingestion, storage, processing, and governance; improve infrastructure efficiency and platform performance.

Implement Data Quality controls, investigate production issues through root cause analysis, and recommend corrective actions.

Write unit and integration tests, apply Test-Driven Development (TDD) practices, and maintain technical documentation.

Mentor junior engineers, conduct architecture and code reviews, and promote DevOps, CI/CD, and Agile practices, including sprint planning and estimation.

Deep practical expertise in distributed data processing, enterprise migration, and modern data pipeline architecture.

A strong engineering mindset focused on automation, scalability, operational reliability, and measurable data quality.

The ability to mentor engineers, explain architectural trade-offs, and translate business requirements into maintainable technical solutions.

Confidence collaborating across Product, Engineering, Analytics, Architecture, and Business teams., Hands-on experience building scalable data solutions with Databricks, Spark, and Delta Lake.

Strong SQL skills and practical understanding of relational database integration and enterprise data migration.

Experience implementing data validation, monitoring, testing, and production reliability practices.

Ability to design distributed processing solutions and lead technical delivery with business and technology stakeholders.

Preferred Qualifications

Experience improving reusable ingestion frameworks, automated deployment workflows, and enterprise data governance.

Familiarity with preparing trusted enterprise datasets for advanced analytics and Generative AI applications.

Thinking Minds’ client offers a collaborative engineering environment, opportunities to guide platform modernization, and scope to develop technical leadership while delivering enterprise data solutions.