Databricks Engineer
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+13 more
Job description
- Design and build production data pipelines on Databricks utilizing PySpark, SQL, Delta Lake, Unity Catalog, and Workflows/Jobs with high standards for monitoring and clear lineage.
- Model bronze, silver, and gold layers alongside defining data contracts, schemas, service level agreements, and quality checks.
- Develop machine learning-oriented data products, including feature stores, curated tables, embedding generation, chunking and indexing for RAG, and batch classification.
- Integrate data pipelines with cloud services, utilizing security patterns such as key vaults, managed identities, and private connectivity.
- Partner with application and machine learning teams to expose stable APIs, tables, and volumes that power orchestration workflows.
- Document data dictionaries, runbooks, and handoff materials while actively participating in code reviews and agile ceremonies.
Requirements
An established organization is seeking a skilled professional to augment the internal data platform team. This role focuses on delivering robust lakehouse pipelines, catalogued engineering datasets, and enterprise data products in parallel with ongoing priority programs. The ideal candidate will quickly adapt to new domains, collaborate closely with internal leads, and contribute to standardizing Unity Catalog governance and architecture., Education: Bachelorβs degree in Computer Science, Information Systems, or a related technical field is preferred.
Experience: Minimum of 5 years of experience in data engineering, including at least 2 years of production experience with Databricks in a cloud environment.
Technical Skills: Proficiency in Python (PySpark and pandas), SQL, Delta Lake, and Unity Catalog. Practical experience with Git, continuous integration repository layouts, and unit testing for data pipelines is required.
Preferred Qualifications
- Experience building pipelines that feed machine learning and generative AI use cases, such as vector search inputs and batch inference outputs.
- Proven ability to work effectively within staff augmentation models alongside internal technology leads.
About the company
Everforth Apex is a world-class IT services company that serves thousands of clients across the globe. When you join Everforth Apex, you become part of a team that values innovation, collaboration, and continuous learning. We offer quality career resources, training, certifications, development opportunities, and a comprehensive benefits package. Our commitment to excellence is reflected in many awards, including ClearlyRateds Best of Staffing in Talent Satisfaction in the United States and Great Place to Work in the United Kingdom and Mexico.
Everforth Apex uses a virtual recruiter as part of the application process. Click for more details. By applying for this job, you agree to receive calls, AI-generated calls, text messages, or emails from Everforth Apex and its affiliates, and contracted partners. Frequency varies for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You can reply STOP to cancel and HELP for help. You can access our privacy policy at, UnitedHealthcare creates and publishes the Transparency in Coverage Machine-Readable Files on behalf of Everforth Apex Systems.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦