Databricks Engineer

Apex Systems LLC
Austin, TX, United States
1 day ago
Apply on www.adzuna.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
2 years minimum
Working hours
Regular working hours
Job source

Tech stack

Unity 3d Application Programming Interfaces (APIs) Unit Testing Cloud Computing Code Review Information Systems Continuous Integration Information Engineering Document-Oriented Databases Python (Programming Language) Machine Learning Operational Databases
+13 more
Cloud Services Search Technologies SQL Databases Feature Store Retrieval-Augmented Generation Git Pandas Build Management Data Lakes Pyspark Information Technology Data Pipelines Databricks

Job description

  • Design and build production data pipelines on Databricks utilizing PySpark, SQL, Delta Lake, Unity Catalog, and Workflows/Jobs with high standards for monitoring and clear lineage.
  • Model bronze, silver, and gold layers alongside defining data contracts, schemas, service level agreements, and quality checks.
  • Develop machine learning-oriented data products, including feature stores, curated tables, embedding generation, chunking and indexing for RAG, and batch classification.
  • Integrate data pipelines with cloud services, utilizing security patterns such as key vaults, managed identities, and private connectivity.
  • Partner with application and machine learning teams to expose stable APIs, tables, and volumes that power orchestration workflows.
  • Document data dictionaries, runbooks, and handoff materials while actively participating in code reviews and agile ceremonies.

Requirements

An established organization is seeking a skilled professional to augment the internal data platform team. This role focuses on delivering robust lakehouse pipelines, catalogued engineering datasets, and enterprise data products in parallel with ongoing priority programs. The ideal candidate will quickly adapt to new domains, collaborate closely with internal leads, and contribute to standardizing Unity Catalog governance and architecture., Education: Bachelor’s degree in Computer Science, Information Systems, or a related technical field is preferred.

Experience: Minimum of 5 years of experience in data engineering, including at least 2 years of production experience with Databricks in a cloud environment.

Technical Skills: Proficiency in Python (PySpark and pandas), SQL, Delta Lake, and Unity Catalog. Practical experience with Git, continuous integration repository layouts, and unit testing for data pipelines is required.

Preferred Qualifications

  • Experience building pipelines that feed machine learning and generative AI use cases, such as vector search inputs and batch inference outputs.
  • Proven ability to work effectively within staff augmentation models alongside internal technology leads.

About the company

Everforth Apex is a world-class IT services company that serves thousands of clients across the globe. When you join Everforth Apex, you become part of a team that values innovation, collaboration, and continuous learning. We offer quality career resources, training, certifications, development opportunities, and a comprehensive benefits package. Our commitment to excellence is reflected in many awards, including ClearlyRateds Best of Staffing in Talent Satisfaction in the United States and Great Place to Work in the United Kingdom and Mexico.

Everforth Apex uses a virtual recruiter as part of the application process. Click for more details. By applying for this job, you agree to receive calls, AI-generated calls, text messages, or emails from Everforth Apex and its affiliates, and contracted partners. Frequency varies for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You can reply STOP to cancel and HELP for help. You can access our privacy policy at, UnitedHealthcare creates and publishes the Transparency in Coverage Machine-Readable Files on behalf of Everforth Apex Systems.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.adzuna.com
Prepare application

Good distractions

Loading talks and stories from around this role…