Data Engineer (Databricks, Azure) - Hybrid Onsite

Marathon TS Inc
Tysons, VA, United States
17 days ago
Apply on www.careerbuilder.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
3 years minimum
Working hours
Regular working hours

Tech stack

Unity 3d Artificial Intelligence Microsoft Azure Spreadsheets Continuous Integration Data Validation Information Engineering Data Infrastructure Python (Programming Language) PostgreSQL Metadata SQL Azure
+11 more
Networking Basics Operational Data Store Standard Sql SQL Databases Private Cloud Environment Azure Data Factory Data Lakes Pyspark Data Pipelines Databricks Programming Languages

Job description

  • Deliver AI outputs to the applications’ operational data store (Azure SQL, PostgreSQL or Lakebase), and load reviewer edits back for prompt refinement.
  • Load and reconcile historical data from multiple sources, including SQL databases and spreadsheets, into curated tables.
  • Orchestrate jobs with Workflows or Lakeflow, with data quality checks, monitoring and alerting.
  • Deploy Databricks assets through CI/CD (Asset Bundles) across development, test and production.
  • Manage Unity Catalog permissions and keep compute costs down.

Requirements

Microsoft Windows Azure, Artificial Intelligence (AI) Programming Languages, Artificial Intelligence (AI), Microsoft Windows Azure, Federal Government, Business Solutions, Regulations, Metadata, Spreadsheets, SQL Databases, * 6+ years in data engineering, including 3+ years on Databricks: PySpark, Delta Lake, Workflows and Unity Catalog.

  • Strong SQL and Python; CI/CD for data pipelines.
  • Experience with large files and unstructured documents, not only tabular data.
  • Azure storage and networking fundamentals.
  • PostgreSQL or Databricks Lakebase, Azure SQL and Azure Data Factory.

Nice to have

  • Experience in regulated or confidential data environments.

Working with us

  • You will work inside a secure, private cloud environment with highly confidential data.
  • You must be authorized to work in the United States and able to pass a background check.
  • We are looking for people who can start within a few weeks.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerbuilder.com
Prepare application

Good distractions

Loading talks and stories from around this role…