Data Architect (Databricks, Azure) - Hybrid Onsite
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+8 more
Job description
Marathon TS is looking to hire an onsite Data Architect (Databricks, Azure) to support a Federal Agency in Washington DC. We are looking for a Data Architect to design the data foundation for a generative AI program at a regulatory client. You will own how business applications and a Databricks lakehouse exchange data, so that every AI workflow has clean inputs, stable output contracts and governed access to highly confidential information., * Define the interface contracts between business applications and AI workflows: inputs, output schemas, versioning and error handling.
- Design ingestion of large document submissions (PDF, Excel and Word, with single submissions of up to hundreds of gigabytes) from Azure storage into Databricks, including metadata, document keys and text extraction.
- Design the Unity Catalog model for confidential data: catalogs, schemas, fine-grained access controls tied to user assignments, and lineage.
- Model AI outputs and reviewer feedback: the AI draft versus the human final, provenance fields (model and prompt version, evidence links, confidence) and the write-back tables used to refine prompts.
- Recommend the operational data store pattern (Azure SQL, PostgreSQL or Databricks Lakebase) and how it stays in sync with the lakehouse.
- Plan and integrate the historical data the AI workflows depend on.
- Keep compute cost-efficient through cluster policies, job design and consumption tracking.
What you bring
- 10+ years in data architecture, including 4+ years designing Databricks lakehouses (Delta Lake, Unity Catalog, Workflows or Lakeflow).
- Azure data and security: ADLS or Blob Storage, private endpoints, Entra ID and Key Vault.
- Experience designing data contracts between applications and a data platform, beyond warehouse or BI modeling.
- Governance for sensitive or regulated data: fine-grained access control, lineage and audit.
- Pipelines for unstructured documents at high volume.
- Databricks certification (Data Engineer Professional or architect level).
Nice to have
- Data patterns for generative AI: vector search and retrieval-augmented generation.
- Experience in financial regulation, audit or federal environments.
Working with us
- You will work inside a secure, private cloud environment with highly confidential data.
- You must be authorized to work in the United States and able to pass a background check.
- We are looking for people who can start within a few weeks.
Requirements
DataArchitect Data Modeling Tool, Microsoft Windows Azure, Artificial Intelligence (AI) Programming Languages, Artificial Intelligence (AI), DataArchitect Data Modeling Tool, Federal Government, Microsoft Windows Azure, Business Solutions, Regulations, Architectural Design
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦