> Markdown version of [/jobs/ext/1483508-data-engineer-veterans-affairs](https://www.wearedevelopers.com/jobs/ext/1483508-data-engineer-veterans-affairs). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer- Veterans Affairs - **Company:** ThunderYard Solutions LLC - **Location:** United States (Remote available) - **Experience:** Experienced - **Salary:** $100,000.0 - $120,000.0 - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Airflow, Data Analysis, Microsoft Azure, Cloud Storage, Continuous Integration, Data Architecture, Data Validation, Data Deduplication, Data Governance, Extract Transform Load (ETL), Data Transformation, Data Systems, Data Warehousing, Relational Databases, Python (Programming Language), Query Optimization, Power BI, Standard Sql, Software Deployment, SQL Databases, Data Streaming, Tableau (Software), Workflow Management Systems, Privacy Controls, Azure Service Bus, Cloud Platform System, Azure Data Factory, Apache Spark, Git, Data Lakes, Pyspark, Real Time Data, Apache Kafka, Data Management, Tools for Reporting, Azure Synapse Analytics, Looker Analytics, Software Version Control, Data Pipelines, Databricks - **Published:** July 29, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=7b91074aae697c25 ## About the Role This role will collaborate with cross-functional teams, including data analysts, software developers, and government stakeholders, to optimize data pipelines, improve data accessibility, and ensure compliance with federal security and privacy standards. The successful candidate will demonstrate strong problem-solving skills, technical leadership, and the ability to work effectively in an agile environment supporting veteran-focused initiatives., * 3+ years of experience as a Data Engineer or in a similar data focused role * Hands on experience with Databricks * Strong experience building ETL/ELT pipelines * Proficiency in Python and SQL * Experience with Apache Spark / PySpark * Experience with Azure, Azure Synapse Analytics, and Azure Data Factory * Solid understanding of data modeling, data warehousing, and analytics use cases, Preferred / Nice to Have * Experience with Delta Live Tables (DLT) or Databricks Auto Loader * Experience with orchestration tools such as Airflow * Familiarity with streaming data technologies (Kafka, Event Hubs, Kinesis) * Experience supporting analytics tools (Power BI, Tableau, Looker) connected to Databricks * Databricks certification (Associate or Professional), * Azure: 3 years (Required) * Databricks: 3 years (Required) * Azure Data Factory: 3 years (Required) ## Description ThunderYard Solutions is seeking a Data Engineer to support the U.S. Department of Veterans Affairs in designing, developing, and maintaining scalable data solutions that support mission-critical healthcare and business operations. The ideal candidate will have expertise in data architecture, ETL/ELT processes, cloud-based data platforms, and analytics technologies, with a strong commitment to delivering secure, high-quality data solutions in a federal environment., * Design, develop, and maintain ETL/ELT pipelines to ingest, transform, and load data from multiple sources such as APIs, relational databases, cloud storage, and streaming platforms * Build scalable batch and near real time data pipelines using Databricks and Apache Spark (PySpark / SQL) * Implement data transformation logic following best practices for performance, reliability, and reusability * Support schema evolution, data validation, deduplication, and error handling in ETL workflows Databricks Platform Development * Develop and optimize pipelines using Delta Lake and medallion (Bronze / Silver / Gold) architecture patterns * Use Databricks Workflows / Jobs or similar orchestration tools to schedule and monitor pipelines * Optimize Spark jobs for performance and cost (partitioning, caching, file sizing, query tuning) * Collaborate on data governance initiatives using Unity Catalog, access controls, and lineage where applicable Collaboration & Operations * Work closely with data architects, analytics teams, and downstream consumers to define data requirements * Troubleshoot pipeline failures and data quality issues and implement long term fixes * Produce documentation for pipelines, datasets, and operational runbooks * Participate in CI/CD practices using Git based version control for notebooks and code deployments ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)