> Markdown version of [/jobs/ext/2632939-mid-level-data-engineer-on-site-in-washington-dc](https://www.wearedevelopers.com/jobs/ext/2632939-mid-level-data-engineer-on-site-in-washington-dc). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Mid-Level Data Engineer (On-Site in Washington, DC) - **Company:** Agile5 Technologies, Inc. - **Location:** Washington, DC, United States - **Experience:** Experienced - **Salary:** $68,000.0 - $152,000.0 - **Contract:** Permanent contract - **Skills:** Agile Methodology, Amazon Web Services, Amazon S3, Apache HTTP Server, Business Logic, Microsoft Azure, Code Review, Databases, Data Validation, Information Engineering, Extract Transform Load (ETL), Data Transformation, Data Migration, Data Profiling, Apache Hadoop, Apache Hive, Python (Programming Language), Power BI, Cloud Services, SQL Databases, SQL Server Integration Services, Talend, Esri GIS (Software), Informatica Powercenter, Apache Spark, Change Data Capture, Git, Data Lakes, Pyspark, Integration Tests, Information Technology, Software Version Control, Data Pipelines, Databricks - **Published:** August 12, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=5dd1484c870417ec ## About the Role * Minimum experience required varies by degree level: PhD with 0 years; Master's degree with 3 years; Bachelor's degree with 5 years; or High School Diploma with 9 years of relevant experience. * Proficiency in Python for data transformation and pipeline development, as well as SQL for query development and schema analysis. * Experience with ETL/ELT processes, data migration methodologies, and cloud data platforms (AWS, Azure, or GCP). * Familiarity with version control systems (Azure DevOps, Git) and data quality concepts including profiling, cleansing, and reconciliation. Education Requirements: Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field is preferred (or equivalent combination of education and experience). Desired Skills / Qualifications: * Experience with Databricks (notebooks, jobs, workspace navigation), PySpark, Apache Spark, and Delta Lake or Apache Iceberg table formats. * Proven track record converting visual ETL tools (Informatica, Talend, SSIS) to code-based pipelines. * Experience with Hive, HiveQL, or Hadoop ecosystem components. * Familiarity with federal IT environments, security requirements, and CI/CD pipelines for data engineering workflows. ## Description Description: The Mid-Level Data Engineer will support data migration, pipeline engineering, and modernization efforts for enterprise data lakehouse architectures. This role involves converting legacy Informatica artifacts into clean Python/PySpark code, migrating database schemas, and building automated data reconciliation pipelines. Working closely with senior engineering leadership and database managers, the ideal candidate will enforce high standards of data quality, data validation, and version control in a secure federal environment., * Execute daily data migration operations including data profiling, schema mapping, pipeline conversion, and automated reconciliation for Low and Medium complexity Informatica artifacts. * Convert Informatica mappings into well-documented Python/PySpark code, ensuring all business logic and data quality controls are preserved. * Migrate legacy Hive tables to Delta Lake format on S3 using Databricks ingestion tools. * Build and execute automated data reconciliation scripts to validate migration accuracy and establish Change Data Capture (CDC) pipelines for ongoing synchronization. * Commit all converted code into Azure DevOps with clear documentation and inline comments while maintaining Unity Catalog configurations. * Perform daily data profiling and side-by-side validation within legacy enclave environments. * Support Power BI and ESRI integration testing and validation. * Participate actively in peer code reviews, daily Agile ceremonies, and collaborative data validation sessions. * Contribute to Data Quality Assessment Reports and support training and knowledge transfer activities. * Performs other duties as assigned. Security Clearance Requirements: * Public Trust / Tier 4 Eligible: No clearance required to apply; must be a U.S. citizen willing to undergo a background check to obtain a Public Trust / Tier 4 clearance. ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)