> Markdown version of [/jobs/ext/2115067-data-engineer](https://www.wearedevelopers.com/jobs/ext/2115067-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** 540 - **Location:** United States (Remote available) - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Amazon Web Services, Automation of Tests, Big Data, Command-Line Interface, Code Review, Data Validation, Information Engineering, Data Infrastructure, Extract Transform Load (ETL), Data Transformation, Software Debugging, Distributed Data Store, Python (Programming Language), Operational Databases, Standard Sql, Software Engineering, SQL Databases, Systems Integration, Apache Spark, Gitlab, Git, Information Technology, Deployment Automation, Api Design, Restful APIs, Data Pipelines, Databricks - **Published:** August 19, 2026 - **Apply:** https://job-boards.greenhouse.io/540/jobs/7886211003 ## About the Role Citizenship & Clearance Requirement: per client requirements, candidates must be U.S. Citizens with an active DoW Secret (or higher) clearance Education Requirement: Bachelor's Degree in Computer Science, Engineering, Data Science, or a related technical field (preferred) 540 Internal Thrive Level: Data Engineer II or III, * 3+ years of data engineering, software engineering, or related technical experience * Strong proficiency with Python and SQL * Experience with Apache Spark or similar distributed data-processing frameworks * Experience working with Databricks or a similar modern data platform * Experience building and maintaining production data pipelines and ETL/ELT processes * Working knowledge of data modeling and transforming raw data into reusable analytics datasets or data products * Experience integrating or consuming data through REST APIs * Experience troubleshooting and optimizing data pipelines for performance and reliability * Familiarity with data validation, quality, governance, security, privacy, and compliance considerations * Experience with Git-based development workflows and modern software engineering practices * Comfort working in terminal and command-line environments * Strong communication skills and experience working with customers or stakeholders to understand and resolve data needs * Ability to navigate ambiguity, learn unfamiliar systems, and independently drive assigned work toward resolution NICE TO HAVE SKILLS & EXPERIENCE * AWS cloud experience * Experience working with very large datasets, including datasets containing billions of records * Experience with Palantir Foundry * Experience with Advana or similar DoW data environments * Experience with GitLab, CI/CD pipelines, or automated deployment workflows * Experience working with federal health, financial, or other regulated and sensitive data * Experience using AI-assisted development tools to accelerate routine engineering tasks, testing, documentation, and debugging ## Description * Design, develop, and maintain scalable, production-ready data pipelines using Spark, Python, and SQL in a Databricks environment * Integrate and transform diverse health, financial, and operational datasets into reliable, reusable data products * Develop data models and analytics datasets that support enterprise reporting and operational decision-making * Build and maintain API-based integrations for ingesting and exchanging data across systems * Monitor, troubleshoot, and optimize scheduled workflows and production pipelines for performance, reliability, and scalability * Implement data validation, quality checks, lineage, and governance practices that improve trust in data products * Contribute to automated testing, code reviews, technical documentation, and reusable engineering patterns * Collaborate with engineers, architects, analysts, and customer stakeholders to translate data needs into practical technical solutions, Select... This position requires an active DoW Secret clearance or higher. Which best describes your current status?* Select... Targeted Annual Salary:* LinkedIn Profile Voluntary Self-Identification For government reporting purposes, we ask candidates to respond to the below self-identification survey. Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiring process or thereafter. Any information that you do provide will be recorded and maintained in a confidential file. As set forth in 540's Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law. Gender Select... Are you Hispanic/Latino? Select... Race & Ethnicity Definitions If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection. As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measure the effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categories is as follows: A "disabled veteran" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability. A "recently separated veteran" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service. An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense. An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985. Veteran Status Select... ## Related Videos - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [WeAreDevelopers LIVE - Modern DevOps for IoT Devices and More](https://www.wearedevelopers.com/videos/1805-wearedevelopers-live-modern-devops-for-iot-devices-and-more) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Data Governance in the Era of AI](https://www.wearedevelopers.com/videos/1622-data-governance-in-the-era-of-ai) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [The 12 Best Jobs for Software Engineers](https://www.wearedevelopers.com/magazine/401-the-12-best-jobs-for-software-engineers) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries)