> Markdown version of [/jobs/ext/654221-data-automation-engineer](https://www.wearedevelopers.com/jobs/ext/654221-data-automation-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Automation Engineer

- **Company:** BIG IMPACT TECH CORP
- **Location:** Washington, DC, United States
- **Experience:** Experienced
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Amazon Web Services, Microsoft Azure, Big Data, Code Review, Information Systems, Data Architecture, Information Engineering, Data Governance, Data Infrastructure, Data Integration, Extract Transform Load (ETL), Data Mart, Data Security, Data Systems, Data Warehousing, DevOps, Dimensional Modeling, Revision Control Systems, Python (Programming Language), Meta-Data Management, Release Management, Power BI, SQL Databases, Tableau (Software), Unstructured Data, Data Processing, Google Cloud, Enterprise Software Applications, Cloud Platform System, Azure Data Factory, Database Optimization, Apache Spark, Infrastructure as Code (IaC), Git, SC Clearance, Data Lakes, Pyspark, Infrastructure Automation Frameworks, Information Technology, Deployment Automation, AWS Glue, Machine Learning Operations, Restful APIs, Terraform, Data Pipelines, Serverless Computing, Databricks
- **Published:** June 26, 2026
- **Apply:** https://www.clearancejobs.com/jobs/8998960/data-automation-engineer

## About the Role

Bachelor's degree in Computer Science, Information Systems, Engineering, Mathematics, or a related field.
5+ years of experience in data engineering, data integration, or data platform development.
3+ years of hands-on experience with Databricks and Apache Spark.
Strong proficiency in Python, PySpark, and SQL.
Experience building and managing data pipelines in cloud environments such as Azure, AWS, or Google Cloud Platform.
Experience with Delta Lake, Lakehouse architecture, and large-scale data processing.
Knowledge of data warehousing concepts, dimensional modeling, and database optimization.
Experience with source control systems such as Git.
Familiarity with CI/CD tools and DevOps practices.
Strong analytical, troubleshooting, and problem-solving skills.
Excellent written and verbal communication skills.
Ability to obtain and maintain a Public Trust or Secret clearance, as required., Databricks Certified Data Engineer Associate or Professional certification.
Experience supporting federal government or Department of Homeland Security (DHS) programs.
Experience with Azure Data Factory, AWS Glue, or equivalent data integration tools.
Knowledge of data governance frameworks and metadata management tools.
Experience with Power BI, Tableau, or other business intelligence platforms.
Familiarity with Terraform, ARM templates, or other Infrastructure as Code tools.
Experience implementing data security controls in regulated environments.
Understanding of machine learning workflows and MLOps within Databricks.
Active security clearance preferred.

Technical Skills

Databricks Lakehouse Platform
Apache Spark / PySpark
Python
SQL
Delta Lake
Azure Databricks / AWS Databricks
Data Warehousing
ETL/ELT Development
Git/GitHub
CI/CD Pipelines
REST APIs
Data Governance & Security
Cloud Platforms (Azure, AWS, GCP)

## Description

We are seeking a highly skilled Databricks Engineer to support U.S. Coast Guard data modernization initiatives. The ideal candidate will design, develop, and maintain scalable data engineering solutions using the Databricks Lakehouse Platform to enable advanced analytics, reporting, and mission-critical decision-making. This role requires expertise in cloud-based data platforms, data integration, ETL/ELT processes, and secure handling of government data., Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and cloud-native services.
Build and optimize ETL/ELT workflows to ingest, transform, and load structured and unstructured data from multiple sources.
Implement and manage Delta Lake architectures to support reliable and high-performance data processing.
Collaborate with data architects, analysts, and business stakeholders to define data requirements and technical solutions.
Develop and maintain data models, data marts, and Lakehouse solutions that support Coast Guard operational and analytical needs.
Monitor, troubleshoot, and optimize Databricks workloads for performance, scalability, and cost efficiency.
Implement data quality, governance, lineage, and security controls in compliance with federal regulations and Coast Guard policies.
Automate deployment processes using Infrastructure as Code (IaC) and CI/CD pipelines.
Support cloud migration initiatives and modernization efforts involving legacy data systems.
Create technical documentation, standard operating procedures, and architecture diagrams.
Participate in code reviews, testing, and release management activities.
Provide operational support and resolve production issues in a timely manner.
Ensure compliance with cybersecurity requirements, including NIST, FISMA, and federal data protection standards.
Support data integration with enterprise systems, APIs, and external data sources.


## Related Videos

- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform)
- [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)

## Related Articles

- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)