> Markdown version of [/jobs/ext/1884923-data-engineer-lakehouse-analytics](https://www.wearedevelopers.com/jobs/ext/1884923-data-engineer-lakehouse-analytics). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - Lakehouse Analytics - **Company:** Silverthorne Advisory Group LLC - **Location:** Washington, DC, United States - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** JavaScript (Programming Language), Application Programming Interfaces (APIs), Artificial Intelligence, Airflow, Amazon Web Services, Data Analysis, Apache HTTP Server, Application Frameworks, Microsoft Azure, BigQuery, Cloud Computing, Information Systems, Databases, Continuous Integration, Data Governance, Extract Transform Load (ETL), Data Transformation, Data Security, Data Warehousing, Dimensional Modeling, Distributed Computing Environment, JSON, Python (Programming Language), Machine Learning, Meta-Data Management, Operational Databases, Oracle (Applications), Cloud Services, SAP (Applications), Scala (Programming Language), Software Construction, SQL Databases, Data Streaming, Workflow Management Systems, Data Processing, Google Cloud, Data Ingestion, Snowflake, Apache Spark, Git, Cloudformation, SC Clearance, Data Layers, Microsoft Fabric, Data Lakes, Infrastructure Automation Frameworks, Information Technology, Apache Kafka, Video Streaming, Software Coding, Terraform, Stream Processing, Data Pipelines, Docker, Amazon Redshift, Databricks - **Published:** August 1, 2026 - **Apply:** https://www.clearancejobs.com/jobs/9070951/data-engineer-lakehouse-analytics ## About the Role Minimum of three (3) years of experience building production data engineering solutions. - Secret clearance or higher required (can be sponsored). - Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent practical experience. - Strong proficiency and hands-on coding experience with languages/technologies such as Python, Spark, Scala, JavaScript/JSON, and SQL. - Experience designing and maintaining ETL/ELT pipelines. - Experience with Apache Spark or equivalent distributed data processing frameworks. - Experience working with modern Lakehouse technologies such as Databricks, Delta Lake, Apache Iceberg, or Apache Hudi. - Experience with workflow orchestration tools such as Apache Airflow. - Experience with cloud platforms including AWS, Azure, or Google Cloud Platform. - Strong understanding of data modeling, data lakes, data warehouses, and dimensional modeling. - Experience implementing data quality, validation, monitoring, and observability. - Experience with Git, CI/CD pipelines, and software engineering best practices. Desired Skills - Experience with enterprise ERP (Oracle or SAP) or other DoW financial management systems. - Experience with Snowflake, Databricks, Palantir Foundry, BigQuery, Amazon Redshift, or Microsoft Fabric. - Experience with streaming technologies such as Apache Kafka, Amazon Kinesis, or Azure Event Hubs. - Experience using dbt for analytics engineering and data transformation. - Experience with Infrastructure as Code tools such as Terraform or CloudFormation. - Experience with Docker and Kubernetes. - Familiarity with Lakehouse table formats including Delta Lake, Apache Iceberg, or Apache Hudi. - Experience supporting AI/ML data pipelines, feature stores, or vector databases. - Knowledge of data governance, metadata management, security, and regulatory compliance. - Experience supporting U.S. Department of War or other federal government modernization initiatives. - Strong communication skills with the ability to collaborate across engineering, analytics, and business teams. - Passion for building scalable, reusable, and secure data platforms that accelerate analytics and enterprise transformation. ## Description You will work across the full data lifecycle -- from ingesting and transforming data to modeling, governing, and delivering high-quality datasets for analytics, machine learning, and operational applications. You will help automate data workflows, improve data quality and reliability, and build secure, cloud-native data solutions that support enterprise modernization initiatives. The position will be hybrid and based in the Washington, DC metropolitan area. Onsite work locations are at the Navy Yard and Arlington, VA. Responsibilities: - Design, develop, and maintain scalable batch and streaming data pipelines. - Build and optimize ETL/ELT processes to ingest data from ERP systems, financial applications, databases, APIs, event streams, and third-party platforms. - Design, validate, test, and stabilize data layer integrations across enterprise financial systems, staging environments, Lakehouse platforms, analytics environments, and reporting solutions. - Develop and maintain modern Lakehouse architectures that support enterprise analytics, operational reporting, and AI/ML workloads. - Design and implement efficient data models that enable trusted, high-quality analytics and reporting. Ensure data quality, integrity, lineage, governance, and observability through automated validation, testing, and monitoring. - Optimize data processing performance, scalability, reliability, and cost across cloud-native environments. - Develop reusable frameworks and standardized components for data ingestion, transformation, orchestration, and integration. - Implement and maintain workflow orchestration using tools such as Apache Airflow or equivalent technologies. - Build and support real-time and near-real-time data processing using streaming technologies such as Kafka or cloud-native messaging services. - Collaborate with software engineers, solution architects, data scientists, financial management experts, and business stakeholders to deliver reliable enterprise data products. - Implement CI/CD pipelines and Infrastructure as Code to automate deployment and management of data platform components. - Monitor production interfaces and proactively identify opportunities to improve automation, performance, scalability, and operational resilience. - Evaluate emerging cloud, analytics, and AI technologies to enhance enterprise data capabilities and support ongoing modernization initiatives. - Prepare technical documentation, executive briefings, and client deliverables supporting project leadership and decision-making. ## Related Videos - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Tips and Tricks for Working with JSON](https://www.wearedevelopers.com/videos/1229-tips-and-tricks-for-working-with-json) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Best Coding Boot Camps in Germany](https://www.wearedevelopers.com/magazine/237-best-coding-boot-camps-in-germany)