> Markdown version of [/jobs/ext/2012135-data-engineer-managing-consultant](https://www.wearedevelopers.com/jobs/ext/2012135-data-engineer-managing-consultant). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - Managing Consultant - **Company:** Guidehouse Inc. - **Location:** McLean, VA, United States - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Amazon Web Services, Amazon S3, Audit Trail, Big Data, Cloud Computing, Continuous Integration, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Masking, Data Security, Data Systems, Github, Identity and Access Management, Python (Programming Language), Meta-Data Management, Performance Tuning, Role-Based Access Control, Requirements Traceability, Software Deployment, SQL Databases, Data Streaming, Real Time Systems, Data Ingestion, Amazon Virtual Private Cloud (VPC), Servicebus, Data Layers, Data Lakes, Pyspark, Apache Kafka, Data Management, Terraform, Data Pipelines, Devsecops, Databricks - **Published:** August 10, 2026 - **Apply:** https://www.dice.com/job-detail/992358ce-af7d-49cc-ba49-efb199935cdf ## About the Role * Bachelor?s degree is required * Minimum EIGHT (8) years of experience in data engineering within cloud-based environments, * Minimum FOUR (4) years of hands-on Databricks experience designing scalable data pipelines. * Advanced proficiency in Python, PySpark, and SQL for large-scale data processing and pipeline development in lakehouse architectures. * Demonstrated experience building and optimizing data pipelines leveraging Delta Lake, medallion architecture, and modern data ingestion frameworks (batch and streaming). * Strong experience with AWS data platforms and services (e.g., S3, IAM, VPC, Glue, streaming frameworks) and integration with enterprise data ecosystems. * Experience delivering production-ready data solutions incorporating metadata management, lineage, data quality, and observability frameworks. * Familiarity with DevSecOps and CI/CD pipeline implementation, including automation, testing, and deployment within cloud data environments. * Knowledge of data governance, security, and compliance requirements within regulated environments, including FISMA and FedRAMP High What Would Be Nice to Have: * Experience supporting federal data platforms or large-scale enterprise data modernization efforts, particularly within IRS, Treasury, or similar regulated environments. * Hands-on experience with Databricks Unity Catalog, Delta Sharing, Genie, and Clean Rooms for governed data access and collaboration. * Experience implementing streaming and near real-time data pipelines using Kafka, Kinesis, EventBridge, or similar technologies. * Familiarity with Informatica EDC/Axon or enterprise metadata/catalog tooling. * Experience with performance optimization techniques (Photon, Z-ordering, liquid clustering) to improve large-scale data workloads. * Exposure to CI/CD automation using Terraform, GitHub Actions, and Databricks Asset Bundles for infrastructure and pipeline deployment. * Advanced cloud or Databricks certifications (AWS, Databricks) in good standing. ## Description * Design, build, and optimize scalable, production-grade data ingestion, transformation, and analytics-ready pipelines using Databricks (Delta Lake, Delta Live Tables, Auto Loader) and AWS services, enabling trusted, timely data access across enterprise use cases. * Engineer standardized, repeatable data pipelines supporting batch and near real-time processing, integrating legacy data sources and modern cloud-native services to advance enterprise data availability and eliminate data access gaps. * Execute full delivery lifecycle by supporting intake, discovery, source profiling, and technical design, while maintaining requirements traceability and aligning solutions to Architecture Review Board (ARB) and governance expectations. * Implement and maintain governed data pipelines with embedded metadata, lineage, and data quality controls, ensuring pipelines meet defined technical, security, and documentation requirements before production deployment. * Develop and operationalize data engineering frameworks that incorporate observability, monitoring, alerting, and resilient error handling to maintain platform stability and support ?99.9% availability targets. * Partner with platform engineering and cloud operations teams to integrate pipelines with AWS services (S3, Glue, Kafka/Kinesis, APIs), enabling secure, scalable data movement and cross-platform interoperability. * Enable governed analytics and self-service data consumption through curated datasets, semantic layers, and SQL warehouse integration, supporting enterprise reporting, dashboards, and advanced analytics use cases. * Apply security and compliance controls aligned to IRS cybersecurity policies, including RBAC/ABAC, data masking, encryption, and audit logging to protect sensitive data and maintain regulatory compliance. ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)