Data Engineer - Mid

• Design, build, and optimize scalable, production-grade data ingestion, transformation, and analytics-ready pipelines using Databricks (Delta Lake, Delta Live Tables, Auto Loader) and AWS services, enabling trusted, timely data access across enterprise use cases.
• Engineer standardized, repeatable data pipelines supporting batch and near real-time processing, integrating legacy data sources and modern cloud-native services to advance enterprise data availability and eliminate data access gaps.
• Execute full delivery lifecycle by supporting intake, discovery, source profiling, and technical design, while maintaining requirements traceability and aligning solutions to Architecture Review Board (ARB) and governance expectations.
• Implement and maintain governed data pipelines with embedded metadata, lineage, and data quality controls, ensuring pipelines meet defined technical, security, and documentation requirements before production deployment.
• Develop and operationalize data engineering frameworks that incorporate observability, monitoring, alerting, and resilient error handling to maintain platform stability and support 99.9% availability targets.
• Partner with platform engineering and cloud operations teams to integrate pipelines with AWS services (S3, Glue, Kafka/Kinesis, APIs), enabling secure, scalable data movement and cross-platform interoperability.
• Enable governed analytics and self-service data consumption through curated datasets, semantic layers, and SQL warehouse integration, supporting enterprise reporting, dashboards, and advanced analytics use cases.
• Apply security and compliance controls aligned to IRS cybersecurity policies, including RBAC/ABAC, data masking, encryption, and audit logging to protect sensitive data and maintain regulatory compliance.

• Bachelor’s degree is required
• EIGHT (8) or more years of experience in data engineering within cloud-based environments,
• FOUR (4) or more years of hands-on Databricks experience designing scalable data pipelines.
• Experience in Python, PySpark, and SQL for large-scale data processing and pipeline development in lakehouse architectures.
• Experience building and optimizing data pipelines leveraging Delta Lake, medallion architecture, and modern data ingestion frameworks (batch and streaming).
• Experience with AWS data platforms and services (e.g., S3, IAM, VPC, Glue, streaming frameworks) and integration with enterprise data ecosystems.
• Experience delivering production-ready data solutions incorporating metadata management, lineage, data quality, and observability frameworks.

What Would Be Nice to Have:

• Familiarity with DevSecOps and CI/CD pipeline implementation, including automation, testing, and deployment within cloud data environments.
• Knowledge of data governance, security, and compliance requirements within regulated environments, including FISMA and FedRAMP High
• Experience supporting federal data platforms or large-scale enterprise data modernization efforts, particularly within IRS, Treasury, or similar regulated environments.
• Hands-on experience with Databricks Unity Catalog, Delta Sharing, Genie, and Clean Rooms for governed data access and collaboration.
• Experience implementing streaming and near real-time data pipelines using Kafka, Kinesis, EventBridge, or similar technologies.
• Familiarity with Informatica EDC/Axon or enterprise metadata/catalog tooling.
• Experience with performance optimization techniques (Photon, Z-ordering, liquid clustering) to improve large-scale data workloads.
• Exposure to CI/CD automation using Terraform, GitHub Actions, and Databricks Asset Bundles for infrastructure and pipeline deployment.
• Advanced cloud or Databricks certifications (AWS, Databricks) in good standing.

Pulled from the full job description

• Referral program
• Tuition reimbursement
• AD&D insurance
• Parental leave
• 401(k)
• Health insurance
• Vision insurance, Guidehouse offers a comprehensive, total rewards package that includes competitive compensation and a flexible benefits package that reflects our commitment to creating a diverse and supportive workplace.

Benefits include:

• Medical, Rx, Dental & Vision Insurance
• Personal and Family Sick Time & Company Paid Holidays
• Parental Leave
• 401(k) Retirement Plan
• Group Term Life and Travel Assistance
• Voluntary Life and AD&D Insurance
• Health Savings Account, Health Care & Dependent Care Flexible Spending Accounts
• Transit and Parking Commuter Benefits
• Short-Term & Long-Term Disability
• Tuition Reimbursement, Personal Development, Certifications & Learning Opportunities
• Employee Referral Program
• Corporate Sponsored Events & Community Outreach
• Care.com annual membership
• Employee Assistance Program
• Supplemental Benefits via Corestream (Critical Care, Hospital Indemnity, Accident Insurance, Legal Assistance and ID theft protection, etc.)
• Position may be eligible for a discretionary variable incentive bonus

About Guidehouse

Guidehouse is an Equal Opportunity Employer-Protected Veterans, Individuals with Disabilities or any other basis protected by law, ordinance, or regulation.

Guidehouse will consider for employment qualified applicants with criminal histories in a manner consistent with the requirements of applicable law or ordinance including the Fair Chance Ordinance of Los Angeles and San Francisco.