> Markdown version of [/jobs/ext/3079513-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/3079513-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer

- **Company:** Capital Technology Group, LLC
- **Location:** United States (Remote available)
- **Experience:** Expert
- **Salary:** $150,000.0 - $200,000.0
- **Contract:** Permanent contract
- **Skills:** Agile Methodology, Artificial Intelligence, Airflow, Amazon Web Services, Amazon S3, Data Analysis, Apache HTTP Server, Cloud Computing, Program Optimization, Code Review, Data Architecture, Information Engineering, Data Integration, Extract Transform Load (ETL), Data Transformation, Data Systems, Data Warehousing, Text Processing, Github, Apache Hive, Integrated Development Environments, Python (Programming Language), PostgreSQL, NoSQL, Performance Tuning, Search Technologies, Software Engineering, SQL Databases, Systems Integration, Unstructured Data, Workflow Management Systems, Enterprise Data Management, Parquet, Large Language Models, Apache Spark, AWS Lambda, Cloudformation, Amazon Relational Database Service, Data Lakes, Pyspark, Semi-structured Data, Information Technology, AWS Glue, Data Management, Data Lakehouse, Data Pipelines, Amazon Elastic Mapreduce (EMR)
- **Published:** September 25, 2026
- **Apply:** https://www.builtincolorado.com/job/lead-data-engineer/11363211?handler=ApplyRedirect

## About the Role

* A strategic data engineer who enjoys designing sophisticated systems and solving challenging problems
 * Strong experience in modern cloud-based solution design
 * Comfortable balancing business needs with technical constraints and long-term strategy
 * A strong communicator
 * Collaborative, proactive, and comfortable navigating ambiguity, * Bachelor's degree in Computer Science, Engineering, or a related technical field
 * 15+ years of professional experience in data engineering, data architecture, or related fields
 * Strong hands-on experience with:
 + Apache Spark (PySpark) required, Python, SQL (PostgreSQL), and dbt for large-scale data engineering, ETL/ELT development, data transformation, and data modeling.
 + AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), AWS Lambda, Amazon S3, Amazon RDS.
 + Developing scalable data pipelines, workflow orchestration, and data integration solutions across enterprise environments.
 + Working with modern data lake technologies and data formats such as Parquet and Iceberg.
 + Designing and optimizing solutions using relational and NoSQL databases
 + Building reliable, high-performance data platforms through performance tuning, system optimization, and enterprise-scale ETL/ELT architectures.
 * Strong analytical and problem-solving skills
 * Experience working in Agile, iterative software development environments
 * Ability to quickly learn and apply new technologies and domain knowledge
 * Excellent written and verbal communication skills, with the ability to explain complex topics to diverse audiences

Nice to Have

 * Experience supporting analytics, data engineering, or modernization initiatives for financial regulators, capital markets, or other highly regulated environments is a plus.
 * Experience with Apache Iceberg and modern data lakehouse architectures.
 * Experience working with unstructured data processing, including document/text processing, embeddings, vector search, and LLM-based data solutions.
 * Exposure to integrating LLMs and generative AI capabilities into enterprise data pipelines and platforms.
 * Experience designing data architectures that support both structured and unstructured data at scale.

## Description

Expert/Leader
Leads the design, development, and maintenance of scalable AWS data platforms, pipelines, ETL/ELT workflows, data models, and orchestration systems. Uses Python, PySpark, SQL, dbt, AWS services, modern lakehouse technologies, and CI/CD practices to deliver secure federal data solutions. Responsibilities include platform modernization, performance optimization, AI-enabled data integration, compliance support, technical mentoring, architecture discussions, and cross-functional Agile collaboration.
The summary above was generated by AI, CTG is seeking a Lead Data Engineer to design, build, and maintain scalable, efficient data pipelines and systems following modern data engineering best practices. The Lead Data Engineer will partner with other Data Engineers to evaluate and prototype new tools and technologies, assess associated risks and benefits, and deliver exceptional value to our clients.
You Will Get To

 * Design, build, and maintain scalable data pipelines, ETL/ELT workflows, and data models using Python, Apache Spark (PySpark), SQL (PostgreSQL), and AWS Glue.
 * Develop and optimize AWS-native data platforms leveraging AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), Amazon S3, RDS, and CloudWatch.
 * Build high-performance ingestion, transformation, and orchestration workflows for structured and semi-structured data using Apache Iceberg and Parquet.
 * Design and optimize analytical data platforms using Amazon Athena, Trino, Hive, OpenSearch, and enterprise data catalog technologies.
 * Build AI-enabled data solutions using Amazon Bedrock, RAG pipelines, and vector search technologies including Amazon S3 Vectors and OpenSearch vector indexes.
 * Develop cloud infrastructure using CloudFormation (Infrastructure as Code), GitHub and enterprise CI/CD pipelines.
 * Improve the reliability, scalability, performance, and maintainability of enterprise data platforms through monitoring, troubleshooting, automation, and continuous optimization.
 * Support mission-critical analytics and reporting solutions within large-scale AWS-based federal data environments, implementing solutions that comply with FedRAMP and NIST SP 800-53 security controls.
 * Mentor junior engineers through technical guidance, architecture discussions, and code reviews while promoting engineering best practices.
 * Collaborate with cross-functional teams in an Agile environment to define requirements, deliver high-quality data solutions, and communicate technical concepts effectively to technical and non-technical stakeholders., Leads end-to-end data engineering and analytics platform development, including Power BI dashboards, semantic models, cloud data pipelines, governance, quality monitoring, CI/CD, and legacy workload modernization. Partners with stakeholders to define scalable solutions, validates KPIs, troubleshoots production issues, establishes engineering standards, and mentors team members.
Top Skills: Amazon AthenaAmazon S3Aws GlueAws LambdaAzure Data FactoryAzure DevopsAzure Key VaultCi/CdData Vault 2.0DatabricksDaxDbtDelta LakeDomoGitMicrosoft FabricMicrosoft PurviewPower BIPythonSnowflakeSparkSQLSsisTableau
Nexcess

Lead Data Engineer

3 Days Ago
Remote
United States
Senior level
Senior level
Cloud * Information Technology * Infrastructure as a Service (IaaS)
Leads and develops the Data Engineering team while remaining hands-on with scalable pipelines, ETL/ELT processes, data warehouses, integrations, and infrastructure. Establishes architecture, reliability, quality, and performance standards; guides technical strategy; improves automation and monitoring; and partners with Finance and business stakeholders to translate data needs into practical solutions.
Top Skills: Apache AirflowApache KafkaSparkAWSCi/CdDbtGoogle Cloud PlatformInfrastructure As CodeJavaAzurePythonScalaSQL

What you need to know about the Colorado Tech Scene

With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.

Key Facts About Colorado Tech

 * Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
 * Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
 * Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
 * Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
 * Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
 * Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next)
- [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment)
- [Bringing AI Model Testing and Prompt Management to Your Codebase with GitHub Models](https://www.wearedevelopers.com/videos/1536-bringing-ai-model-testing-and-prompt-management-to-your-codebase-with-github-models)

## Related Articles

- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)