> Markdown version of [/jobs/ext/278693-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/278693-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - **Company:** Attain - **Location:** United States (Remote available) - **Experience:** Expert - **Salary:** $125,000.0 - $150,000.0 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Airflow, Amazon Web Services, Data Analysis, Audit Trail, Big Data, Cloud Computing, Data Architecture, Information Engineering, Data Governance, Data Retention, Data Systems, Software Debugging, Disaster Recovery, Python (Programming Language), Machine Learning, Uptime, Operational Databases, Software Architecture, SQL Databases, Macros, System Availability, Snowflake, Information Technology, Apache Flink, Performance Monitor, AWS Data Analytics, Real Time Data, Apache Kafka, Data Management, Data Pipelines, Legacy Systems - **Published:** May 21, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=6100b22992f71db6 ## About the Role Do you have experience in SQL?, Do you have a Master's degree?, * Deep hands-on experience with Snowflake, Airflow (MWAA), AWS, Python, and SQL in production environments. * Strong ability to design, build, debug, and operate data pipelines at scale. * Experience owning production systems with uptime and reliability expectations. * Proven ability to lead incident response and drive postmortems that result in real system improvements. * Solid data architecture and system design skills, including disaster recovery, high availability, and upgrades without downtime. * Experience implementing observability and data quality controls across complex data systems. * Demonstrated success defining and enforcing engineering standards adopted by other teams. * Strong cost management experience, including running optimization efforts and forecasting capacity. * Experience leading technical initiatives across teams without direct authority. * Experience implementing governance and compliance controls in regulated or high-risk environments. * Ability to mentor through standards, examples, and shared practices rather than formal people management. * 8+ years of data engineering experience, with several years owning critical platforms or services., * Experience leading AI and LLM adoption in data platforms, including validation of AI-generated pipelines and guardrail design. * Advanced Airflow experience, including reusable DAG patterns, scheduler tuning, and failure prevention. * Experience with dbt at scale, including testing standards, macros, and deployment practices. * Experience with OpenLineage, schemachange, or similar tools for lineage and schema management. * Experience leading platform migrations or modernization efforts with minimal disruption. * Experience streaming or near-real-time data systems such as Kafka, Kinesis, or Flink. * Experience running resilience testing such as failure injection or disaster recovery drills. * Cloud certifications such as AWS Certified Solutions Architect, AWS Data Analytics Specialty, or Snowflake SnowPro. * Bachelor's or master's degree in computer science, Data Engineering, or equivalent practical experience ## Description We're looking for a Lead Data Engineer to own the reliability and operational excellence of our data platform. You'll work hands-on across Snowflake, Airflow, AWS, Python, and SQL, building and operating systems that support analytics, machine learning, and business decisions., Build, maintain, and operate data pipelines and curated data products across Snowflake, Airflow (MWAA), AWS, Python, and SQL. * Optimize Snowflake architecture at scale. Design warehouse strategies, implement clustering and materialized views, enforce resource monitors, control costs, and tune performance across large datasets. * Implement observability and data quality controls. Build monitoring for freshness, volume, schema, distribution, and lineage. Define data quality SLOs and ensure teams can see issues before users do. * Support and operate production data systems. Troubleshoot failures, respond to alerts, debug pipeline issues, and improve system behavior based on real incidents. * Own SLAs, uptime, and service commitments for data platforms and critical services. Monitor performance, track reliability metrics, and make sure availability and freshness targets are met. * Lead major incident responses and postmortems. Coordinate resolution during critical outages, drive root cause analysis, and implement corrective actions that prevent recurrence. * Architect data platforms across dev, test, and production. Design for disaster recovery, upgrades without downtime, automation that recovers from failure, and availability that supports business continuity. * Define and enforce data platform standards. Establish orchestration patterns, DAG anti-patterns, deployment practices, observability standards, data quality patterns, and operational runbooks used across the organization. * Lead cost optimization programs. Identify savings across Snowflake (compute/storage) and AWS resources, forecast capacity needs, and balance cost against reliability and performance goals. * Lead AI and LLM adoption and governance. Build and validate AI-assisted pipelines, define approved use cases and guardrails, and ensure AI-generated output meets reliability and quality expectations. * Make architectural decisions with an organization-wide impact. Evaluate build versus buy, set migration strategies for legacy systems, define data contracts and API standards, and balance innovation with operational stability. * Lead technical planning and remove blockers. Coordinate with Cloud, Security, Analytics, Data Science, and Product teams to resolve dependencies and keep delivery moving. * Prevent knowledge silos. Write documentation, maintain runbooks, record architectural decisions, and create shared practices that let engineers operate independently and onboard quickly. * Design and enforce data governance controls for regulated environments. Implement PII handling, access controls, audit logging, data retention policies, and compliance validation for SOC 2, GDPR, HIPAA, or similar frameworks., Attain Finance Supports Equal Employment Opportunity. CURO (dba Cash Money®, LendDirect®, Heights Finance, Southern Finance, Covington Credit, Quick Credit, and First Heritage Credit) is committed to a policy of providing equal employment opportunity to all qualified employees and applicants. This commitment is reflected in all aspects of our daily operations. We do not discriminate on the basis of race, color, sex, religion, national origin, marital status, age, disability, veteran status, or genetic information in any personnel practice, including recruitment, hiring, training, compensation, promotion, and discipline. Additionally, we do not discriminate based on any other characteristic protected by applicable state/provincial or local law where a particular employee works. In addition, it is the policy of Attain Finance to provide reasonable accommodation to qualified employees who have protected disabilities to the extent required by federal law and any state/provincial law where a particular employee works. Notice to Attain Finance job applicants:Attain Finance will never ask for your personal banking information, transfer of funds, a credit card, or for you to purchase any equipment to process a job application or for training. Authorized Attain Finance representatives' email addresses will end in @attainfinace.com, @curo.com, @first-heritage.com, @heightsfinance.com, and @cashmoney.ca. EEO Statement: Attain Finance Supports Equal Employment Opportunity. CURO (dba Cash Money®, LendDirect®, Heights Finance, Southern Finance, Covington Credit, Quick Credit, and First Heritage Credit) is committed to a policy of providing equal employment opportunity to all qualified employees and applicants. This commitment is reflected in all aspects of our daily operations. We do not discriminate on the basis of race, color, sex, religion, national origin, marital status, age, disability, veteran status, or genetic information in any personnel practice, including recruitment, hiring, training, compensation, promotion, and discipline. Additionally, we do not discriminate based on any other characteristic protected by applicable state/provincial or local law where a particular employee works. In addition, it is the policy of Attain Finance to provide reasonable accommodation to qualified employees who have protected disabilities to the extent required by federal law and any state/provincial law where a particular employee works. Notice to Attain Finance job applicants: Attain Finance will never ask for your personal banking information, transfer of funds, a credit card, or for you to purchase any equipment to process a job application or for training. Authorized Attain Finance representatives' email addresses will end in @attainfinace.com, @curo.com, @first-heritage.com, @heightsfinance.com, and @cashmoney.ca. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [DevOps at Netflix](https://www.wearedevelopers.com/videos/270-devops-at-netflix) - [Crafting Custom Frameworks with Rust: A Deep Dive into Procedural Macros](https://www.wearedevelopers.com/videos/849-crafting-custom-frameworks-with-rust-a-deep-dive-into-procedural-macros) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) - [Leading with Reliability: Applying SRE Principles to Build Stronger Engineering Organizations](https://www.wearedevelopers.com/videos/100185-leading-with-reliability-applying-sre-principles-to-build-stronger-engineering-organizations) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud)