Lead Data Engineer

Attain
United States
3 months ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Compensation
$125,000.0 - $150,000.0
Working hours
Regular working hours
Job source

Tech stack

Artificial Intelligence Airflow Amazon Web Services Data Analysis Audit Trail Big Data Cloud Computing Data Architecture Information Engineering Data Governance Data Retention Data Systems
+20 more
Software Debugging Disaster Recovery Python (Programming Language) Machine Learning Uptime Operational Databases Software Architecture SQL Databases Macros System Availability Snowflake Information Technology Apache Flink Performance Monitor AWS Data Analytics Real Time Data Apache Kafka Data Management Data Pipelines Legacy Systems

Job description

We’re looking for a Lead Data Engineer to own the reliability and operational excellence of our data platform. You’ll work hands-on across Snowflake, Airflow, AWS, Python, and SQL, building and operating systems that support analytics, machine learning, and business decisions., Build, maintain, and operate data pipelines and curated data products across Snowflake, Airflow (MWAA), AWS, Python, and SQL.

  • Optimize Snowflake architecture at scale. Design warehouse strategies, implement clustering and materialized views, enforce resource monitors, control costs, and tune performance across large datasets.
  • Implement observability and data quality controls. Build monitoring for freshness, volume, schema, distribution, and lineage. Define data quality SLOs and ensure teams can see issues before users do.
  • Support and operate production data systems. Troubleshoot failures, respond to alerts, debug pipeline issues, and improve system behavior based on real incidents.
  • Own SLAs, uptime, and service commitments for data platforms and critical services. Monitor performance, track reliability metrics, and make sure availability and freshness targets are met.
  • Lead major incident responses and postmortems. Coordinate resolution during critical outages, drive root cause analysis, and implement corrective actions that prevent recurrence.
  • Architect data platforms across dev, test, and production. Design for disaster recovery, upgrades without downtime, automation that recovers from failure, and availability that supports business continuity.
  • Define and enforce data platform standards. Establish orchestration patterns, DAG anti-patterns, deployment practices, observability standards, data quality patterns, and operational runbooks used across the organization.
  • Lead cost optimization programs. Identify savings across Snowflake (compute/storage) and AWS resources, forecast capacity needs, and balance cost against reliability and performance goals.
  • Lead AI and LLM adoption and governance. Build and validate AI-assisted pipelines, define approved use cases and guardrails, and ensure AI-generated output meets reliability and quality expectations.
  • Make architectural decisions with an organization-wide impact. Evaluate build versus buy, set migration strategies for legacy systems, define data contracts and API standards, and balance innovation with operational stability.
  • Lead technical planning and remove blockers. Coordinate with Cloud, Security, Analytics, Data Science, and Product teams to resolve dependencies and keep delivery moving.
  • Prevent knowledge silos. Write documentation, maintain runbooks, record architectural decisions, and create shared practices that let engineers operate independently and onboard quickly.
  • Design and enforce data governance controls for regulated environments. Implement PII handling, access controls, audit logging, data retention policies, and compliance validation for SOC 2, GDPR, HIPAA, or similar frameworks., Attain Finance Supports Equal Employment Opportunity. CURO (dba Cash Money®, LendDirect®, Heights Finance, Southern Finance, Covington Credit, Quick Credit, and First Heritage Credit) is committed to a policy of providing equal employment opportunity to all qualified employees and applicants. This commitment is reflected in all aspects of our daily operations. We do not discriminate on the basis of race, color, sex, religion, national origin, marital status, age, disability, veteran status, or genetic information in any personnel practice, including recruitment, hiring, training, compensation, promotion, and discipline. Additionally, we do not discriminate based on any other characteristic protected by applicable state/provincial or local law where a particular employee works. In addition, it is the policy of Attain Finance to provide reasonable accommodation to qualified employees who have protected disabilities to the extent required by federal law and any state/provincial law where a particular employee works.

Notice to Attain Finance job applicants:Attain Finance will never ask for your personal banking information, transfer of funds, a credit card, or for you to purchase any equipment to process a job application or for training. Authorized Attain Finance representatives’ email addresses will end in @attainfinace.com, @curo.com, @first-heritage.com, @heightsfinance.com, and @cashmoney.ca. EEO Statement: Attain Finance Supports Equal Employment Opportunity. CURO (dba Cash Money®, LendDirect®, Heights Finance, Southern Finance, Covington Credit, Quick Credit, and First Heritage Credit) is committed to a policy of providing equal employment opportunity to all qualified employees and applicants. This commitment is reflected in all aspects of our daily operations. We do not discriminate on the basis of race, color, sex, religion, national origin, marital status, age, disability, veteran status, or genetic information in any personnel practice, including recruitment, hiring, training, compensation, promotion, and discipline. Additionally, we do not discriminate based on any other characteristic protected by applicable state/provincial or local law where a particular employee works. In addition, it is the policy of Attain Finance to provide reasonable accommodation to qualified employees who have protected disabilities to the extent required by federal law and any state/provincial law where a particular employee works. Notice to Attain Finance job applicants: Attain Finance will never ask for your personal banking information, transfer of funds, a credit card, or for you to purchase any equipment to process a job application or for training. Authorized Attain Finance representatives’ email addresses will end in @attainfinace.com, @curo.com, @first-heritage.com, @heightsfinance.com, and @cashmoney.ca.

Requirements

Do you have experience in SQL?, Do you have a Master’s degree?, * Deep hands-on experience with Snowflake, Airflow (MWAA), AWS, Python, and SQL in production environments.

  • Strong ability to design, build, debug, and operate data pipelines at scale.
  • Experience owning production systems with uptime and reliability expectations.
  • Proven ability to lead incident response and drive postmortems that result in real system improvements.
  • Solid data architecture and system design skills, including disaster recovery, high availability, and upgrades without downtime.
  • Experience implementing observability and data quality controls across complex data systems.
  • Demonstrated success defining and enforcing engineering standards adopted by other teams.
  • Strong cost management experience, including running optimization efforts and forecasting capacity.
  • Experience leading technical initiatives across teams without direct authority.
  • Experience implementing governance and compliance controls in regulated or high-risk environments.
  • Ability to mentor through standards, examples, and shared practices rather than formal people management.
  • 8+ years of data engineering experience, with several years owning critical platforms or services., * Experience leading AI and LLM adoption in data platforms, including validation of AI-generated pipelines and guardrail design.
  • Advanced Airflow experience, including reusable DAG patterns, scheduler tuning, and failure prevention.
  • Experience with dbt at scale, including testing standards, macros, and deployment practices.
  • Experience with OpenLineage, schemachange, or similar tools for lineage and schema management.
  • Experience leading platform migrations or modernization efforts with minimal disruption.
  • Experience streaming or near-real-time data systems such as Kafka, Kinesis, or Flink.
  • Experience running resilience testing such as failure injection or disaster recovery drills.
  • Cloud certifications such as AWS Certified Solutions Architect, AWS Data Analytics Specialty, or Snowflake SnowPro.
  • Bachelor’s or master’s degree in computer science, Data Engineering, or equivalent practical experience

Benefits & conditions

3.73.7 out of 5 stars United States Remote $125,000 - $150,000 a year - Full-time, Pulled from the full job description

  • 401(k)
  • Health insurance
  • Paid time off
  • Vision insurance
  • Dental insurance
  • Life insurance, The base salary range represents the low and high end of the anticipated salary range for this position based on the U.S. average. The actual base salary offered for this full-time position will be determined by various factors, including but not limited to, location, skills, knowledge, competencies, and experience.

All full-time salaried employees are eligible for the following benefits, starting on day one: Flexible Paid Time Off Program, Medical, Dental, Vision, Life Insurance, Disability, and other voluntary coverages. You will also be eligible to participate in our 401k program, starting on the first of the month following 30 days of employment with a company match.

This employer participates in E-Verify for US-based hires.

AttainFinance

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on indeed.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · WWC Europe 2026

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

3:03 min

Exploring declarative and procedural macro subtypes in Rust environments

Mykhailo Maidan · LIVE

2:22 min

Leveraging unique cultural backgrounds in engineering design

Ixchel Ruiz · LIVE

2:57 min

Core technical practices for robust data engineering

Sandhya Menon Sandhya Menon · WWC Europe 2026

1:59 min

Navigating the disadvantages and maintenance costs of procedural macros

Mykhailo Maidan · LIVE

Videos

See all

Related articles

See all