Lead Data Engineer

PySpark Coaching MuleSoft Indexing AWS Glue Marketing Annuities Scheduling Leadership Automation Governance Data Lakes PostgreSQL Warehousing Scalability Wholesaling Star Schema AWS Kinesis Fundraising Claude Code Data Quality Data Privacy Data Security Data Modeling MicroStrategy AWS SageMaker Claude Cowork Data Pipelines Apache Airflow Tax Accounting GitHub Copilot Apache Parquet Apache Iceberg Responsible AI Test Automation Data Management Social Security Data Processing Amazon DynamoDB Amazon Redshift Computer Science Microsoft Access Data Integration Data Engineering Virtual Training Snowflake Schema Agile Methodology Programming Tools Unstructured Data Amazon CloudWatch Database Analysis Advanced Analytics AWS CloudFormation Query Optimization Data Preprocessing Workflow Management Process Improvement Systems Engineering Amazon Web Services Data Infrastructure Technical Leadership Solution Architecture Business Intelligence Employment Applications CA Workload Automation Ae Change Data Capture (CDC) SQL (Programming Language) Git (Version Control System) Employee Assistance Programs Extract Transform Load (ETL) Amazon Elastic Compute Cloud Health And Wellness Coaching Python (Programming Language) Systems Development Life Cycle Customer Relationship Management Troubleshooting (Problem Solving) Enterprise Application Integration Application Programming Interface (API) Tableau (Business Intelligence Software), The Lead Data Engineer provides technical leadership for Lincoln’s Data & Analytics platform on AWS. This role designs, builds, and maintains scalable data pipelines and optimizes data systems, with deep experience in Aurora PostgreSQL and Amazon Redshift and strong knowledge of AWS services including Glue, Lambda, S3, Athena, and EC2. The role leads ETL development within an Agile team and partners with business, IT, and analytics stakeholders to deliver secure, governed, high-quality data that enables advanced analytics and reporting across Lincoln.

What you’ll be doing

• Design, build, and maintain robust batch and streaming data pipelines (including change data capture, for example AWS DMS) to support business intelligence and analytics.
• Integrate structured and unstructured data from internal and external sources.
• Develop ETL workflows using AWS Glue, Lambda, custom Python, PySpark, and SQL-based scripts.
• Analyze database workloads and optimize queries, indexing, partitioning, and storage in Aurora PostgreSQL and Amazon Redshift for integrity, performance, and cost.
• Build S3-based data lake and warehouse layers (Glue Data Catalog, Athena, Parquet) with fine-grained access controls.
• Own end-to-end data flows from source to consumption; traces and resolve issues across the full chain.
• Apply data modeling standards (normal form, star, snowflake) and create technical designs from business requirements.
• Orchestrate and schedule workflows (Stonebranch, Step Functions, Airflow) and monitor them with CloudWatch and automated data quality checks.
• Ensure data security, governance, and compliance; monitor, troubleshoot, and optimize data workflows.
• Integrate with upstream and downstream applications and collaborate with data scientists, analysts, and business stakeholders to deliver high-quality datasets; help shape data infrastructure strategy on AWS.
• Set team standards, review code, mentor engineers, and own production support and process improvement., This position may be subject to Lincoln’s Political Contribution Policy. An offer of employment may be contingent upon disclosing to Lincoln the details of certain political contributions. Lincoln may decline to extend an offer or terminate employment for this role if it determines political contributions made could have an adverse impact on Lincoln’s current or future business interests, misrepresentations were made, or for failure to fully disclose applicable political contributions and or fundraising activities.

Any unsolicited resumes or candidate profiles submitted through our web site or to personal e-mail accounts of employees of Lincoln Financial are considered property of Lincoln Financial and are not subject to payment of agency fees.

Lincoln Financial (“Lincoln” or “the Company”) is an Equal Opportunity employer and, as such, is committed in policy and practice to recruit, hire, compensate, train and promote, in all job classifications, without regard to race, color, religion, sex, age, national origin or disability. Opportunities throughout Lincoln are available to employees and applicants are evaluated on the basis of job qualifications. If you are a person with a disability that impedes your ability to express your interest for a position through our online application process, or require TTY/TDD assistance, contact us by calling (866) 922-6543.

This Employer Participates in E-Verify. See the E-Verify notices.

Must-haves:

• Bachelor’s degree (or equivalent) in Computer Science, Information Systems, Engineering, or a related field.
• 7+ years in AWS data engineering or a similar role.
• Strong expertise in Aurora PostgreSQL and Amazon Redshift, including database analysis, query optimization, advanced SQL, execution plans, indexing, distribution and sort keys, and workload tuning.
• Holistic view of end-to-end data processing, from source to consumption, including dependencies, bottlenecks, and downstream impact.
• Data and application integration experience with upstream source systems and downstream consumers (APIs, files, messaging, and reporting layers).
• Hands-on AWS experience: Glue, Lambda, S3, Athena, EC2, and IAM.
• Python / PySpark and SQL development for ETL and data integration in Agile, SDLC-based environments.
• Data modeling, data governance, data quality, and data security practices.
• DevOps and scheduling: Git, CI/CD pipelines, automated testing, CloudFormation or similar, and Stonebranch scheduling knowledge (or a similar scheduler such as Autosys).
• Hands-on use of approved AI-assisted development tools (GitHub Copilot or Claude Code), with validation of AI output and adherence to Lincoln data privacy and responsible AI requirements.
• Proven ability to mentor engineers and lead delivery of complex projects.

Nice-to-haves:

• AWS Certified Data Engineer - Associate, Solutions Architect, or an AWS specialty certification.
• EMR, Kinesis, Lake Formation, DynamoDB, and Apache Iceberg experience.
• Microsoft 365 Copilot and Claude Cowork experience; coaching teams on safe AI use.
• RAG data preparation, embeddings, vector stores, and AWS AI services (Bedrock, SageMaker, Amazon Q).
• Salesforce CRM and MuleSoft (API-led integration) as upstream or downstream systems; MicroStrategy and Tableau reporting and BI background.
• Financial data experience is a plus, especially Investment products like Mutual Funds, Alternative Funds and Fund Data management.

Core Competencies:

• Technical leadership and AWS data engineering expertise.
• Collaboration and Agile delivery with business, IT, and analytics teams.
• Analytical depth, operational ownership, and a data-centric mindset (quality, governance, security).
• AI fluency: responsible, validated use of AI tools.

• Clearly defined career tracks and job levels, along with associated behaviors for each of Lincoln’s core values and leadership attributes
• Leadership development and virtual training opportunities
• PTO/parental leave
• Competitive 401K and employee benefits
• Free financial counseling, health coaching and employee assistance program
• Tuition assistance program
• Work arrangements that work for you
• Effective productivity/technology tools and training

The pay range for this position is $120,375 - $192,600 with anticipated pay for new hires between the minimum and midpoint of the range and could vary above and below the listed range as permitted by applicable law. Pay is based on non-discriminatory factors including but not limited to work experience, education, location, licensure requirements, proficiency and qualifications required for the role. The base pay is just one component of Lincoln’s total rewards package for employees. In addition, the role may be eligible for the Annual Incentive Program, which is discretionary and based on the performance of the company, business unit and individual. Other rewards may include long-term incentives, sales incentives and Lincoln’s standard benefits package.

At Lincoln Financial, we love what we do. We make meaningful contributions each and every day to empower our customers to take charge of their lives. Working alongside dedicated and talented colleagues, we build fulfilling careers and stronger communities through a company that values our unique perspectives, insights and contributions and invests in programs that empower each of us to take charge of our own future., Lincoln Financial (NYSE: LNC) helps people to confidently plan for their version of a successful future. We focus on identifying a clear path to financial security, with products including annuities, life insurance, group protection, and retirement plan services.

With our 120-year track record of expertise and integrity, millions of customers trust our solutions and service to help put their goals in reach.

Lincoln Financial Distributors, a broker-dealer, is the wholesale distribution organization of Lincoln Financial. Lincoln Financial is the marketing name for Lincoln Financial Corporation and its affiliates including The Lincoln National Life Insurance Company, Fort Wayne, IN, and Lincoln Life & Annuity Company of New York, Syracuse, NY. Lincoln Financial affiliates, their distributors, and their respective employees, representatives and/or insurance agents do not provide tax, accounting or legal advice.

Lincoln is committed to creating an inclusive environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status.

Follow us on Facebook , X , LinkedIn , Instagram , and YouTube . For the latest company news, visit our newsroom .

Be Aware of Fraudulent Recruiting Activities

If you are interested in a career at Lincoln, we encourage you to review our current openings and apply on our website. Lincoln values the privacy and security of every applicant and urges all applicants to diligently protect their sensitive personal information from scams targeting job seekers. These scams can take many forms including fake employment applications, bogus interviews and falsified offer letters.