> Markdown version of [/jobs/ext/3700920-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/3700920-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior, Data Engineer

- **Company:** Epsilon, Inc.
- **Location:** Chicago, IL, United States (Remote available)
- **Experience:** Expert
- **Salary:** $145,870.0 - $182,000.0
- **Contract:** Temporary contract
- **Skills:** Artificial Intelligence, Amazon Web Services, Amazon S3, Audit Trail, Unit Testing, Big Data, Continuous Integration, Extract Transform Load (ETL), Amazon DynamoDB, Greenplum, Identity and Access Management, Python (Programming Language), PostgreSQL, SQL Databases, Delivery Pipeline, Large Language Models, Mttr, Cloudformation, Pyspark, Integration Tests, Information Technology, Deployment Automation, AWS Glue, Cloudwatch, Data Pipelines, Amazon Elastic Mapreduce (EMR)
- **Published:** October 12, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=a4383f4a43e40098

## About the Role

Requirements: Employer will accept a Master's degree in Computer Science, Information Technology, Engineering or related field and two years of experience in the job offered or two years in any occupation in which the required experience was gained.

Position requires two years in the following

## Description

Duties: Develop and execute software tests to identify software problems and their causes. Write and optimize complex SQL Scripts and build Python scripts to execute large-scale transformations on Greenplum. Create test plans, run rigorous unit/integration tests, and deliver reproducible, well-documented results. Design and maintain AI-enabled, PySpark data pipelines on AWS primarily EMR, Glue, Athena, DynamoDB, and S3 to process large datasets at scale. Harden and tune jobs for performance, cost, and compliance, and manage data partitioning, cataloging, and lineage. Automate deployments and infrastructure with CloudFormation and parameterized configurations, and schedule batch workloads to keep pipelines timely and reliable. Build internal dashboards and data products (e.g., Amplify apps) and instrument monitoring/alerting to catch issues early. client tickets, translating ambiguous requests into clear technical tasks., * Build PySpark ETL on AWS EMR/Glue reading/writing S3 (partitioning, predicate pushdown, broadcast joins) to deliver analytics-ready fact/dimension tables on schedule.

 * Author and tune complex SQL on Greenplum/Postgres (window functions, ROW_NUMBER dedupe, distribution keys, AO/AOCO compression, VACUUM/ANALYZE) to process large datasets efficiently.
 * Manage AWS Glue Data Catalog and query curated layers with Athena; design S3 layouts (prefix/partition strategy, lifecycle policies) to cut query time and storage cost.
 * Implement CI/CD with CodeCommit, CodeBuild, and CodePipeline; parameterize environments and use CloudFormation for IaC to enable repeatable, auditable, and fast rollbacks.
 * Engineer observability with CloudWatch logs/metrics/alarms and job health checks, create SLA dashboards and alerting to detect failures early and reduce MTTR.
 * Enforce data quality via SQL/Python unit tests, reconciliation (row counts, sums, duplicates), referential-integrity checks, and documented UAT evidence for change control.
 * Apply security and data-governance controls: normalize and encrypt sensitive identifiers, use KMS-managed keys, and enforce IAM least privilege with audit logging.
 * Develop GenAI-assisted delivery integrates LLMs (AWS Bedrock) with PySpark/SQL to auto-suggest transformations, generate test cases, and summarize validation results.
 * Build lightweight internal data apps with AWS Amplify + DynamoDB/Athena to manage lookup keys/brand lists and enable stakeholder self-service with controlled access.

Required to work from Company Office in Chicago IL certain days per week. Telecommuting allowed for balance of week.