> Markdown version of [/jobs/ext/1925654-data-engineer](https://www.wearedevelopers.com/jobs/ext/1925654-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineer

- **Company:** Calsoft Private Limited
- **Location:** Irvine, CA, United States (Remote available)
- **Experience:** Expert
- **Salary:** $125,000.0 - $180,000.0
- **Contract:** Permanent contract
- **Skills:** Clean Code Principles, Airflow, Amazon Web Services, Amazon S3, Unit Testing, Cloud Computing, Code Review, Databases, Continuous Integration, Data as a Services, Data Validation, Information Engineering, Data Infrastructure, Extract Transform Load (ETL), Data Warehousing, Distributed Computing Environment, Fault Tolerance, Monitoring of Systems, Identity and Access Management, Python (Programming Language), PostgreSQL, Machine Learning, Metadata, MySQL, Object-Oriented Software Development, Performance Tuning, Query Optimization, Cloud Services, SQL Databases, Data Streaming, Data Processing, Sql Optimization, System Availability, Database Optimization, Apache Spark, Git, Cloudformation, Data Lakes, Pyspark, Deployment Automation, Apache Kafka, Cloudwatch, Terraform, Data Pipelines, Docker
- **Published:** August 5, 2026
- **Apply:** https://www.dice.com/job-detail/01090ed2-540c-45ec-8cb1-5932ac618c46

## About the Role

* Strong Python programming skills.

 * Experience building production-grade data engineering solutions.

 * Knowledge of object-oriented design, testing, and clean coding practices.

AWS

Hands-on experience with several of the following:

 * S3

 * Glue

 * Lambda

 * EMR

 * Athena

 * Redshift

 * RDS

 * Step Functions

 * EventBridge

 * CloudWatch

 * IAM

 * ECS/EKS (preferred)

Data Engineering

 * ETL/ELT pipeline development

 * Data warehousing concepts

 * Batch and streaming data processing

 * Data modeling

 * Data validation and quality frameworks

 * Metadata and lineage concepts

Databases

 * Advanced SQL

 * PostgreSQL

 * MySQL

 * Redshift

 * Performance tuning and query optimization

Development Practices

 * Git

 * CI/CD pipelines

 * Unit testing

 * Docker

 * Agile/Scrum development

Preferred Skills

 * Apache Spark or PySpark

 * Kafka or Kinesis

 * Airflow or AWS Managed Workflows

 * Infrastructure as Code (Terraform or CloudFormation)

 * Data Lake architecture

 * Lakehouse concepts

 * Experience supporting Machine Learning data pipelines

 * Experience with observability and monitoring tools

Desired Experience

 * 6-10 years of experience in Data Engineering.

 * Experience building cloud-native data platforms on AWS.

 * Experience handling large-scale structured and semi-structured datasets.

 * Strong understanding of distributed data processing and performance optimization.

 * Experience working in enterprise production environments with high availability requirements.

## Description

Data Engineering & Platform Development

 * Design, develop, and maintain scalable batch and streaming data pipelines on AWS.

 * Build robust ETL/ELT frameworks for ingesting data from multiple internal and external sources.

 * Develop reusable data services that support analytics, reporting, machine learning, and operational workloads.

 * Design efficient data models optimized for both analytical and operational use cases.

 * Ensure data quality, consistency, lineage, and governance across the data platform.

Cloud & Infrastructure

 * Build and optimize cloud-native data infrastructure using AWS services.

 * Design highly available, fault-tolerant, and scalable data processing architectures.

 * Automate deployment, monitoring, and operational workflows using Infrastructure as Code and CI/CD practices.

 * Optimize storage, compute utilization, and overall cloud costs.

Performance & Reliability

 * Improve pipeline performance, scalability, and reliability.

 * Monitor production workloads and proactively resolve bottlenecks.

 * Optimize SQL queries, data partitioning, indexing strategies, and processing performance.

 * Troubleshoot production issues and implement preventive improvements.

Collaboration

 * Work closely with Data Scientists, Software Engineers, Product Owners, and Architects to understand business requirements.

 * Support downstream analytics, dashboards, and reporting solutions.

 * Participate in architecture discussions, code reviews, and technical design sessions.

 * Mentor junior engineers and promote engineering best practices.


## Related Videos

- [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94)
- [MySQL Protocol Features You Should Be Aware Of](https://www.wearedevelopers.com/videos/100267-mysql-protocol-features-you-should-be-aware-of)
- [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Branch your database like your code:  How schema changes and pull requests go hand in hand](https://www.wearedevelopers.com/videos/350-branch-your-database-like-your-code-how-schema-changes-and-pull-requests-go-hand-in-hand)
- [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews)

## Related Articles

- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)
- [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers)