> Markdown version of [/jobs/ext/480254-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/480254-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** EXL SERVICE - **Location:** United States - **Experience:** Expert - **Salary:** $130,000.0 - $150,000.0 - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Amazon S3, Apache HTTP Server, Basic Access Authentication, Big Data, Information Engineering, Data Files, Extract Transform Load (ETL), Relational Databases, Database Queries, File Systems, Amazon DynamoDB, Hadoop Distributed File System, JSON, Python (Programming Language), MongoDB, NoSQL, OAuth, Oracle (Applications), Test Data, Extensible Markup Language (XML), Parquet, Flask (Web Framework), Apache Spark, Boto3, AWS Lambda, Fastapi, Pandas, Pyspark, Information Technology, Avro, AWS Glue, Apache Kafka, Cloudwatch, Restful APIs, GPT, Data Pipelines, Docker - **Published:** June 3, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=2623aa90a431c062 ## About the Role Do you have experience in XML?, Do you have a Bachelor's degree?, * Hands-on experience with Python and PySpark * Experience with AWS services: + S3, Glue, Lambda, EMR, Step Functions, EventBridge, Athena * Experience with Kafka integration * Strong SQL skills (writing complex queries) * Experience working with data file formats (Parquet, Avro, ORC, JSON, XML) * Experience using Python libraries (pandas, requests, boto3) * Experience building REST APIs (FastAPI or Flask) Experience * 4+ years of experience in Data Engineering or related field * Bachelor's degree in Computer Science or related field (or equivalent) Base Compensation Range: $130,000 - $150,000 The posted range is the hiring range for this role - a subset of the broader range available to employees over time - and reflects base salary across our national hiring scale. Final offers are based on several factors, including the candidate's skills and experience, internal pay equity, work location, market conditions for the role, and the specific scope and responsibilities of the position. The top of the range is reserved for candidates who notably exceed the requirements; the lower end applies to those with less experience or fewer preferred qualifications. For positions based in higher-cost zones (e.g., California, New York, New Jersey), actual compensation may exceed the posted range; your recruiter will share specifics during the process. Responsibilities: The client is specifically looking for candidates with strong hands-on experience in the following technologies: * Building data pipelines using Python and PySpark on AWS Glue, EMR, and Lambda * Developing and securing RESTful APIs (FastAPI) deployed on Docker/EKS, with OAuth2/JWT-based authentication * Hands-on experience with Apache Iceberg tables for CDC and latest snapshot handling * Designing event-based pipelines using Apache Kafka / MSK for data consumption and publishing * Ability to lead and communicate complex technical designs, and leverage Copilot/GPT for agentic coding across the stack Qualifications: The client is specifically looking for candidates with strong hands-on experience in the following technologies: * Building data pipelines using Python and PySpark on AWS Glue, EMR, and Lambda * Developing and securing RESTful APIs (FastAPI) deployed on Docker/EKS, with OAuth2/JWT-based authentication * Hands-on experience with Apache Iceberg tables for CDC and latest snapshot handling * Designing event-based pipelines using Apache Kafka / MSK for data consumption and publishing * Ability to lead and communicate complex technical designs, and leverage Copilot/GPT for agentic coding across the stack ## Description * Build and test data processing applications using PySpark and Python * Develop data pipelines using AWS Glue ETL or EMR * Create AWS Lambda functions using Python (pandas, json, requests, awswrangler) * Work with data from: + Relational databases (Oracle) + NoSQL databases (DynamoDB, MongoDB) + File systems (S3, HDFS) * Implement event-driven pipelines using Kafka * Develop REST APIs using FastAPI or Flask * Implement basic authentication (OAuth2 / JWT) for APIs * Build workflow orchestration pipelines using Step Functions and EventBridge * Work with big data file formats such as Parquet, Avro, ORC, JSON * Optimize Spark jobs using standard techniques (partitioning, joins, etc.) * Use Glue Crawlers to catalog datasets * Monitor and troubleshoot jobs using CloudWatch * Support deployment using Docker containers ## Related Videos - [Tips and Tricks for Working with JSON](https://www.wearedevelopers.com/videos/1229-tips-and-tricks-for-working-with-json) - [From event streaming to event sourcing 101](https://www.wearedevelopers.com/videos/91-from-event-streaming-to-event-sourcing-101) - [ Evaluating AI models for code comprehension](https://www.wearedevelopers.com/videos/1462-evaluating-ai-models-for-code-comprehension) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [Introducing JSON Structure](https://www.wearedevelopers.com/videos/100219-introducing-json-structure) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Software Engineer Salary in The UK](https://www.wearedevelopers.com/magazine/231-software-engineer-salary-in-the-uk)