> Markdown version of [/jobs/ext/2038512-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/2038512-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** Insight Global - **Location:** United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Amazon Web Services, Big Data, Customer Data Management, Data as a Services, Data Governance, Extract Transform Load (ETL), Data Stores, Data Systems, Distributed Computing Environment, Distributed Systems, Python (Programming Language), Machine Learning, MongoDB, SQL Databases, Data Streaming, Technical Data Management Systems, Apache Spark, Cloudformation, Data Lakes, Apache Kafka, Spark Streaming, Data Management, Tools for Reporting, Amazon Simple Queue Service (SQS), Terraform, Data Pipelines, Guidewire - **Published:** August 12, 2026 - **Apply:** https://dejobs.org/x/x/02F088FB18FE4417A473A138CE48F48B/job/ ## About the Role Must-Haves: Strong proficiency in Python and SQL Recent, hands-on experience with Apache Spark in production environments Deep expertise building and optimizing ETL/ELT pipelines Extensive experience with AWS data services, Experience with distributed processing technologies beyond managed platforms Experience with data streaming technologies: Kafka Spark Streaming SQS-based event architectures Strong data modeling experience, preferably within insurance or financial services Experience designing and maintaining data lakes Knowledge of medallion architecture (Bronze/Silver/Gold) Schema evolution and incremental processing experience Familiarity with MongoDB and document-based data stores Infrastructure as Code experience (Terraform or CloudFormation) Experience troubleshooting and resolving data quality issues across distributed systems Experience implementing data quality frameworks such as Great Expectations Strong business acumen with the ability to translate technical solutions into business outcomes Ability to optimize pipelines for performance, scalability, and cost efficiency Excellent communication skills and ability to explain the business problem being solved Plusses: Experience in Insur-Tech or Technology Experience handling insurance claims data and complex feature relationships Experience supporting machine learning platforms or feature stores ## Description Insight Global is seeking a Senior Data Engineer for a leading insur-tech client. This individual will join a highly technical Data Engineering team focused on modernizing a core subrogation platform and revamping customer data integration capabilities. The ideal candidate is a deeply technical engineer who thrives in large-scale data environments, understands the business impact of data solutions, and can design highly efficient data pipelines within AWS. This role will be responsible for building and optimizing ETL/ELT frameworks, maintaining data lakes and feature stores, supporting machine learning initiatives, and enabling seamless customer integrations through APIs and Guidewire-based solutions. Success in this position requires strong Spark expertise, deep AWS knowledge, excellent communication skills, and the ability to connect technical decisions to measurable business outcomes. Day-to-Day: Design, build, and optimize ETL/ELT pipelines that transform customer and enterprise data into actionable datasets Develop and maintain scalable AWS-based data platforms and data lakes Partner with Product and Market teams to create reporting tools and dashboards that support business strategy Build and maintain feature stores used by ML Engineering teams Develop customer data integration pipelines via APIs, Guidewire, and other ingestion methods Implement data quality controls and monitoring frameworks Troubleshoot data issues across distributed systems and streaming architectures Optimize Spark workloads and cloud resources for performance and cost savings ## Related Videos - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [40 Minutes to Build a Serverless COVID-19 REST and GraphQL APIs](https://www.wearedevelopers.com/videos/208-40-minutes-to-build-a-serverless-covid-19-rest-and-graphql-apis) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [NoSQL Data Modeling for Front-end Developers](https://www.wearedevelopers.com/videos/297-nosql-data-modeling-for-front-end-developers) ## Related Articles - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries)