> Markdown version of [/jobs/ext/2877642-mid-level-lead-data-engineer-python-aws-spark-hybrid-tx](https://www.wearedevelopers.com/jobs/ext/2877642-mid-level-lead-data-engineer-python-aws-spark-hybrid-tx). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Mid-Level/Lead DATA ENGINEER-Python, AWS, Spark (Hybrid-TX) - **Company:** State Farm Insurance - **Location:** Richardson, TX, United States (Remote available) - **Experience:** Experienced - **Salary:** $85,500.0 - $130,000.0 - **Contract:** Temporary to permanent - **Skills:** Java (Programming Language), Airflow, Amazon Web Services, Amazon S3, Automation of Tests, Bash Shell, Data Architecture, Information Engineering, Extract Transform Load (ETL), Data Security, Data Systems, IBM DB2, Relational Databases, DevOps, Distributed Computing Environment, Amazon DynamoDB, Github, R (Programming Language), Apache Hive, Python (Programming Language), PostgreSQL, DataOps, Solution Deployment Descriptor, SQL Databases, Unstructured Data, Automated Data Processing (ADP), Enterprise Software Applications, Snowflake, Apache Spark, Gitlab, Servicebus, Pyspark, Infrastructure Automation Frameworks, Information Technology, Star Schema, Terraform, Software Version Control, Data Pipelines, Serverless Computing, Amazon Redshift, Databricks, Vulnerability Analysis - **Published:** September 13, 2026 - **Apply:** https://www.careerjet.com/job/usa15ed5957d6430a7a97b33119350c877/eaa ## About the Role * Professional experience as a Data Engineer. * Proficiency in programming languages such as Python, Spark SQL (or PySpark), R, Java, Bash, etc. * Hands-on experience with AWS services including ETL tools (Glue, EMR Serverless), Lambda, Step Functions, EventBridge, S3, DynamoDB, Kinesis Firehose, Redshift, Iceberg, and SageMaker. * Experience with distributed data processing frameworks such as Apache Spark, Databricks. * Experience with infrastructure as code tools such as OpenTofu (formerly Terraform) for managing cloud resources and deployments. * Familiarity with CI/CD pipelines including automated testing, security scans, and tools like Airflow. Additional * Experience or ability to rapidly gain P&C data domain knowledge, including rating, underwriting, and/or claims. * Experience with relational databases such as DB2, Postgres, Redshift, etc. * Experience with version control systems such as GitHub or GitLab. * Data access skills using SQL, and Athena. * Experience in designing, building, and maintaining data pipelines for automated data processing. * Knowledge of data modeling techniques such as star schema and snowflake schema, with an understanding of data architecture., * Adaptability * Work Ethic * Critical Thinking * Strategic Business Focus * Technical/Functional Expertise ## Description Our Property & Casualty Data Pipeline team is hiring an Experienced Data Engineer to help advance one of the company's highest priorities. In this role, you will build and scale the data products and pipelines that power better business decision, profitable growth, and stronger customer retention. You will develop automated, reliable analytical data assets that help transform how we operate and how we serve our customers. This position sits within Enterprise Technology and offers the opportunity to work in a highly collaborative environment focused on delivering meaningful business impact. You will partner across teams, solve complex data challenges, and help shape modern data solutions aligned to enterprise goals. It is a strong fit for someone who wants to expand their technical skills, work on high-visibility initiatives, and contribute in a fast-moving, innovative setting. Your Responsibilities May Include: * Utilizes industry-adopted languages and frameworks in coding, testing, security, DevOps, DataOps and data engineering practices * Develops and maintains reusable, scalable, and compliant data solutions across multiple platforms and compute environments * Responsible for the identification, acquisition, cleansing, profiling, and ETL (extracting, transformation, and loading) of data used in analytic discovery and production solution deployment across multiple platforms * Establishes business domain knowledge for existing State Farm data sources and investigates, recommends, and initiates acquisition of data resources, both internal and external * Identifies and consults on emerging technologies and critical core systems, including techniques, tools, data sources, and platforms in the data engineering field * Familiar with handling datasets containing mixes of structured and unstructured data * Exhibits DataOps mindset where team is accountable for ensuring data aligns to enterprise needs and leveraging automation to deliver quality data solutions * Collects and analyzes information to identify customer's technical needs, suggest solutions and develop implementation and integration plans (e.g., technical proposals) * Responsible for the analysis, design, deployment, support, and security of technology to ensure the organization is efficiently managing its technology and data-related assets in accordance with market best-practices and external regulations * Applies a wide application of complex principles, theories, and concepts in computer science for data engineering solutions ## Related Videos - [WeAreDevelopers LIVE - Modern DevOps for IoT Devices and More](https://www.wearedevelopers.com/videos/1805-wearedevelopers-live-modern-devops-for-iot-devices-and-more) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [Enabling automated 1-click customer deployments with built-in quality and security](https://www.wearedevelopers.com/videos/83-enabling-automated-1-click-customer-deployments-with-built-in-quality-and-security) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Data Analyst Salary in Switzerland](https://www.wearedevelopers.com/magazine/276-data-analyst-salary-in-switzerland) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)