> Markdown version of [/jobs/ext/2526797-sr-data-engineer](https://www.wearedevelopers.com/jobs/ext/2526797-sr-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Sr Data Engineer - **Company:** Insight Global - **Location:** Oakland, CA, United States - **Experience:** Expert - **Salary:** $158,080.0 - $197,600.0 - **Contract:** Permanent contract - **Skills:** Geographic Information Systems, Amazon Web Services, ArcGIS (Software), Cloud Computing, Code Review, Data Infrastructure, Distributed Systems, Python (Programming Language), Machine Learning, Quantum GIS (QGIS), Azure Machine Learning, Software Engineering, Technical Data Management Systems, Management of Software Versions, Parquet, Git, Pyspark, Software Coding, Code Restructuring, Software Version Control, Data Pipelines - **Published:** August 6, 2026 - **Apply:** https://jobs.insightglobal.com/jobs/find_a_job/california/oakland/sr-data-engineer/job-560730/ ## About the Role * Strong experience in Python-based code development * Demonstrated expertise in code architecture, software engineering practices, and maintainable pipeline design, in a cloud-based environment * Experience with the Palantir Foundry platform (current technology, plan to move to AWS in future, should have this experience as well) * Familiarity with PySpark and distributed computing. * Experience with release engineering * Repository governance experience; audit ready and branching strategy (highly preferred) * Traceability * Experience working with spatial-temporal datasets and geospatial packages such as Sedona, Geopandas, and Rasterio. * Experience transitioning data pipelines from consulting or external vendors into production environments * Experience designing cloud-optimized geospatial datasets (GeoParquet, Parquet, Zarr) with efficient partitioning and support for large-scale spatial operations and aggregations. * Proficiency with version control (Git) and collaborative workflows (e.g., pull requests, code reviews) * Ability to assess and improve legacy or externally developed codebases * Excellent communication skills Preferred Qualifications * Experience with a GIS platform such as QGIS or ArcGIS * Experience working with ML model outputs * Experience designing production-grade data and ML platforms with strong emphasis on reproducibility, dataset versioning, release traceability, repository governance, and maintainable software architecture. ## Description We are looking for a Sr Data Engineer focused on Foundry, governance, release engineering, and geospatial data products. This person will be developing, engineering and improving an existing platform and pipeline ecosystem., We are seeking a Data Engineer to support the transition of data pipelines from an internal IT team to our internal team. This role will focus on both enhancing existing pipelines and improving the underlying code and data infrastructure to ensure long-term maintainability, scalability, and clarity. Key Responsibilities * Refactor and enhance data pipelines to be modular, maintainable, and well-documented * Establish and promote best practices for code architecture, version control, and code management * Collaborate closely with a core team of data scientists, machine learning engineers, and data engineers and a broader team of cross-functional partners * Communicate technical data to non-technical stakeholders to build trust and understanding in the methodology. * Support the development of machine learning models currently being developed in-house ## Related Videos - [Parquet, Delta, Iceberg & Ducklake - An introduction for developers](https://www.wearedevelopers.com/videos/100075-parquet-delta-iceberg-ducklake-an-introduction-for-developers) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Résumé-Driven Development: How IT trends affect the job market for software developers](https://www.wearedevelopers.com/magazine/59-resume-driven-development-how-it-trends-affect-the-job-market-for-software-developers)