Sr Data Engineer

We are looking for a Sr Data Engineer focused on Foundry, governance, release engineering, and geospatial data products. This person will be developing, engineering and improving an existing platform and pipeline ecosystem., We are seeking a Data Engineer to support the transition of data pipelines from an internal IT team to our internal team. This role will focus on both enhancing existing pipelines and improving the underlying code and data infrastructure to ensure long-term maintainability, scalability, and clarity. Key Responsibilities

• Refactor and enhance data pipelines to be modular, maintainable, and well-documented
• Establish and promote best practices for code architecture, version control, and code management
• Collaborate closely with a core team of data scientists, machine learning engineers, and data engineers and a broader team of cross-functional partners
• Communicate technical data to non-technical stakeholders to build trust and understanding in the methodology.
• Support the development of machine learning models currently being developed in-house

• Strong experience in Python-based code development
• Demonstrated expertise in code architecture, software engineering practices, and maintainable pipeline design, in a cloud-based environment
• Experience with the Palantir Foundry platform (current technology, plan to move to AWS in future, should have this experience as well)
• Familiarity with PySpark and distributed computing.
• Experience with release engineering
• Repository governance experience; audit ready and branching strategy (highly preferred)
• Traceability
• Experience working with spatial-temporal datasets and geospatial packages such as Sedona, Geopandas, and Rasterio.
• Experience transitioning data pipelines from consulting or external vendors into production environments
• Experience designing cloud-optimized geospatial datasets (GeoParquet, Parquet, Zarr) with efficient partitioning and support for large-scale spatial operations and aggregations.
• Proficiency with version control (Git) and collaborative workflows (e.g., pull requests, code reviews)
• Ability to assess and improve legacy or externally developed codebases

• Excellent communication skills Preferred Qualifications

• Experience with a GIS platform such as QGIS or ArcGIS
• Experience working with ML model outputs
• Experience designing production-grade data and ML platforms with strong emphasis on reproducibility, dataset versioning, release traceability, repository governance, and maintainable software architecture.

Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.