> Markdown version of [/jobs/ext/2518784-data-engineer-officer-state-street-investment-management](https://www.wearedevelopers.com/jobs/ext/2518784-data-engineer-officer-state-street-investment-management). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer, Officer - State Street Investment Management - **Company:** State Street - **Location:** Boston, MA, United States - **Experience:** Expert - **Salary:** $70,000.0 - $118,750.0 - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Agile Methodology, Artificial Intelligence, Amazon Web Services, Microsoft Azure, Code Generation, Continuous Integration, Information Engineering, Data Infrastructure, Data Warehousing, Software Debugging, Distributed Computing Environment, Github, Python (Programming Language), Systems Development Life Cycle, Reliability Engineering, SQL Databases, Data Streaming, Unstructured Data, Data Ingestion, GitHub Copilot, Snowflake, Apache Spark, Git, Data Lakes, Pyspark, Information Technology, Collibra, Data Pipelines, Docker, Jenkins, Databricks, Artifactory - **Published:** August 3, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=3e1a812498afbdbe ## About the Role * Master's degree in Computer Science or a related engineering discipline. * 6-10 years of technology experience with Data Engineeting focus * Strong hands-on in Python, Spark, SQL, Databricks and Snowflake, with the ability to design, debug, and optimize distributed data processing; * Experience building Lakehouse platforms using Snowflake and Databricks on AWS (Azure acceptable), supporting high concurrency analytical workloads. * Proficiency in data engineering and automation * 5+ years of hands-on experience building data pipelines on cloud platforms * Some experience working on modern Data Platform Architecture, Lakehouse, Data Products, and cloud-native analytics ecosystems. * Good understanding of Data Warehousing, Data Lakes, and Data Quality concepts * Experience working with both unstructured data sources and processing patterns * Exposure with GIT, CI/CD Process for Deployment and Releases. * Strong operational and engineering excellence mindset, with experience establishing SDLC standards, CI/CD, observability, reliability engineering, incident management, and continuous improvement practices for mission-critical platforms. * Familiarity with AI-assisted development tools (e.g., GitHub Copilot, Codex, Anthropic tools)., * Previous experience in data/platform development within a financial institution. * Experience with both structured and unstructured data management tools and associated patterns * Familiarity with modern governance frameworks (Unity Catalog, Collibra, Alation). * Expertise in AI-assisted coding using GitHub Copilot or Claude Code Assistant. ## Description * Design, develop, test, and maintain data pipelines using Python, PySpark, and SQL. * Design, build, and manage robust workflows and data ingestion, transformation solutions using Databricks and Snowflake * Hands-on experience with Databricks, Snowflake on AWS (Azure acceptable) * Design and maintain CI/CD pipelines using tools such as Jenkins, Artifactory, GitHub. * Develop containerized applications for APIs, event based, streaming and batch using Docker. * Prior experience on working in Agile project management practices * Work under minimal supervision to analyze, design, develop, test, and debug * Participate in cross-team group activities to complete assignments. * Work with cross-functional teams and other engineers on the core platform capabilities * Develop Iceberg, unity catalog based tables for consistent, performant analytical access across multiple compute engines. * Partner with product owners, analytics teams, and downstream consumers to translate business requirements into technical solutions * Apply AI code generation tools (e.g., Anthropic Code, Codex, and GitHub Copilot) to accelerate development, improve quality, and increase productivity. * Excellent verbal and written communication and should be able to work with all the stakeholders. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [Bringing AI Model Testing and Prompt Management to Your Codebase with GitHub Models](https://www.wearedevelopers.com/videos/1536-bringing-ai-model-testing-and-prompt-management-to-your-codebase-with-github-models) ## Related Articles - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)