> Markdown version of [/jobs/ext/1750717-lead-data-engineer-finance-technology-hadoop-pyspark-scala-java](https://www.wearedevelopers.com/jobs/ext/1750717-lead-data-engineer-finance-technology-hadoop-pyspark-scala-java). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer - Finance Technology (Hadoop, PySpark, Scala/Java)

- **Company:** Target Brands, Inc.
- **Location:** Brooklyn Park, MN, United States
- **Experience:** Expert
- **Salary:** $132,000.0 - $238,000.0
- **Contract:** Permanent contract
- **Skills:** Java (Programming Language), Application Programming Interfaces (APIs), Artificial Intelligence, Amazon Web Services, Amazon S3, Data Analysis, Microsoft Azure, Big Data, BigQuery, Cloud Computing, Computer Programming, Data as a Services, Data Architecture, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Security, Data Systems, Distributed Computing Environment, Distributed Systems, Apache Hadoop, Machine Learning, Performance Tuning, Scala (Programming Language), Software Engineering, Data Streaming, Data Processing, Snowflake, Apache Spark, Electronic Medical Records, Pyspark, Information Technology, Apache Flink, Real Time Data, Apache Kafka, Graphql, Api Design, Data Pipelines, Databricks
- **Published:** July 16, 2026
- **Apply:** https://target.wd5.myworkdayjobs.com/targetcareers/job/7000-Target-Pkwy-NNCD-0375-Brooklyn-ParkMN-55445/Lead-Data-Engineer---Finance-Technology--Hadoop--PySpark--Scala-Java-_R0000444860/apply

## About the Role

* BS or MS in Computer Science, Machine Learning, Artificial Intelligence, Applied Mathematics or a related technical field preferred
 * 7plus years of experience in data engineering, software development, or distributed systems
 * Expertise in Big Data technologies such as Hadoop, Spark, and distributed processing frameworks
 * Strong programming skills in PySpark, Scala, and/or Java
 * Experience with cloud platforms (AWS, GCP, or Azure) and associated data services (e.g. S3, BigQuery, Databricks, EMR, Snowflake)
 * Proficiency in API development using REST, GraphQL, or gRPC to support real-time and batch data access
 * Experience with real-time and streaming data architectures like Kafka, Flink or Kinesis
 * Strong knowledge of data modeling, ETL pipeline design and performance optimization
 * Understanding of data governance, security, and compliance in large-scale data environments
 * Strong problem-solving skills and the ability to thrive in complex, evolving environments
 * Excellent communication and collaboration skills, with experience working across cross-functional teams, Please paste this url into your preferred browser to learn about benefits eligibility for this role: https://tgt.biz/BenefitsForYou_E

## Description

As a Lead Data Engineer, you will serve as the technical lead for the engineering team, designing and developing scalable, high-performance data solutions. You will drive the evolution of our data architecture, ensuring it meets both functional and non-functional business requirements while delivering reliability, efficiency, and scalability. Your expertise in big data technologies, distributed systems, and cloud platforms will help shape the engineering roadmap and establish best practices for data processing, analytics, and real-time data services. You will play a key role in architecting and optimizing data pipelines using Hadoop, Spark, Scala/Java, and cloud technologies to support enterprise-wide data initiatives., * Architect and build scalable, high-performance data pipelines and distributed data processing solutions using Spark, Scala/Java and cloud platforms (AWS, GCP, or Azure)

 * Design and implement real-time and batch data processing solutions, ensuring data is efficiently processed and readily available for analytical and operational use
 * Develop APIs and data services that provide low-latency, high-throughput access to data for downstream applications, enabling real-time decision-making
 * Optimize and enhance data models, workflows, and processing frameworks to improve performance, scalability, and cost efficiency
 * Drive data governance, security, and compliance best practices
 * Collaborate with product teams and business stakeholders to understand requirements and deliver data-driven solutions
 * Lead the design, implementation, and lifecycle management of data services and platforms
 * Stay current with emerging technologies and promote the adoption of best practices in big data engineering, cloud computing, and API development
 * Provide technical leadership and mentor engineers to foster engineering excellence and best practices

Core responsibilities of this job are articulated within this job description. Job duties may change at any time due to business needs.


## Related Videos

- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)
- [Putting the Graph In GraphQL With The Neo4j GraphQL Library](https://www.wearedevelopers.com/videos/257-putting-the-graph-in-graphql-with-the-neo4j-graphql-library)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)
- [GraphQL + Apollo + Next.js: A Lovely Trio](https://www.wearedevelopers.com/videos/311-graphql-apollo-next-js-a-lovely-trio)

## Related Articles

- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [The Biggest German Tech Companies](https://www.wearedevelopers.com/magazine/424-the-biggest-german-tech-companies)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Best Companies to work for in London: Top 25 Companies in 2023](https://www.wearedevelopers.com/magazine/187-best-companies-to-work-for-in-london-top-25-companies-in-2023)