> Markdown version of [/jobs/ext/1180414-senior-software-engineer-big-data-scala-spark](https://www.wearedevelopers.com/jobs/ext/1180414-senior-software-engineer-big-data-scala-spark). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Software Engineer (Big Data: Scala, Spark,... - **Company:** Mastercard - **Location:** O'Fallon, MO, United States - **Experience:** Expert - **Salary:** $115,000.0 - $184,000.0 - **Contract:** Permanent contract - **Skills:** Java (Programming Language), Artificial Intelligence, Amazon Web Services, Amazon Elastic Compute Cloud, Amazon S3, Data Analysis, Big Data, Cloud Computing, Cloud Engineering, Cloudera Impala, Information Systems, System Configuration, Data Architecture, Information Engineering, Data Infrastructure, Data Integrity, Extract Transform Load (ETL), Data Systems, Data Visualization, Distributed Data Store, Distributed Systems, Apache Hadoop, Monitoring of Systems, Apache Hive, Identity and Access Management, Python (Programming Language), Performance Tuning, Scrum Methodology, Cloudera, Scala (Programming Language), Data Streaming, Systems Integration, Cloud Platform System, Cloudera Manager, Apache Spark, Electronic Medical Records, Backend, AI Platforms, Information Technology, Qlikview, Apache Kafka, Tools for Reporting, Virtual Agents, Data Pipelines, Databricks - **Published:** July 4, 2026 - **Apply:** https://www.juju.com/job/00000000gdjduf ## About the Role BS/BA degree in Computer Science, Engineering, Information Systems, or related field/ related experience. Strong hands on experience with Scala, Python/Java in backend or data intensive systems Experience working with Cloudera Data Platform (CDP) and Spark Familiarity with Cloudera Manager for cluster administration, monitoring, or troubleshooting Strong understanding of data modeling concepts, distributed systems, and large scale data processing Excellent problem solving skills and ability to work independently in complex environments GOOD TO HAVE / STRONGLY PREFERRED Hands on experience building, integrating, or supporting AI driven agents or intelligent automation solutions Experience with Databricks for data engineering or ML workloads Experience working in AWS (e.g., S3, EC2, EMR, Glue, Lambda, IAM, or equivalent services) Knowledge of streaming and big-data technologies: Kafka Hadoop ecosystem Hive/Impala Exposure to model monitoring, or AI platform enablement Experience with ETL tools such as Informatica Experience working in Agile / Scrum teams within large enterprises ## Description _Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential._, Have you ever wanted to be part of something BIG?, Mastercard is seeking a Backend Software Engineer to join the AI&DPE Data Engineering platform team, responsible for building and evolving large scale backend data systems, real time and batch pipelines, and AI enabled services that power analytics, decisioning, and automation across the organization. This is a Software Engineer backend engineering role, focused on Scala/Python/Java, distributed data platforms (Cloudera/Spark), and cloud based architectures, with a strong emphasis on AI agent creation and intelligent automation. Visualization tools (e.g., Qlik) are consumers of the platform, not the core focus of this role. You will work with massive transactional datasets, modern big data and cloud platforms, and AI driven workflows to transform how Mastercard processes, enriches, and operationalizes data at global scale., Software Engineer Backend with Data Platform Engineering Design, develop, and maintain backend services and data pipelines using Scala, Python and Java Build and optimize batch and streaming workloads on Cloudera Data Platform (CDP) using Spark Work with Cloudera Manager to support platform configuration, monitoring, performance tuning, and operational stability Design and implement high quality datamarts and curated datasets with strong emphasis on data integrity, performance, and reliability AI Agents & Intelligent Automation Design, build, and integrate AI powered agents that operate to support: Anomaly detection and operational intelligence workflow automation Apply generative AI driven, or rule based agents to reduce manual effort and improve scalability across backend systems Cloud & Modern Data Architecture Build and support data and compute workloads in AWS environments Leverage Databricks for large scale data processing, advanced analytics Contribute to cloud native and hybrid architectures integrating on prem and cloud platforms Integration & Downstream Enablement Enable downstream consumers (analytics, visualization, reporting tools such as Qlik) through well designed, reliable backend data interfaces Partner with analytics, fraud, and business teams to ensure backend systems meet evolving needs without compromising platform stability Documentation & Collaboration Create clear technical documentation, including architecture diagrams, data flows, and design specifications Participate in Agile/Scrum ceremonies and cross functional design reviews Mentor and upskill team members in backend engineering, big data, and AI agent concepts ## Related Videos - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [Developing the Backend with Stefan Lingler, CTO at Shpock](https://www.wearedevelopers.com/videos/100360-developing-the-backend-with-stefan-lingler-cto-at-shpock) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Why and when should we consider Stream Processing frameworks in our solutions](https://www.wearedevelopers.com/videos/1085-why-and-when-should-we-consider-stream-processing-frameworks-in-our-solutions) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Nest.js - TypeScript in the backend can also be clean](https://www.wearedevelopers.com/videos/1033-nest-js-typescript-in-the-backend-can-also-be-clean) ## Related Articles - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries) - [7 Cloud Computing Trends Coming in 2025 for Developers](https://www.wearedevelopers.com/magazine/412-7-cloud-computing-trends-coming-in-2025-for-developers) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)