> Markdown version of [/jobs/ext/4686880-senior-data-platform-architect-databricks-pyspark](https://www.wearedevelopers.com/jobs/ext/4686880-senior-data-platform-architect-databricks-pyspark). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Data Platform Architect Databricks (PySpark)

- **Company:** Purple Drive Technologies LLC
- **Location:** Dallas, TX, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Training Data, Artificial Intelligence, Amazon Web Services, Microsoft Azure, Big Data, Cloud Database, Cloud Engineering, Data Architecture, Information Engineering, Data Governance, Data Infrastructure, Data Security, Data Systems, Software Design Patterns, Distributed Computing Environment, Enterprise Architecture Framework, Apache Hadoop, Hadoop Distributed File System, Apache Hive, Industry Standard Architecture, Python (Programming Language), Performance Tuning, Cloud Services, Data Streaming, Enterprise Data Management, Google Cloud, Cloud Platform System, Data Ingestion, Apache Spark, Generative AI, Togaf, Pyspark, Data Management, Stream Processing, Data Pipelines, Databricks
- **Published:** November 12, 2026
- **Apply:** https://www.dice.com/job-detail/f3201548-c7a7-476e-94ba-439435c397d9

## About the Role

* 7-12 years of experience in Data Engineering, Data Platforms, or Solution Architecture.
 * Strong hands-on experience with Python, PySpark, and Spark SQL.
 * Strong understanding of Apache Spark and distributed data processing.
 * Experience working with the Hadoop ecosystem, including technologies such as HDFS, Hive, and related big data platforms.
 * Hands-on expertise in Databricks Lakehouse architecture and implementation.
 * Experience designing and implementing modern cloud-native data architectures.
 * Experience with at least one major cloud platform: Microsoft Azure, AWS, or Google Cloud Platform (Google Cloud Platform).
 * Strong knowledge of batch, streaming, and real-time data processing architectures.
 * Experience defining enterprise data architecture standards, reference architectures, solution blueprints, and reusable design patterns.
 * Understanding of data security, governance, scalability, performance optimization, and architecture best practices.
 * Strong communication, stakeholder management, and technical leadership skills., * Databricks certification.
 * Microsoft Azure, AWS, or Google Cloud Platform cloud certification.
 * Previous experience working with Citibank consumer banking or wealth management platforms.
 * Exposure to Enterprise Architecture frameworks, preferably TOGAF.
 * Experience in Banking, Financial Services, or large-scale enterprise data modernization programs.
 * Exposure to AI/ML and Generative AI data platform architecture.

Key Skills

 * Data Platform Architecture
 * Databricks Lakehouse
 * Python
 * PySpark
 * Apache Spark / Spark SQL
 * Hadoop Ecosystem
 * Distributed Data Processing
 * Cloud Data Architecture - Azure / AWS / Google Cloud Platform
 * Batch, Streaming & Real-Time Data Processing
 * Enterprise Architecture
 * Reference Architecture & Solution Blueprints
 * Data Governance & Security
 * AI/ML & Generative AI-Ready Data Platforms
 * Technical Leadership & Stakeholder Management

## Description

We are looking for an experienced Senior Data Platform Architect - Databricks (PySpark) to design and drive enterprise-scale data platform architecture and modernization initiatives. The role requires strong expertise in Data Engineering, Databricks Lakehouse, Python, PySpark, Spark SQL, distributed data processing, and cloud-native data architectures.

The selected candidate will collaborate closely with the Enterprise Architecture team to define target-state architectures, establish technology roadmaps, and develop scalable, secure, and reusable data architecture solutions. The role will also provide technical leadership to engineering teams and support AI/ML and Generative AI initiatives through the development of AI-ready data platforms., * Design and define scalable enterprise data platform architectures using Databricks Lakehouse and modern cloud-native technologies.

 * Collaborate with Enterprise Architecture teams to establish target-state architectures, technology roadmaps, and data modernization strategies.
 * Create, maintain, and govern reference architectures, solution blueprints, design patterns, and architecture standards.
 * Design and implement scalable batch, streaming, and real-time data processing solutions.
 * Provide technical direction for distributed data processing using Python, PySpark, Spark SQL, and the Hadoop ecosystem.
 * Drive the architecture and evolution of enterprise data platforms across Azure, AWS, or Google Cloud Platform cloud environments.
 * Define architecture standards and best practices for data ingestion, transformation, storage, processing, and integration.
 * Support AI/ML and Generative AI initiatives by designing AI-ready data platforms and enabling scalable, high-quality data pipelines.
 * Conduct architecture reviews and ensure solutions meet enterprise requirements for security, scalability, performance, governance, and reliability.
 * Evaluate emerging data technologies and recommend suitable solutions aligned with enterprise architecture standards.
 * Mentor data engineering teams and provide technical leadership throughout solution design, development, and implementation.
 * Collaborate with business stakeholders, technical teams, and architecture leaders to influence technology decisions and ensure alignment with business objectives.