> Markdown version of [/jobs/ext/3198647-senior-databricks-snowflake-data-architect-bestx](https://www.wearedevelopers.com/jobs/ext/3198647-senior-databricks-snowflake-data-architect-bestx). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Databricks / Snowflake Data Architect - BestX

- **Company:** State Street Corporation
- **Location:** London, UK
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Agile Methodology, Artificial Intelligence, Amazon Web Services, Amazon S3, Business Analytics Applications, Apache HTTP Server, Architectural Patterns, Batch Processing, BigQuery, Databases, Continuous Integration, Data Architecture, Information Engineering, Data Governance, Data Infrastructure, Extract Transform Load (ETL), Data Sharing, Data Vault Modeling, Data Warehousing, DevOps, Dimensional Modeling, Distributed Systems, Python (Programming Language), Meta-Data Management, Performance Tuning, Rapid Prototyping Process, Power BI, DataOps, Azure Machine Learning, Software Engineering, SQL Databases, Data Streaming, Tableau (Software), Snowflake, Apache Spark, KDB+, Containerization, Data Lakes, Apache Kafka, Spark Streaming, Data Management, Machine Learning Operations, Cloud Migration, Vertica, Serverless Computing, Amazon Redshift, Databricks
- **Published:** September 11, 2026
- **Apply:** https://www.totaljobs.com/job/senior-data-architect/state-street-corporation-job107968510

## About the Role

* Experience in designing and implementing enterprise-scale data platforms.
 * Strong hands-on experience with Databricks, including Delta Lake, Unity Catalog, Spark, Structured Streaming, and Lakehouse architecture
 * Strong hands-on experience with Snowflake, including data modelling, performance optimisation, Snowpark, data sharing, and governance capabilities
 * Expertise in Apache Spark, Python, SQL, and modern ELT/ETL frameworks
 * Proven ability to rapidly evaluate, prototype, and compare emerging technologies and architectural patterns
 * Experience designing large-scale analytical platforms supporting high-volume and high-velocity datasets
 * Strong understanding of data architecture principles including dimensional modelling, Data Vault, Lakehouse methodologies, and distributed computing
 * Experience with cloud platforms, preferably AWS, including services such as S3, EMR, Glue, ECS/EKS, Lambda, and related analytics services
 * Experience building streaming architectures using technologies such as Kafka, Spark Streaming, Event Bridge, or equivalent
 * Deep understanding of data governance, security, lineage, and regulatory controls in financial services environments
 * Strong software engineering and DevOps mindset, including CI/CD and Infrastructure-as-Code
 * Excellent stakeholder management and communication skills, capable of influencing architecture decisions across multiple teams

Preferred Skills

 * Experience building analytics platforms for capital markets, including FX, Fixed Income, Equities, or Derivatives
 * Knowledge of Transaction Cost Analysis (TCA), execution analytics, and market microstructure
 * Experience with AI/ML platforms, feature stores, and MLOps frameworks
 * Experience evaluating alternative analytical databases such as DuckDB, ClickHouse, Redshift, BigQuery, KDB+, or Apache Iceberg-based architectures
 * Experience with data catalogue and governance platforms
 * Exposure to modern visualisation and BI tools such as Power BI, Tableau, Sigma, or ThoughtSpot
 * Experience leading cloud migration or platform modernisation initiatives
 * Experience operating in highly regulated financial services environments

## Description

We are seeking an experienced Senior Databricks / Snowflake Data Architect to join the BestX engineering team. In this role, you will lead the design and evolution of BestX's next-generation cloud-native data platform, leveraging Databricks, Snowflake, and modern data engineering technologies to support multi-asset Transaction Cost Analysis (TCA), analytics, and reporting capabilities.

The ideal candidate combines strong data architecture expertise with a hands-on engineering mindset and a passion for evaluating emerging technologies. You will help shape the strategic direction of our data platform through rapid prototyping, proof-of-concepts, and scalable production implementations

Key Responsibilities

 * Design and develop the architecture for BestX's next-generation multi-asset TCA and analytics platform
 * Lead technology evaluations, proof-of-concepts, and rapid prototyping initiatives across modern data platforms and cloud-native technologies
 * Architect and implement scalable data Lakehouse solutions using Databricks, Snowflake, and cloud-native services
 * Design high-performance data models, ingestion frameworks, and transformation pipelines supporting both real-time and batch processing workloads
 * Establish best practices for data governance, lineage, security, metadata management, and platform observability
 * Build and optimise analytical platforms capable of processing large-scale market, transaction, and reference datasets
 * Work closely with quantitative researchers, product teams, and engineering teams to translate business requirements into scalable technical solutions
 * Define architectural standards and technology roadmaps for data warehousing, Lakehouse, streaming, and AI/ML workloads
 * Drive adoption of modern Agile, DevOps, CI/CD, Infrastructure-as-Code, and DataOps practices
 * Develop a strong understanding of Transaction Cost Analysis (TCA), best execution workflows, and relevant regulatory requirements including MiFID II and MiFIR
 * Mentor engineering teams and provide technical leadership across strategic platform initiatives
 * Take ownership of architecture decisions from concept through production delivery


## Related Videos

- [Shifting Stress to Progress— Understanding DevOps to do DevOps Better](https://www.wearedevelopers.com/videos/268-shifting-stress-to-progress-understanding-devops-to-do-devops-better)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform)
- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)
- [Mobile vs. Backend DevOps](https://www.wearedevelopers.com/videos/1662-mobile-vs-backend-devops)

## Related Articles

- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Best Companies to work for in London: Top 25 Companies in 2023](https://www.wearedevelopers.com/magazine/187-best-companies-to-work-for-in-london-top-25-companies-in-2023)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)