Data Architect
Cyber 1 Armor
Milpitas, United States of America
2 days ago
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Experience level
SeniorJob location
Milpitas, United States of America
Tech stack
Amazon Web Services (AWS)
Azure
Batch Processing
Big Data
Code Review
Computer Programming
Continuous Integration
Data as a Services
Data Architecture
Information Engineering
Data Governance
Data Infrastructure
Data Vault Modeling
Database Queries
Software Debugging
DevOps
Python
Machine Learning
Performance Tuning
Microsoft PowerPoint
Query Optimization
Scala
SQL Databases
Data Streaming
Enterprise Data Management
Azure
Google Cloud Platform
Spark
Build Management
Data Lake
PySpark
Information Technology
Collibra
Kafka
Machine Learning Operations
Data Lakehouse
Cloud Migration
Cloud Optimization
Terraform
Data Pipelines
Databricks
Job description
We are seeking a hands-on Data Architect with deep technical expertise in Azure and Databricks to design and build enterprise data platforms and products. This role requires a unique blend of architectural vision and hands-on implementation skills - you'll design solutions AND write code to bring them to life. Key Responsibilities Architecture & Design
- Design scalable, secure, and cost-effective data architectures on Azure and Databricks
- Define data platform standards, patterns, and best practices
- Create reference architectures for data lakehouse implementations
- Design end-to-end data product architectures from ingestion to consumption
Hands-On Implementation
- Build proof-of-concepts and reference implementations personally
- Write production-grade Spark code (PySpark, Scala, SQL)
- Implement complex data pipelines and transformations
- Configure and optimize Databricks environments and Azure services
- Lead by example through hands-on technical contributions
Data Platform Development
- Design and implement data lakehouse architectures using Delta Lake
- Architect streaming and batch processing solutions at scale
- Build data governance frameworks using Unity Catalog and Azure Purview
- Design data quality and observability solutions
Technical Leadership
- Provide technical guidance to data engineering teams
- Conduct architecture reviews and code reviews
- Mentor engineers on Databricks and Azure best practices
- Evaluate and recommend new technologies and tools
Stakeholder Engagement
- Translate business requirements into technical architectures
- Present architectural designs to technical and business stakeholders
- Collaborate with security, infrastructure, and platform teams
- Drive architectural decisions and alignment across teams, * Azure (comprehensive knowledge across data services)
- Databricks (expert-level implementation and optimization)
- Apache Spark (architecture, tuning, advanced optimization)
- PySpark and/or Scala (production-level coding)
- SQL (complex queries, performance tuning)
- Data architecture patterns and best practices
What Makes This Role Unique This is NOT a traditional "ivory tower" architect role. We need someone who:
- Designs architectures in the morning and implements them in the afternoon
- Gets into the weeds with Spark optimization and cluster tuning
- Can architect a solution AND build a working prototype
- Debugs production issues alongside the engineering team
- Writes code daily, not just PowerPoint slides
Requirements
- 8+ years of data engineering/architecture experience
- Deep hands-on expertise in Databricks (Delta Lake, Unity Catalog, Medallion architecture, cluster optimization)
- Extensive Azure experience (Data Lake Storage, Data Factory, Synapse Analytics, Event Hubs, Purview)
- Expert-level Apache Spark skills (performance tuning, optimization, architecture)
- Strong programming skills in PySpark, Scala, and/or Python
- Advanced SQL knowledge and query optimization
Architecture Experience
- Proven track record designing and implementing large-scale data platforms
- Experience with data lakehouse, data mesh, or modern data stack architectures
- Deep understanding of data modeling techniques (dimensional, normalized, data vault)
- Knowledge of real-time streaming architectures (Kafka, Event Hubs, Structured Streaming)
- Experience with data governance, security, and compliance frameworks
Technical Depth
- THIS IS NOT A PURE ARCHITECTURE ROLE - Daily coding and hands-on work required
- Ability to debug complex Spark jobs and optimize performance
- Experience with infrastructure as code (Terraform, ARM templates)
- Understanding of DevOps practices and CI/CD for data pipelines
- Knowledge of data quality, testing, and observability frameworks, * Azure Solutions Architect Expert certification
- Databricks Certified Data Engineer Professional or Solutions Architect
- Experience with machine learning platforms and MLOps
- Background in cloud migration and modernization projects
- Knowledge of FinOps and cloud cost optimization
- Experience with multiple cloud platforms (AWS, Google Cloud Platform)
- Previous consulting or advisory experience, * Senior engineer who's ready to take on architecture responsibilities
- Or architect who refuses to stop coding and building
- Believes the best way to validate an architecture is to implement it
- Values pragmatic solutions over theoretical perfection, * Bachelor's degree in Computer Science, Engineering, Mathematics, or related field
- Master's degree or equivalent experience preferred