> Markdown version of [/jobs/ext/3400287-grocery-domain-senior-databricks-architect](https://www.wearedevelopers.com/jobs/ext/3400287-grocery-domain-senior-databricks-architect). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Grocery Domain -Senior Databricks Architect

- **Company:** Compunnel Inc.
- **Location:** San Jose, CA, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Amazon S3, Cloud Storage, Cluster Analysis, Code Review, Databases, Data Architecture, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Transformation, Data Migration, Data Security, Data Systems, Software Design Patterns, Key Management, Enterprise Messaging Systems, Performance Tuning, Cloud Services, Azure Data Lake, SQL Databases, Talend, Data Logging, Data Processing, Enterprise Software Applications, Data Ingestion, Azure Data Factory, Apache Spark, Data Lakes, Pyspark, Semi-structured Data, Apache Kafka, Software Coding, Restful APIs, Data Pipelines, Serverless Computing, Databricks
- **Published:** September 22, 2026
- **Apply:** https://www.dice.com/job-detail/a5d71b79-2725-4283-bbc4-989d34282e0a

## About the Role

10+ years of experience
Bachelor's Degree

Skills
Databricks
Apache Spark
PySpark
SQL
Delta Lake
Medallion Architecture
Unity Catalog
Talend
CDC and SCD Implementation
Cloud Data Platforms
Azure Data Lake Storage (ADLS) or S3
Azure Data Factory
Kafka
REST APIs and Cloud-Native Services
Batch and Incremental Data Processing
Data Security and Access Control
Monitoring and Performance Management
Data Migration Strategy
ETL/ELT Architecture
Data Modeling
Data Governance
Data Quality and Reconciliation Frameworks
Retail/Grocery Domain Expertise
Technical Leadership
Mentoring and Coaching
Technical Documentation
Stakeholder Management
Code Review Facilitation
Performance Optimization
Production Support and Troubleshooting
Cross-functional Collaboration

## Description

Lead the end-to-end architecture for migrating Talend ETL/ELT workloads to Databricks
Define target-state Databricks Lakehouse architecture, including data ingestion, transformation, storage, orchestration, governance, and consumption layers
Design scalable, secure, highly available, and cost-optimized data solutions
Define architecture standards, design patterns, coding standards, and best practices for Databricks development
Establish reusable migration frameworks and patterns for converting Talend jobs into Databricks/Spark-based pipelines
Evaluate existing Talend jobs and determine appropriate migration strategies, including re-platforming, re-engineering, consolidation, or retirement
Analyze Talend workflows, jobs, mappings, dependencies, schedules, and data transformations
Develop migration strategies for batch and incremental data processing workloads
Translate Talend transformations and business rules into PySpark, SQL, and Databricks implementations
Identify opportunities to simplify and optimize legacy ETL processes during migration
Define data reconciliation and validation strategies to ensure parity between Talend and Databricks outputs
Establish migration sequencing based on business criticality, dependencies, complexity, and risk
Support migration of high-volume and business-critical data pipelines
Design and implement solutions using Databricks, Apache Spark, Delta Lake, PySpark, and SQL
Design Medallion Architecture using Bronze, Silver, and Gold layers
Implement incremental processing, CDC, SCD Type 1/Type 2, data quality, error handling, and audit frameworks
Optimize Spark workloads, Delta tables, SQL queries, partitioning, clustering, and job execution
Leverage Databricks Workflows/Jobs, Unity Catalog, and modern Databricks data engineering capabilities
Design data pipelines integrating structured and semi-structured data from databases, files, APIs, and enterprise applications
Establish monitoring, logging, alerting, operational support, and performance management practices
Design integrations between Databricks and cloud storage, databases, messaging platforms, APIs, and enterprise applications
Define secure connectivity, networking, secrets management, and access-control patterns
Work with business and technology stakeholders to understand Retail/Grocery data requirements and translate them into scalable data architecture
Provide technical leadership and mentoring to data engineers and developers
Conduct architecture, design, and code reviews
Troubleshoot complex production and performance issues
Collaborate with onshore/offshore teams, enterprise architects, infrastructure teams, security teams, and business stakeholders
Prepare architecture diagrams, technical design documents, migration plans, and implementation standards
Communicate technical risks, dependencies, and recommendations to client leadership