Technical Lead - DataBricks

We are looking for a highly skilled and hands-on Databricks Lead Data Engineer to design, build, optimize, and manage scalable data engineering solutions using Databricks, Apache Spark, PySpark, SQL, and cloud-based data platforms. The ideal candidate should have strong technical leadership capabilities along with deep hands-on experience in developing production-grade data pipelines, lakehouse architectures, and enterprise data solutions., * Lead the design, development, and implementation of scalable data pipelines and data lakehouse solutions using Databricks, PySpark, Spark SQL, and Delta Lake.

• Work hands-on in building batch and streaming ETL/ELT pipelines from multiple source systems into cloud data platforms.
• Design and implement medallion architecture layers such as Bronze, Silver, and Gold for efficient data processing and analytics consumption.
• Optimize Spark jobs, Databricks notebooks, clusters, workflows, and SQL queries for performance, reliability, and cost efficiency.
• Collaborate with business stakeholders, architects, data analysts, and data scientists to understand requirements and translate them into robust technical solutions.
• Provide technical leadership, code reviews, best practices, and mentoring support to junior and mid-level data engineers.
• Implement data quality checks, data validation rules, monitoring, logging, error handling, and restartability mechanisms.
• Ensure adherence to data governance, security, access control, and compliance standards across data engineering solutions.
• Support production deployments, troubleshoot pipeline failures, perform root cause analysis, and drive continuous improvement.

• Strong hands-on experience in Databricks development, including notebooks, workflows, jobs, clusters, Delta Lake, and Unity Catalog.
• Advanced programming experience in PySpark, Python, Spark SQL, and SQL.
• Strong understanding of data engineering concepts, data warehousing, data lakehouse architecture, ETL/ELT design, and data modeling.
• Experience in building scalable data pipelines on cloud platforms such as Azure, AWS, or GCP.
• Hands-on experience with orchestration tools such as Azure Data Factory, Airflow, Databricks Workflows, or similar tools.
• Experience working with file formats such as Parquet, Avro, JSON, CSV, and Delta format.
• Good understanding of performance tuning techniques including partitioning, caching, broadcast joins, cluster sizing, compaction, and query optimization.
• Experience with CI/CD, Git, DevOps practices, release management, and environment migration.
• Strong troubleshooting, analytical, problem-solving, and communication skills.
• Experience with Delta Live Tables, Structured Streaming, Auto Loader, Unity Catalog, and Databricks SQL.
• Exposure to data governance, lineage, cataloging, masking, encryption, and role-based access control.
• Experience in migration from legacy ETL tools or on-premise data warehouses to Databricks lakehouse platform.
• Knowledge of cloud storage services such as ADLS, Blob Storage, S3, or Google Cloud Storage.
• Experience in BFSI, healthcare, retail, or large enterprise data platform environments is an added advantage.
• Databricks, Azure, AWS, or data engineering certifications are preferred.

Mandatory Skills: DataBricks - Data Engineering .

Experience: 5-8 Years .

3.83.8 out of 5 stars Minneapolis, MN 55401 $60,000 - $135,000 a year, Pulled from the full job description

• Health insurance
• Paid time off
• Dental insurance
• Disability insurance, The expected compensation for this role ranges from $60,000 to $135,000 .

Final compensation will depend on various factors, including your geographical location, minimum wage obligations, skills, and relevant experience. Based on the position, the role is also eligible for Wipro’s standard benefits including a full range of medical and dental benefits options, disability insurance, paid time off (inclusive of sick leave), other paid and unpaid leave options.

Wipro Limited (NYSE: WIT, BSE: 507685, NSE: WIPRO) is a leading technology services and consulting company focused on building innovative solutions that address clients’ most complex digital transformation needs. Leveraging our holistic portfolio of capabilities in consulting, design, engineering, and operations, we help clients realize their boldest ambitions and build future-ready, sustainable businesses. With over 230,000 employees and business partners across 65 countries, we deliver on the promise of helping our customers, colleagues, and communities thrive in an ever-changing world. For additional information, visit us at www.wipro.com.