> Markdown version of [/jobs/ext/4026374-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/4026374-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** Ravh IT Solutions - **Location:** Irvine, CA, United States - **Experience:** Expert - **Contract:** Contract - **Skills:** Application Programming Interfaces (APIs), Amazon Web Services, Amazon S3, Automation of Tests, Microsoft Azure, Big Data, Cloud Computing, Cloud Database, Cloud Storage, Code Review, Collaborative Software, Continuous Integration, Data Architecture, Data Validation, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Transformation, Data Security, Data Systems, Data Warehousing, Relational Databases, Database Development, Database Queries, Dimensional Modeling, Distributed Computing Environment, Distributed Systems, Python (Programming Language), Meta-Data Management, Performance Tuning, Query Optimization, Azure Data Lake, SQL Databases, Unstructured Data, Management of Software Versions, Workflow Management Systems, Enterprise Data Management, Data Processing, Google Cloud, Cloud Platform System, Data Ingestion, Azure Data Factory, Apache Spark, Change Data Capture, Git, Microsoft Fabric, Data Lakes, Pyspark, Information Technology, Optimization Algorithms, AWS Glue, Apache Kafka, Spark Streaming, Data Lakehouse, Software Version Control, Data Pipelines, Databricks - **Published:** October 30, 2026 - **Apply:** https://www.dice.com/job-detail/e7748f90-0365-417d-bdb5-8ba82f1e7f60 ## About the Role We are seeking a highly experienced Senior Data Engineer with 10+ years of experience in data engineering and strong hands-on expertise in Databricks, PySpark, Python, SQL, and large-scale data processing. The ideal candidate will have extensive experience designing, developing, and optimizing enterprise-level data pipelines, implementing complex ETL/ELT processes, and building scalable data engineering solutions. The candidate must possess strong technical expertise in Apache Spark, distributed computing, data transformation, and modern data lakehouse architectures. The ability to develop production-ready data pipelines, troubleshoot complex data processing challenges, and optimize performance across large datasets is essential., * Databricks: Extensive hands-on experience developing and managing enterprise data engineering solutions. * PySpark: Strong expertise in distributed data processing, complex transformations, Spark DataFrames, and performance tuning. * Python: Advanced programming skills for data engineering, automation, and pipeline development. * SQL: Strong proficiency in complex queries, joins, aggregations, window functions, and query optimization. * Apache Spark: Deep understanding of Spark architecture, execution plans, partitioning, shuffling, caching, and optimization techniques. * Delta Lake: Hands-on experience with Delta Tables, ACID transactions, schema evolution, and lakehouse architecture. * ETL/ELT: Extensive experience designing and developing production-grade data pipelines. * Cloud Platforms: Strong experience with at least one major cloud platform: Microsoft Azure, AWS, or Google Cloud Platform. * Data Modeling: Experience with data warehousing, dimensional modeling, and large-scale data architecture. * Workflow Orchestration: Experience scheduling, monitoring, and managing complex data workflows. * Version Control: Experience with Git and collaborative software development practices., * Experience with Azure Databricks, Azure Data Lake Storage (ADLS), Azure Data Factory, or Microsoft Fabric. * Experience with AWS services such as Amazon S3, AWS Glue, or other cloud-based data processing services. * Experience with Spark Structured Streaming and Kafka. * Strong understanding of incremental loads, Change Data Capture (CDC), and Slowly Changing Dimensions (SCD). * Experience with CI/CD pipelines, automated testing, and deployment processes. * Knowledge of data governance, metadata management, and enterprise data security. * Experience supporting and optimizing mission-critical data pipelines in production environments. Candidate Requirements * 10+ years of overall experience in data engineering or closely related data development roles. * Strong, hands-on experience with Databricks and PySpark. * Proven experience building and maintaining complex, production-grade data pipelines. * Strong knowledge of Python, SQL, Apache Spark, and cloud-based data platforms. * Excellent analytical, troubleshooting, and problem-solving skills. * Ability to work independently and collaborate effectively with cross-functional teams. Education Bachelor's degree in Computer Science, Information Technology, Engineering, or a related technical field, or equivalent practical experience. Key Skills Databricks | PySpark | Python | SQL | Apache Spark | Delta Lake | Delta Tables | ETL/ELT | Data Pipelines | Data Engineering | Azure Databricks | AWS | Google Cloud Platform | Data Lakehouse | Data Warehousing | Spark Optimization | Cloud Data Engineering ## Description * Design, develop, and maintain scalable, high-performance data pipelines using Databricks, PySpark, Python, and SQL. * Build, implement, and optimize complex ETL/ELT workflows to process large volumes of structured and unstructured data. * Develop advanced data transformations and processing logic using PySpark and Apache Spark. * Design and implement scalable data lakehouse architectures using Databricks and Delta Lake. * Optimize Spark jobs, cluster configurations, partitioning, caching, and query execution for improved performance. * Develop data ingestion frameworks to integrate data from multiple sources, including relational databases, APIs, files, and cloud storage. * Implement incremental data processing, Change Data Capture (CDC), and efficient data-loading strategies. * Develop and manage automated workflows using Databricks Workflows and other orchestration tools. * Implement data quality checks, validation frameworks, error handling, and monitoring mechanisms. * Work with Delta Tables, schema evolution, ACID transactions, and data versioning. * Collaborate with data architects, business stakeholders, data scientists, and engineering teams to deliver enterprise-grade data solutions. * Troubleshoot complex production issues and ensure the reliability, scalability, and availability of data pipelines. * Implement data security, governance, access controls, and engineering best practices. * Participate in technical design discussions, code reviews, and performance optimization initiatives.