> Markdown version of [/jobs/ext/245492-databricks-engineer](https://www.wearedevelopers.com/jobs/ext/245492-databricks-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Databricks Engineer - **Company:** REVSTAR MEDIA INC. - **Location:** United States (Remote available) - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Agile Methodology, Artificial Intelligence, Airflow, Amazon S3, Big Data, Cloud Engineering, Cloud Storage, Data Architecture, Data Validation, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Security, Data Systems, DevOps, Python (Programming Language), Machine Learning, Meta-Data Management, Performance Tuning, Azure Data Lake, SQL Databases, Data Streaming, Workflow Management Systems, Feature Engineering, Data Ingestion, Apache Spark, Data Lakes, Infrastructure Automation Frameworks, Data Lineage, Low Latency, Integration Frameworks, Apache Kafka, Data Management, Machine Learning Operations, Terraform, Software Version Control, Data Pipelines, Databricks - **Published:** May 16, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=e578397657651918 ## About the Role Do you have experience in Spark?, This is a technical hands-on role, requiring expertise in Apache Spark, Delta Lake, and MLOps, as well as experience working with large-scale data architectures. You will collaborate with architects and business stakeholders to ensure solutions align with customer needs and best practices., Must-Have: * 3+ years of hands-on experience in data engineering, with a focus on big data processing and cloud-native architectures. * 2+ years of hands-on experience with Databricks, including Apache Spark, Delta Lake, and MLflow. * Databricks Certifications (Mandatory): * + Databricks Certified Data Engineer Associate (or higher) * Proficiency in Python, SQL, and Spark-based frameworks. * Experience in developing and optimizing large-scale ETL/ELT pipelines. * Strong understanding of Lakehouse architecture and cloud-agnostic data solutions. * Familiarity with CI/CD pipelines and Infrastructure-as-Code (IaC) for Databricks (e.g., Terraform, Databricks CLI). * Knowledge of data governance, security, and compliance best practices. * Experience working in Agile development environments, following DevOps/MLOps best practices. Nice-to-Have: * Additional Databricks Certifications (e.g., Databricks Certified Machine Learning Associate). * Experience with real-time streaming solutions (e.g., Kafka, Kinesis, Event Hub). * Familiarity with cloud storage and orchestration tools (e.g., Apache Airflow, Prefect). * Background in AI/ML integration within Databricks, assisting in feature engineering and model deployment. * Experience working in client-facing roles or consulting environments. ## Description We are seeking a highly skilled Databricks Engineer to join our team. This role will be hands-on, working closely with architects, data scientists, and customers to build, optimize, and deploy high-performance data and AI solutions., As a Databricks Engineer, you will be responsible for building and optimizing data pipelines, implementing data processing frameworks, and enabling AI/ML solutions within Databricks. You will work across data ingestion, transformation, and orchestration while ensuring scalability, performance, and security., Data Engineering & Pipeline Development * Develop and optimize data pipelines using Apache Spark and Delta Lake within Databricks. * Implement ETL/ELT workflows, ensuring efficient data ingestion, transformation, and storage. * Design Lakehouse architecture-based solutions that scale across structured and unstructured data sources. * Integrate Databricks with cloud storage solutions (Azure Data Lake, AWS S3, Google Cloud Storage) for seamless data management. Performance Optimization & Automation * Optimize Spark jobs for scalability, cost efficiency, and low latency. * Implement monitoring and alerting solutions to track job performance and detect failures. * Develop automated data validation, testing, and quality assurance processes. Management AI/ML Integration & MLOps Support * Support ML model training and deployment within Databricks, integrating with MLflow for experiment tracking and model versioning. * Collaborate with data scientists and ML engineers to enable scalable AI solutions. * Implement feature engineering pipelines and integrate models into production environments. Security, Governance & Best Practices * Ensure data security, access control, and compliance with industry standards (GDPR, HIPAA, SOC 2, etc.). * Follow Databricks best practices for data lineage, governance, and metadata management. * Document processes, configurations, and best practices for internal and client use. ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) ## Related Articles - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)