> Markdown version of [/jobs/ext/1649354-data-engineer](https://www.wearedevelopers.com/jobs/ext/1649354-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** Horizontal Talent - **Location:** San Francisco, CA, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Java (Programming Language), Agile Methodology, Artificial Intelligence, Amazon Web Services, Business Analytics Applications, Microsoft Azure, Big Data, Continuous Integration, Information Engineering, Data Governance, Data Infrastructure, Extract Transform Load (ETL), Dataspaces, Data Systems, DevOps, Distributed Data Store, Distributed Systems, Apache Hadoop, Python (Programming Language), NoSQL, Cloud Services, SQL Databases, Data Streaming, Data Processing, Google Cloud, Azure Data Factory, Apache Spark, Apache Flink, Integration Frameworks, Apache Kafka, Dynamic Data, Machine Learning Operations, Data Pipelines, Databricks - **Published:** July 21, 2026 - **Apply:** https://www.dice.com/job-detail/0310daaf-8714-4707-b39e-2ce11fb8ad60 ## About the Role * 8+ years of experience in data engineering with a strong big data background. * Hands-on experience with Databricks and Azure Data Factory. * Experience with DevOps tools and practices related to data engineering and CI/CD. * Strong working knowledge of Hadoop, Spark, Kafka, or similar distributed data technologies. * Proficiency in Python, Scala, or Java. * Solid understanding of SQL and NoSQL databases. * Experience building ETL processes, data pipelines, and real-time or batch data solutions. * Strong problem-solving skills and the ability to communicate effectively with technical and business stakeholders. Preferred Skills * Experience with cloud data platforms such as Azure, AWS, or Google Cloud Platform. * Knowledge of Flink or other streaming/data processing frameworks. * Familiarity with data modeling and distributed computing concepts. * Background supporting machine learning pipelines or AI-enabled data solutions. * Experience in Agile environments and with team leadership or mentoring. * Knowledge of California emissions-related data or compliance work is a plus. Horizontal Talent is committed to fostering an inclusive workplace where diverse perspectives are valued and everyone has the opportunity to thrive. We welcome candidates of all backgrounds and experiences who are excited to contribute to a collaborative, respectful, and equitable environment. ## Description Join a dynamic data engineering team supporting large-scale analytics and cloud data solutions for a hybrid onsite role in San Francisco, CA. This position focuses on building reliable, high-performance pipelines and modern data platform capabilities using Databricks, Azure Data Factory, and DevOps practices. Responsibilities * Design, build, and maintain scalable data pipelines and analytics solutions for big data environments. * Develop and optimize workflows that support efficient, reliable data processing and delivery. * Work closely with cross-functional partners to translate data needs into actionable technical solutions. * Support data platform operations with a focus on quality, availability, performance, and stability. * Apply cloud and DevOps practices to automate data workflows and improve delivery efficiency. * Troubleshoot issues across data pipelines and environments to help ensure smooth day-to-day operations. * Contribute to data governance, security, and compliance practices within the data ecosystem. * Share knowledge and help mentor others while promoting collaborative ways of working. ## Related Videos - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)