Data Engineer - Databricks (Newcastle or Erskine)
Role details
Job location
Tech stack
Job description
We're on a mission to harness the power of data to drive innovation and deliver transformative solutions. We're looking for a talented Data Engineer with deep expertise in Databricks to join our growing team and help shape the future of our data projects. This is a fantastic opportunity to work on cutting-edge AI and data engineering projects using the latest tools and technologies in a collaborative and innovative environment., * Build and maintain scalable data pipelines using Databricks.
- Manage and optimize Databricks Workspace, Clusters, Jobs, Repos, and Delta Live Tables.
- Develop modular ETL workflows using dbt and orchestrate them with Apache Airflow.
- Write advanced PySpark code, including UDFs and Pandas UDFs, for efficient data processing.
- Ensure data quality, reliability, and performance across all systems.
- Collaborate with data scientists, analysts, and engineers to deliver data solutions that support business goals.
- Contribute to the mentoring and development of junior team members.
- Support senior team members in identifying and addressing data science opportunities., * Work with a forward-thinking team on impactful AI projects
- Access to continuous learning and development opportunities
- Flexible working arrangements
- Inclusive and supportive company culture
At DXC Technology, we believe strong connections and community are key to our success. Our work model prioritizes in-person collaboration while offering flexibility to support wellbeing, productivity, individual work styles, and life circumstances. We're committed to fostering an inclusive environment where everyone can thrive.
Requirements
- Proven experience with Databricks and its ecosystem
- Strong proficiency in PySpark, especially with UDFs and Pandas UDFs
- Hands-on experience with dbt and Airflow
- Databricks Certified Data Engineer Associate certification or willingness to achieve one
- Excellent problem-solving skills and a passion for data and AI
- Bachelor's degree in a relevant field or equivalent combination of education and experience
- Typically, 6+ years of relevant work experience in industry, with a minimum of 2+ years in a similar role.
- Proficiencies in data cleansing, exploratory data analysis, and data visualization
- Continuous learner that stays abreast with industry knowledge and technology