Azure Databricks Developer

Gurus Infotech, Inc.
Louisville, United States of America
3 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Louisville, United States of America

Tech stack

Java
Azure
Big Data
Cloud Computing
Data Governance
ETL
Data Stores
Data Warehousing
Hadoop
Hive
Python
Meta-Data Management
NoSQL
Cloudera
Azure
Scala
Enterprise Data Management
Data Processing
Informatica Powercenter
Delivery Pipeline
Spark
Data Lake
Information Technology
Data Pipelines
Databricks

Job description

  1. The Senior Data Engineer will be responsible for the build of Enterprise Data platform.

  2. Setting up the data pipelines that are scalable, robust and resilient and build pipelines to validate, ingest, normalize/enrich and business-specific processing of healthcare data.

  3. Build Azure Data Lake leveraging Databricks technology to consolidate all data across the company and serve the data to various products and services.

  4. The scope of this role will include working with engineering, product management, program management and operations teams in delivering pipeline platform, foundational and application-specific pipelines and building the Data Lake in collaboration with business and other teams. Responsibilities

  5. Design, Develop, Operate and drive scalable and resilient data platform to address the business requirements

  6. Drive technology and business transformation through the creation of the Azure Data Lake

  7. Ensure industry best practices around data pipelines, metadata management, data quality, data governance and data privacy

  8. Partner with Product Management and Business leaders to drive Agile delivery of both existing and new offerings; assist with the Leadership and collaboration with engineering organizations within Change to manage and optimize the portfolio

Requirements

  1. 10+ years working experience in Data processing / ETL / Big data technologies like Informatica, Hadoop, Cloudera

  2. 4+ years working experience in Databricks (essential) and Python

  3. Experience with Cloud / Azure architectural components

  4. Experience building data pipelines and infrastructure

  5. Deep understanding of Data warehousing concepts, reporting and Analytical concepts

  6. Experience with Big Data tech stack, including Hadoop, Java, Spark, Scala, and Hive, NoSQL data stores

  7. Bachelor s degree in Mathematics, Physical Sciences, Engineering or Computer Science Nice to Have skills

  8. Experience in leading design and development of large systems

  9. Demonstrates strong drive to learn and advocate for development best practices

  10. Proven track record of building and delivering enterprise-class products

  11. Full Stack Experience (End to End Development)

  12. Crisp and effective executive communication skills, including significant experience presenting cross-functionally and across all levels

Apply for this position