Lead Data Engineer

Insight Global
San Francisco, United States of America
1 month ago

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

San Francisco, United States of America

Tech stack

API
Amazon Web Services (AWS)
Data analysis
Azure
Cloud Computing
Computer Programming
Data as a Services
Data Validation
Information Engineering
ETL
Data Warehousing
IBM InfoSphere DataStage
Hive
IBM Cognos Business Intelligence
Python
NoSQL
Raw Data
Software Engineering
SQL Databases
SQL Server Analysis Services
Data Streaming
Tableau
Spark
Kafka
Data Pipelines
Databricks

Requirements

Must Haves:

  • At least 5 years of strong hands-on expertise with Databricks and Datastage (or comparable ETL tools)

  • Experience with migrations

  • Exs: Migrating SQL to cloud, SQL to Azure Databricks, (on prem to the cloud)

  • 8-10 years of software development and data engineering experience with SQL, ADF, SSAS cubes, Cognos, Tableau, ThoughtSpot and other BI tools

  • Advanced SQL skills for processing raw data, Kafka ingestions, ADF pipelines, data validation and QA with experience across SQL and NoSQL databases

  • Programming experience with Python and/or Scala

  • Experience ingesting data via APIs and streaming sources

  • Experience building data pipelines, data warehouses, and analytics platforms

  • Proficiency with Spark, Hive, Kafka, and cloud data services

  • Experience with Azure, AWS or equivalent cloud platforms

Apply for this position