Senior Data Engineer

Intersources Inc.

Modesto, CA, United States

4 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

5 years minimum

Working hours

Regular working hours

Job source

www2.jobdiva.com

Tech stack

Airflow Amazon Web Services Data Validation Information Engineering Extract Transform Load (ETL) Data Warehousing Relational Databases Python (Programming Language) SQL Databases Data Streaming Working Model 2D Google Cloud

+6 more

Azure Data Factory Information Technology Luigi Data Pipelines Programming Languages Control M

Job description

Design, build and launch efficient & reliable data pipelines to move and transform data (both large and small amounts)
Apply proven expertise and build high-performance scalable data warehouses
Securely source external data from numerous partners
Intelligently design data models for optimal storage and retrieval
Deploy inclusive data quality checks to ensure high quality of data
Optimize existing pipelines and maintain of all domain-related data pipelines
Ownership of the end-to-end data engineering component of the solution
Experience in creation of data pipelines using Python.
Python for data manipulation & transformation (python dictionaries, data frames, data stream, joins of all kinds, outside of SQL IDEs)
Research skills - go figure it out and come back with working model. Pros/cons analysis skills

Requirements

Strong work ethic, independent problem-solving skills, self-motivated, comfortable with startup culture
Excellent and crisp communicator
Must have domain experience in supply chain (E-commerce /manufacturing etc), * BS/MS in Computer Science or a related technical field
10+ years of Python or other modern programming language development experience
10+ years of SQL and relational databases experience
10+ years experience in custom ETL design, implementation and maintenance
5+ years of experience with workflow management engines (i.e. Airflow, Luigi, Prefect, Dagster, Digdag, Google Cloud Composer, AWS Step Functions, Azure Data Factory, UC4, Control-M)