Senior Data Engineer

Nucorevision, Inc.

Washington, DC, United States

3 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

10 years minimum

Working hours

Regular working hours

Job source

indeed.com

Tech stack

Microsoft Azure Big Data Data Files Extract Transform Load (ETL) Data Sharing Data Structures Relational Databases Apache Hadoop Python (Programming Language) Open Source Technology Scala (Programming Language) SQL Databases

+7 more

Data Streaming Apache Spark Microsoft Fabric Information Technology Data Management Data Pipelines Databricks

Job description

Create and maintain optimal data pipeline architecture, including optimal extraction, transformation and loading of data, leveraging Microsoft Fabric notebooks, Apache Spark (SQL, Python, Scala) and other Microsoft Azure or open-source tools as required. Maintain high quality documentation about data structures in shared data governance tools for the business to access. Collaborate with data owners, data stewards, analysts, software developers, and Database Administrators (DBAs). Experience with Apache Spark, Python, SQL or relational databases, Hadoop, Databricks and data pipeline management in Spark for batch and streaming. Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.

Requirements

Do you have experience in Technical documentation?, Do you have a Master’s degree?, * Must be a US citizen

Must Reside in the DMV Area (DC, Maryland, Virginia)
Must have a minimum of 10-years Experience
Master’s Degree Highly Preferred, * Years of Experience: 10+ years
Recommended Certifications: Power Platform certifications preferred.
Relevant federal IT experience, technical capability, and operational support experience aligned to assigned labor category.