Senior Data Engineer in McLean

Energy Jobline
Richmond, VA, United States
3 days ago
Apply on www.energyjobline.com
Prepare application

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Working hours
Regular working hours

Tech stack

Amazon Web Services Amazon S3 Big Data Databases Continuous Integration Data Integrity Relational Databases DevOps Python (Programming Language) PostgreSQL MySQL Real Time Systems
+6 more
Pyspark Data Lineage Apache Kafka Data Delivery Data Pipelines Amazon Elastic Mapreduce (EMR)

Requirements

n

Core : Expert-level proficiency in Python and PySpark.

\n

AWS Ecosystem: Deep hands-on experience with AWS Glue and Amazon EMR.

\n

Big Data Architecture: Proven track record of building scalable pipelines for large-scale data (Terabyte-scale ).

\n

Data Integrity: Demonstrated experience implementing frameworks for Data Lineage and Data Quality.

\n

Database Knowledge: Strong experience working with S3-based lakes and relational databases (RDS/PostgreSQL/MySQL)., n

Industry: Experience within the Financial Services industry is a huge plus.

\n

Real-time Processing: Experience with Kinesis or Kafka integration for low-latency data delivery.

\n

DevOps: Familiarity with CI/CD for data pipelines and Infrastructure as Code.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.energyjobline.com
Prepare application

Good distractions

Loading talks and stories from around this role…