Python/ Aws Data engineer

NimbusAITech LLC
Richmond, VA, United States
6 days ago
Apply on www.dice.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
3 years minimum
Working hours
Regular working hours
Job source

Tech stack

Amazon Web Services Big Data Code Review Information Engineering Extract Transform Load (ETL) Python (Programming Language) Standard Sql Software Engineering Data Ingestion Snowflake Apache Spark Software Application Programming
+6 more
Backend Pyspark AWS Glue Real Time Data Data Management Data Pipelines

Job description

Client is seeking a Python Data Engineer to support a technology initiative with one of our long-standing financial services clients. This individual will join an established engineering team supporting a real-time data platform and the data pipelines and infrastructure surrounding it., · Design, develop, automate, and support Python-based data pipelines and ETL processes

· Ingest, transform, and integrate data from external source systems into the client’s technology ecosystem

· Develop and maintain data processing solutions using Python, PySpark, AWS Glue, and Spark

· Automate and improve existing data ingestion and conversion processes

· Build and maintain backend infrastructure and pipelines supporting existing real-time data models

· Support platform stability, including technical and network-related upgrades

· Work independently while collaborating closely with the Tech Lead and other engineers

· Proactively identify and communicate technical risks, issues, and opportunities for improvement

· Develop maintainable code and participate in testing and code reviews

Requirements

The Python Data Engineer will focus on building, automating, and improving data ingestion and ETL processes, including integrating historical data from external systems into the client’s technology ecosystem. The ideal candidate will have strong hands-on experience with Python, PySpark, AWS Glue, Spark, and data pipelines and be comfortable working independently within an established engineering team., · At least 3 years of experience in software development, data engineering, or application development

· Experience developing applications and/or data pipelines using Python

· Experience with PySpark and Spark

· Experience with AWS, including AWS Glue

· Experience building or maintaining ETL, data ingestion, or data pipeline solutions

· Experience working with SQL

Preferred Qualifications

· 3-7+ years of relevant software or data engineering experience

· Strong hands-on experience with Python, PySpark, AWS Glue, and Spark

· Experience designing, automating, and supporting large-scale data ingestion and ETL pipelines

· Experience with asyncio or asynchronous processing in Python

· Experience with Snowflake

· Experience supporting backend infrastructure or data platforms in a production environment

· Experience working with real-time data systems

· Strong communication skills with the ability to work independently and proactively surface risks and issues

· Previous experience working within a large enterprise or financial services environment

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:28 min

Defining big data and machine learning fundamentals

Ayon Roy · LIVE

3:09 min

Balancing data science skillings alongside systems engineering rigor

Nico Schmidt · LIVE

1:52 min

Structuring and scaling the backend engineering team

Stefan Lingler Stefan Lingler +1 · Coffee With Developers

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

2:10 min

Why organizations combine big data and machine learning

Ayon Roy · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

Videos

See all

Related articles

See all