Data Engineer

Spectraforce
Richmond, United States of America
1 month ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English

Job location

Richmond, United States of America

Tech stack

Java
Agile Methodologies
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Automation of Tests
Cloud Computing Security
Code Review
Continuous Integration
Information Engineering
ETL
Data Security
Dataspaces
Data Systems
Software Debugging
DevOps
Distributed Computing Environment
Identity and Access Management
Python
Operational Databases
Performance Tuning
Cloud Services
SQL Databases
Scripting (Bash/Python/Go/Ruby)
Freeform SQL
Snowflake
Spark
State Machines
Backend
GIT
PySpark
Cloudwatch
Data Pipelines
Serverless Computing
Go

Job description

*Design, develop, and maintain data pipelines leveraging Python, Spark/PySpark, and cloud-native services. *Build and optimize data workflows, ETL processes, and transformations for large-scale structured and semi-structured datasets. *Write advanced and efficient SQL queries against Snowflake, including joins, window functions, and performance tuning. *Develop backend and automation tools using Golang and/or Python as needed. *Implement scalable, secure, and high-quality data solutions across AWS servies such as S3, Lambda, Glue, Step Functions, EMR, and CloudWatch. *Troubleshoot complex production data issues, including pipeline failures, data quality gaps, and cloud environment challenges. *Perform root-cause analysis and implement automation to prevent recurring issues. *Collaborate with data scientists, analysts, platform engineers, and product teams to enable reliable, high-quality data access. *Ensure compliance with enterprise governance, data quality, and cloud security standards. *Participate in Agile ceremonies, code reviews, and DevOps practices to ensure high engineering quality.

Requirements

*Skills-Data Engineer- Python , Spark/PySpark, AWS, Golang, Able to write complex SQL queries against Snowflake tables / Troubleshoot issues, Java/Python, AWS (Glue, EC2, Lambda). *Proficiency in Python with experience building scalable data pipelines or ETL processes. *Strong hands-on experience with Spark/PySpark for distributed data processing. *Experience writing complex SQL queries (Snowflake preferred), including optimization and performance tuning. *Working knowledge of AWS cloud services used in data engineering (S3, Glue, Lambda, EMR, Step Functions, CloudWatch, IAM). *Experience with Golang for scripting, backend services, or performance-critical processes. *Strong debugging, troubleshooting, and analytical skills across cloud and data ecosystems. *Familiarity with CI/CD workflows, Git, and automated testing.

Apply for this position