Sr Data Engineer
Spectraforce
McLean, VA, United States
2 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on leoforce.us
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Working hours
Regular working hours
Job source
Tech stack
Artificial Intelligence
Amazon Web Services
Application Integration Architecture
Big Data
Information Engineering
Extract Transform Load (ETL)
Programming Tools
Distributed Computing Environment
Distributed Systems
Electronic Data Interchange (EDI)
Python (Programming Language)
Machine Learning
+15 more
Query Optimization
Cloud Services
Data Streaming
Unstructured Data
Data Processing
Freeform SQL
Application Enhancement Tool
Cloud Platform System
Database Performance
Event Driven Architecture
AWS Glue
Apache Kafka
Restful APIs
Stream Processing
Data Pipelines
Job description
- Design, develop, and maintain scalable and reliable data pipelines using Python and Scala.
- Build and optimize data processing solutions leveraging AWS services, including EMR and AWS Glue.
- Develop and maintain ETL/ELT workflows for large-scale data processing.
- Implement real-time data streaming and event-driven architectures using Apache Kafka.
- Develop, integrate, and maintain REST APIs for data exchange and application integration.
- Write complex SQL queries and optimize database performance.
- Work with distributed computing technologies to process and transform large volumes of structured and unstructured data.
- Collaborate with cross-functional teams to deliver high-quality data engineering solutions.
- Explore and leverage AI-powered tools and technologies to improve data engineering efficiency.
Requirements
The candidate should have experience building robust data pipelines, working with large-scale datasets, implementing real-time streaming solutions, and developing REST APIs., * Strong hands-on experience with Python and/or Scala.
- Extensive experience with AWS cloud services.
- Hands-on experience with AWS EMR and AWS Glue.
- Experience with Apache Kafka and real-time streaming.
- Strong SQL development and query optimization skills.
- Experience designing and developing REST APIs.
- Strong understanding of data engineering concepts, ETL/ELT pipelines, and distributed data processing.
Preferred Qualifications
- Experience working with AI/ML technologies or AI-powered development tools.
- Experience building scalable cloud-based data platforms.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on leoforce.us
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦