Big Data Developer

System Soft
Chantilly, VA, United States
1 day ago
Apply on www.thejobnetwork.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
10 years minimum
Working hours
Shift work

Tech stack

Java (Programming Language) Artificial Intelligence Airflow Big Data Computer Programming Databases Data Deduplication Data Integration Extract Transform Load (ETL) Data Transformation Data Mining Data Profiling
+9 more
Information Lifecycle Management Python (Programming Language) Linux System Administration Software Engineering Data Ingestion Sql Optimization Fastapi Machine Learning Operations Restful APIs

Job description

n \n

  • Design and optimize large-scale data ingestion, ETL, data extraction, cleansing, transformation, and data loading pipelines. \n

  • Strong programming experience in Python and/or Java, with advanced SQL and database knowledge. \n

  • Hands-on experience with Python technologies, including environment management, asynchronous development, FastAPI, and REST APIs. \n

  • Experience with Big Data environments and the complete data lifecycle, from data ingestion to visualization. \n

  • Strong experience with Apache Airflow, ETL development, and Linux-based environments. \n

  • Develop data profiling, deduplication, matching logic, and custom data integration solutions. \n

  • Leverage Big Data for AI/ML-based analytics and collaborate with Data Scientists to support AI/ML workflows. \n

  • Maintain data layer documentation and perform basic Linux server administration. \n

Requirements

We are seeking an experienced Data Engineer with 10+ years of software development/programming experience to design, develop, and optimize large-scale data ingestion, ETL, data transformation, and processing solutions in a Big Data environment., Minimum Qualification: High School Diploma/GED with 10+ years of relevant software development experience

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.thejobnetwork.com
Prepare application

Good distractions

Loading talks and stories from around this role…