Big Data Developer
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+9 more
Job description
n \n
-
Design and optimize large-scale data ingestion, ETL, data extraction, cleansing, transformation, and data loading pipelines. \n
-
Strong programming experience in Python and/or Java, with advanced SQL and database knowledge. \n
-
Hands-on experience with Python technologies, including environment management, asynchronous development, FastAPI, and REST APIs. \n
-
Experience with Big Data environments and the complete data lifecycle, from data ingestion to visualization. \n
-
Strong experience with Apache Airflow, ETL development, and Linux-based environments. \n
-
Develop data profiling, deduplication, matching logic, and custom data integration solutions. \n
-
Leverage Big Data for AI/ML-based analytics and collaborate with Data Scientists to support AI/ML workflows. \n
-
Maintain data layer documentation and perform basic Linux server administration. \n
Requirements
We are seeking an experienced Data Engineer with 10+ years of software development/programming experience to design, develop, and optimize large-scale data ingestion, ETL, data transformation, and processing solutions in a Big Data environment., Minimum Qualification: High School Diploma/GED with 10+ years of relevant software development experience
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦