Data and Cloud Engineer

Job Description: Design, implement and deliver large scale enterprise applications using big data open-source solutions such as Apache Hadoop, Apache Spark, Kafka, and Elastic Search. Implement data pipelines and data driven applications using Python on distributed computing frameworks like Hadoop, Apache Spark, etc.

• Responsibilities: Use AWS services and GCP services to build data pipelines and migrate on-prem data pipelines and data applications to Cloud infrastructure.
• Proficient in using AWS cloud-based services to implement batch and online steaming applications.
• Work closely with Data science teams to integrate data, algorithms into data lake systems and automate different Machine Learning workflows and assist with data infrastructure needs.
• Experience in working both, On-Prem and Cloud.
• Design and implement efficient data pipelines (ETLs) in order to integrate data from a variety of sources into Data Warehouse.
• Design and implement data model changes that align with warehouse standards.
• Design and implement backfill or other warehouse data management processes.
• Develop and execute testing strategies to ensure high quality warehouse data.
• Provide documentation, training, and consulting for data warehouse users.
• Perform requirement and data analysis in order to support warehouse project definition.
• Excellent database troubleshooting skills.
• Working technical knowledge of PowerShell.
• Strong object-oriented design and analysis skills.
• Verbal and written communication skills and the ability to interact professionally with a diverse group, executives, managers, and subject matter experts.
• Working in agile methodology, involve in Grooming, Sprint Planning, and daily Scrum meetings.
• Position may work at various and unanticipated worksites throughout the United States. Telecommuting permitted.

Qualifications: Requires a Bachelor’s degree Computer Science, Engineering, or a directly related field plus five (5) years of software engineering experience. Experience must include: Two (2) years of experience in: SQL scripting; Big data technologies: HDFS, YARN, Spark, Hive, Sqoop, Impala, Airflow, Zookeeper, and Kafka; Enterprise GitHub: branch, release, DevOps, and CI/CD pipeline; and Data engineering or ETL development. Three (3) months of experience in: Snowflake or other cloud-based data platforms (AWS, GCP, Databricks) and Python web frameworks: Django or Flask. Employer will also accept a Master’s degree plus three (3) years of engineering experience in lieu of Bachelor’s plus five (5) years of software engineering experience.

40 hours/week, 9:00am-5:00pm, Salary range: $190,000 to $193,000 per year.