Big Data Engineer

Aivra Health Llc
San Jose, United States of America
yesterday

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English

Job location

Remote
San Jose, United States of America

Tech stack

Airflow
Amazon Web Services (AWS)
Big Data
Information Engineering
Data Integrity
ETL
Data Warehousing
Distributed Computing Environment
Hadoop
Hive
Python
Machine Learning
Standard Sql
Unstructured Data
Data Storage Technologies
Cloud Platform System
Snowflake
Spark
GIT
Kubernetes
Information Technology
Kafka
Data Pipelines
Docker
Databricks

Job description

We are seeking a highly skilled Big Data Engineer to design, develop, and maintain scalable big data solutions that process large volumes of structured and unstructured data. You will collaborate with Data Engineers, Data Scientists, and Software Engineers to build reliable data pipelines and support advanced analytics initiatives., * Design, build, and maintain scalable big data pipelines.

  • Develop ETL/ELT workflows for large-scale data processing.
  • Optimize data storage, retrieval, and processing performance.
  • Work with distributed computing frameworks and cloud-based data platforms.
  • Integrate data from multiple sources into enterprise data lakes and warehouses.
  • Monitor data quality and ensure data integrity.
  • Collaborate with cross-functional teams to support business intelligence and machine learning initiatives.
  • Troubleshoot and resolve data pipeline and infrastructure issues.
  • Maintain technical documentation and follow best engineering practices.

Requirements

  • Bachelor''s degree in Computer Science, Information Technology, Data Engineering, or a related field.
  • Strong knowledge of SQL and Python or Scala.
  • Experience with distributed data processing frameworks.
  • Understanding of ETL/ELT concepts and data warehousing.
  • Strong analytical, problem-solving, and communication skills.

Preferred Skills

  • Apache Spark
  • Hadoop
  • Kafka
  • Hive
  • Scala
  • Python
  • SQL
  • Databricks
  • Snowflake
  • Apache Airflow
  • AWS
  • Docker
  • Kubernetes
  • Git

Benefits & conditions

  • Health Insurance
  • Paid Time Off
  • Flexible Work Schedule
  • 401(k)
  • Learning & Certification Support
  • Career Growth Opportunities
  • Employee Assistance Program (EAP)
  • Professional Development Opportunities

Apply for this position