Big Data Engineer
Aivra Health Llc
San Jose, United States of America
yesterday
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
EnglishJob location
Remote
San Jose, United States of America
Tech stack
Airflow
Amazon Web Services (AWS)
Big Data
Information Engineering
Data Integrity
ETL
Data Warehousing
Distributed Computing Environment
Hadoop
Hive
Python
Machine Learning
Standard Sql
Unstructured Data
Data Storage Technologies
Cloud Platform System
Snowflake
Spark
GIT
Kubernetes
Information Technology
Kafka
Data Pipelines
Docker
Databricks
Job description
We are seeking a highly skilled Big Data Engineer to design, develop, and maintain scalable big data solutions that process large volumes of structured and unstructured data. You will collaborate with Data Engineers, Data Scientists, and Software Engineers to build reliable data pipelines and support advanced analytics initiatives., * Design, build, and maintain scalable big data pipelines.
- Develop ETL/ELT workflows for large-scale data processing.
- Optimize data storage, retrieval, and processing performance.
- Work with distributed computing frameworks and cloud-based data platforms.
- Integrate data from multiple sources into enterprise data lakes and warehouses.
- Monitor data quality and ensure data integrity.
- Collaborate with cross-functional teams to support business intelligence and machine learning initiatives.
- Troubleshoot and resolve data pipeline and infrastructure issues.
- Maintain technical documentation and follow best engineering practices.
Requirements
- Bachelor''s degree in Computer Science, Information Technology, Data Engineering, or a related field.
- Strong knowledge of SQL and Python or Scala.
- Experience with distributed data processing frameworks.
- Understanding of ETL/ELT concepts and data warehousing.
- Strong analytical, problem-solving, and communication skills.
Preferred Skills
- Apache Spark
- Hadoop
- Kafka
- Hive
- Scala
- Python
- SQL
- Databricks
- Snowflake
- Apache Airflow
- AWS
- Docker
- Kubernetes
- Git
Benefits & conditions
- Health Insurance
- Paid Time Off
- Flexible Work Schedule
- 401(k)
- Learning & Certification Support
- Career Growth Opportunities
- Employee Assistance Program (EAP)
- Professional Development Opportunities