Big Data Engineer
Aivra Health Llc
San Jose, CA, United States
25 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Job source
Tech stack
Airflow
Amazon Web Services
Big Data
Information Engineering
Data Integrity
Extract Transform Load (ETL)
Data Warehousing
Distributed Computing Environment
Apache Hadoop
Apache Hive
Python (Programming Language)
Machine Learning
+13 more
Standard Sql
Unstructured Data
Data Storage Technologies
Cloud Platform System
Snowflake
Apache Spark
Git
Kubernetes
Information Technology
Apache Kafka
Data Pipelines
Docker
Databricks
Job description
We are seeking a highly skilled Big Data Engineer to design, develop, and maintain scalable big data solutions that process large volumes of structured and unstructured data. You will collaborate with Data Engineers, Data Scientists, and Software Engineers to build reliable data pipelines and support advanced analytics initiatives., * Design, build, and maintain scalable big data pipelines.
- Develop ETL/ELT workflows for large-scale data processing.
- Optimize data storage, retrieval, and processing performance.
- Work with distributed computing frameworks and cloud-based data platforms.
- Integrate data from multiple sources into enterprise data lakes and warehouses.
- Monitor data quality and ensure data integrity.
- Collaborate with cross-functional teams to support business intelligence and machine learning initiatives.
- Troubleshoot and resolve data pipeline and infrastructure issues.
- Maintain technical documentation and follow best engineering practices.
Requirements
- Bachelor’’s degree in Computer Science, Information Technology, Data Engineering, or a related field.
- Strong knowledge of SQL and Python or Scala.
- Experience with distributed data processing frameworks.
- Understanding of ETL/ELT concepts and data warehousing.
- Strong analytical, problem-solving, and communication skills.
Preferred Skills
- Apache Spark
- Hadoop
- Kafka
- Hive
- Scala
- Python
- SQL
- Databricks
- Snowflake
- Apache Airflow
- AWS
- Docker
- Kubernetes
- Git
Benefits & conditions
- Health Insurance
- Paid Time Off
- Flexible Work Schedule
- 401(k)
- Learning & Certification Support
- Career Growth Opportunities
- Employee Assistance Program (EAP)
- Professional Development Opportunities
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.dice.comGood distractions
Talks and stories from around this role — technically off-topic, practically not.
Moments
Explore playlistsVideos
See allRelated articles
See all
EM
Eli McGarvie
over 3 years ago
DS
Dhannush Subramani
Top Big Data Technologies That You Need to Know
about 4 years ago
LM
Luis Minvielle
How to Become an AI Engineer
over 2 years ago
EM
Eli McGarvie
Data Engineer Salary UK
about 3 years ago
BB
Benedikt Bischof
Making Data Warehouses Fast: A Developer’s Story
about 4 years ago
CH
Chris Heilmann
Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production
almost 2 years ago