Data Engineer

INFOBAHN SOFTWORLD INC

Santa Clara, CA, United States

8 days ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Temporary contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Amazon Web Services Amazon S3 Apache HTTP Server Cloud Database Continuous Integration Data Architecture Data Validation Information Engineering Data Governance Data Infrastructure Extract Transform Load (ETL) Data Systems

+11 more

Database Queries Apache Hive Operational Databases SQL Databases Data Logging Data Layers Data Lakes Deployment Automation Splunk Data Pipelines Databricks

Job description

We are seeking a Senior Data Engineer with strong experience designing, developing, and supporting scalable data pipelines using Databricks and AWS. The engineer will support business-critical data solutions across supply chain, capacity planning, and global sourcing., * Design, develop, and support scalable data pipelines in Databricks and AWS.

Build and maintain data ingestion pipelines using Amazon S3.
Onboard new data feeds and perform schema validation and data quality checks.
Develop and manage Bronze, Silver, and Gold data layers.
Design and maintain Delta Lake / Apache Iceberg tables.
Enhance ETL frameworks and improve pipeline reliability and scalability.
Perform data validation using SQL and Amazon Athena.
Troubleshoot production data pipeline issues and ensure timely resolution.
Monitor production pipelines and datasets using Splunk.
Implement and support CI/CD deployments for data engineering workflows.
Support migration of legacy Hive workloads to unified ODP platforms.
Maintain business-critical reporting datasets and ensure data accuracy and availability.
Collaborate with business stakeholders, data engineering teams, and technical teams to understand requirements and deliver reliable data solutions.

Requirements

The ideal candidate will have hands-on experience with S3 ingestion, Databricks, Delta/Iceberg tables, ETL frameworks, data quality, SQL/Athena validation, CI/CD, production troubleshooting, and data-layer architecture., * Strong hands-on experience as a Senior Data Engineer / Data Pipeline Engineer.

Expertise in Databricks and AWS.
Strong experience developing scalable ETL/data pipelines.
Experience with Amazon S3 and cloud-based data ingestion.
Experience with Delta Lake and/or Apache Iceberg.
Strong SQL skills and experience with Amazon Athena.
Experience with Bronze/Silver/Gold data architecture.
Experience with schema validation and data quality frameworks.
Production troubleshooting and incident resolution experience.
Experience with CI/CD and deployment automation.
Experience with Splunk or similar monitoring/logging tools.
Experience working with business-critical datasets and reporting pipelines.

Preferred / Nice-to-Have

Experience with Hive-to-lakehouse/platform migrations.
Experience supporting supply chain, capacity planning, or global sourcing data use cases.
Experience with unified ODP/data platform environments.
Strong stakeholder management and communication skills.