Data Engineer - 1668
aKube Inc
Santa Monica, CA, United States
5 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Temporary to permanent
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
4 years minimum
Compensation
$192,400
Working hours
Regular working hours
Job source
Tech stack
Airflow
Amazon Elastic Compute Cloud
Amazon S3
Code Review
Continuous Integration
Data Architecture
Data Validation
Information Engineering
Data Governance
Extract Transform Load (ETL)
Database Queries
DevOps
+16 more
Distributed Computing Environment
Github
Apache Hive
Identity and Access Management
JSON
SQL Databases
Workflow Management Systems
Parquet
Data Logging
Data Ingestion
Snowflake
Data Lakes
Pyspark
Semi-structured Data
Data Pipelines
Databricks
Job description
- Develop and maintain ETL/ELT pipelines using Databricks, PySpark, and SQL for marketing acquisition reporting.
- Migrate existing Snowflake SQL scripts and transformations into Databricks Unity Catalog.
- Build ingestion pipelines for external marketing vendor data.
- Perform data parsing, normalization, transformation, and validation.
- Implement data quality checks, monitoring alerts, and issue resolution workflows.
- Troubleshoot pipeline failures and improve processing performance.
- Support SLA monitoring, logging improvements, and pipeline reliability.
- Collaborate with analytics teams to ensure accurate reporting datasets.
- Participate in code reviews, documentation, and engineering best practices.
Requirements
- 4+ years of Data Engineering experience.
- Strong SQL skills, including complex joins, window functions, and analytical queries.
- Hands-on production experience with PySpark and/or Spark SQL.
- Databricks experience, including Delta Lake and Unity Catalog.
- Strong understanding of ETL/ELT pipelines, data modeling, and distributed processing., * Minimum 4 years of professional Data Engineering experience or equivalent.
- Experience building production pipelines in Databricks.
- Strong understanding of Delta Lake, Unity Catalog, Lakehouse architecture, and data governance.
- Experience with AWS services such as S3, IAM, EC2, Glue, or Lambda.
- Experience with Apache Airflow or similar orchestration tools.
- Experience processing JSON, Parquet, CSV, and other semi-structured data.
- Experience with Git/GitHub, CI/CD, and modern DevOps practices.
Nice to Have:
- Experience migrating SQL transformations from Snowflake to Databricks.
- Experience working with marketing acquisition or advertising data.
- Experience implementing data quality monitoring and pipeline reliability improvements.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦