Data Engineer

Insight Global
Huntington Beach, CA, United States
2 months ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience required
1 year minimum
Working hours
Regular working hours
Job source

Tech stack

BigQuery Cloud Database Python (Programming Language) NumPy Performance Tuning SQL Databases Data Processing Scripting Snowflake Pandas Code Restructuring Data Pipelines
+1 more
Databricks

Job description

  • Build, refactor, and optimize Snowflake ELT pipelines using Coalesce.

  • Translate existing WhereScape jobs into Coalesce components, ensuring logic parity and improved maintainability.

  • Develop efficient, reusable Python scripts for automation, data processing, and orchestration.

  • Implement scalable data models, transformations, and performance-optimized SQL.

  • Partner with architects and data engineers to align pipelines with new platform standards.

  • Support testing, validation, documentation, and productionizing of migrated pipelines.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global’s Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.

Requirements

  • 4+ years of hands-on experience as a Data Engineer.

  • 3+ years of experience with Snowflake (SQL, modeling, performance tuning).

  • Experience developing objects in Snowpark (snowflake framework)

  • 1+ years of Hands-on experience with Coalesce.

  • Strong Python development experience (automation, transformations, pandas, NumPy).

  • Background working on platform migrations or modernization projects.

  • Orchestration tool experience like airflow or astronomer or similar tools

  • On call support on rotation * Experience with cloud platforms and cloud-based data technologies

  • Experience with Databricks, big query, and/or redshift

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on dejobs.org

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:27 min

Explaining query execution overhead and caching limitations in BigQuery

Adnan Rahic · JS Congress

2:03 min

Accelerating pandas dataframes using cudf module plugins

Ankit Patel Ankit Patel · WWC 2024

2:34 min

Maximizing execution memory effectively via python numpy broadcasting

Jodie Burchell · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

2:30 min

Leveraging BigQuery ML for scalable SQL-based segmentation experiments

Julian Joseph · LIVE

3:33 min

Refactoring data science workflows using Rapids QDF and Pandas

Paul Graham Paul Graham · LIVE

Videos

See all

Related articles

See all