Snowflake & Hadoop Lead- Full-time role

Smart Caliber Technology
Alpharetta, GA, United States
1 day ago
Apply on www.adzuna.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
7 years minimum
Working hours
Regular working hours
Job source

Tech stack

Query Performance Java (Programming Language) Adaptable Database Systems Big Data Computer Programming Information Engineering Data Governance Extract Transform Load (ETL) Data Transformation Data Security Data Systems Data Warehousing
+16 more
Distributed Computing Environment Apache Hadoop Hadoop Distributed File System MapReduce Apache Hive Python (Programming Language) Performance Tuning Query Optimization SQL Databases Data Processing Freeform SQL Data Ingestion Snowflake Apache Spark Pyspark Data Management

Job description

Hadoop Development & Migration

  • Design, develop, and implement Hadoop MapReduce jobs for complex data processing requirements.
  • Work with HDFS, Hive, and HiveQL to query, process, and manage large datasets.
  • Leverage Apache Spark and PySpark for distributed data processing and streaming analytics.
  • Lead and support migration of Hadoop-based data platforms and workloads to Snowflake.
  • Analyze existing Hadoop environments and develop efficient migration strategies.

Snowflake Development

  • Design and implement scalable data models using Snowflake.
  • Develop complex SQL queries to extract, transform, and analyze large datasets.
  • Create and optimize Snowflake views and materialized views for improved query performance.
  • Implement Snowflake performance tuning and query optimization techniques.
  • Work with Snowpark and Snowspark for advanced data processing.
  • Establish data governance, data quality, and security practices within Snowflake.

Data Engineering & Leadership

  • Design and implement scalable data ingestion and transformation pipelines.
  • Ensure data accuracy, consistency, quality, and security across data platforms.
  • Collaborate with cross-functional teams to understand business requirements and deliver data solutions.
  • Provide technical leadership and guidance on Snowflake and Hadoop development initiatives.

Requirements

  • Very strong hands-on experience with Snowflake.
  • Strong experience in Hadoop-to-Snowflake migration.
  • Strong programming skills in Python, Java, and SQL.
  • Deep understanding of the Hadoop ecosystem:
  • HDFS
  • MapReduce
  • Hive / HiveQL
  • Apache Spark
  • PySpark
  • Strong knowledge of Snowflake architecture, data warehousing, and query optimization.
  • Experience with Snowpark / Snowspark.
  • Strong understanding of data ingestion, ETL/ELT, and data transformation.
  • Experience in data modeling, data quality, governance, and data security.

Required Experience

  • 7+ years of overall IT experience.
  • Strong hands-on Snowflake development experience.
  • Proven experience leading or supporting Hadoop migration projects.
  • Experience working with large-scale data processing and enterprise data warehouse solutions.
  • Ability to work in a hybrid environment (3 days onsite per week).

Preferred Candidate Profile

  • Strong technical expertise in Snowflake and Big Data technologies.
  • Experience in Financial Services or Banking environments is an advantage.
  • Excellent problem-solving, communication, and collaboration skills.
  • Ability to take ownership of technical deliverables and lead data engineering initiatives.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.adzuna.com
Prepare application

Good distractions

Loading talks and stories from around this role…