Snowflake & Hadoop Lead- Full-time role
Smart Caliber Technology
Alpharetta, GA, United States
1 day ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.adzuna.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
7 years minimum
Working hours
Regular working hours
Job source
Tech stack
Query Performance
Java (Programming Language)
Adaptable Database Systems
Big Data
Computer Programming
Information Engineering
Data Governance
Extract Transform Load (ETL)
Data Transformation
Data Security
Data Systems
Data Warehousing
+16 more
Distributed Computing Environment
Apache Hadoop
Hadoop Distributed File System
MapReduce
Apache Hive
Python (Programming Language)
Performance Tuning
Query Optimization
SQL Databases
Data Processing
Freeform SQL
Data Ingestion
Snowflake
Apache Spark
Pyspark
Data Management
Job description
Hadoop Development & Migration
- Design, develop, and implement Hadoop MapReduce jobs for complex data processing requirements.
- Work with HDFS, Hive, and HiveQL to query, process, and manage large datasets.
- Leverage Apache Spark and PySpark for distributed data processing and streaming analytics.
- Lead and support migration of Hadoop-based data platforms and workloads to Snowflake.
- Analyze existing Hadoop environments and develop efficient migration strategies.
Snowflake Development
- Design and implement scalable data models using Snowflake.
- Develop complex SQL queries to extract, transform, and analyze large datasets.
- Create and optimize Snowflake views and materialized views for improved query performance.
- Implement Snowflake performance tuning and query optimization techniques.
- Work with Snowpark and Snowspark for advanced data processing.
- Establish data governance, data quality, and security practices within Snowflake.
Data Engineering & Leadership
- Design and implement scalable data ingestion and transformation pipelines.
- Ensure data accuracy, consistency, quality, and security across data platforms.
- Collaborate with cross-functional teams to understand business requirements and deliver data solutions.
- Provide technical leadership and guidance on Snowflake and Hadoop development initiatives.
Requirements
- Very strong hands-on experience with Snowflake.
- Strong experience in Hadoop-to-Snowflake migration.
- Strong programming skills in Python, Java, and SQL.
- Deep understanding of the Hadoop ecosystem:
- HDFS
- MapReduce
- Hive / HiveQL
- Apache Spark
- PySpark
- Strong knowledge of Snowflake architecture, data warehousing, and query optimization.
- Experience with Snowpark / Snowspark.
- Strong understanding of data ingestion, ETL/ELT, and data transformation.
- Experience in data modeling, data quality, governance, and data security.
Required Experience
- 7+ years of overall IT experience.
- Strong hands-on Snowflake development experience.
- Proven experience leading or supporting Hadoop migration projects.
- Experience working with large-scale data processing and enterprise data warehouse solutions.
- Ability to work in a hybrid environment (3 days onsite per week).
Preferred Candidate Profile
- Strong technical expertise in Snowflake and Big Data technologies.
- Experience in Financial Services or Banking environments is an advantage.
- Excellent problem-solving, communication, and collaboration skills.
- Ability to take ownership of technical deliverables and lead data engineering initiatives.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.adzuna.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦