Data Engineer

We are seeking a talented Data Engineer with 2-4 years of experience in building and maintaining scalable data pipelines on the AWS Cloud using Snowflake. The ideal candidate should have hands-on experience with AWS data services, ETL/ELT development, cloud data warehousing, and data integration. The candidate will be responsible for designing, developing, and optimizing data pipelines that support business intelligence, analytics, and reporting initiatives., * Design, develop, and maintain scalable ETL/ELT data pipelines using AWS services and Snowflake.

• Build robust data ingestion frameworks to load data from multiple sources, including relational databases, APIs, flat files, and cloud storage.
• Develop and optimize data models, schemas, and database objects in Snowflake.
• Implement data transformation logic using SQL, Snowflake Stored Procedures, and Snowpark (preferred).
• Load structured and semi-structured data (CSV, JSON, XML, Parquet, Avro) into Snowflake.
• Develop batch and incremental data processing pipelines using AWS services.
• Optimize Snowflake performance through clustering, query tuning, partitioning, and warehouse optimization.
• Integrate data from AWS S3 into Snowflake using Snowpipe, COPY INTO, or external stages.
• Monitor, troubleshoot, and resolve ETL failures and performance bottlenecks.
• Implement data quality checks, validation, and reconciliation processes.
• Collaborate with business analysts, data architects, and application teams to deliver high-quality data solutions.
• Follow coding standards, documentation practices, and data governance policies.
• Participate in Agile ceremonies, code reviews, and production support activities.

• Strong hands-on experience with Snowflake Cloud Data Platform.
• Experience with Snowflake architecture, virtual warehouses, databases, schemas, stages, and file formats.
• Strong SQL development skills with query optimization techniques.
• Experience with Snowflake Tasks, Streams, Stored Procedures, Secure Views, and Time Travel.
• Experience using Snowpipe and COPY INTO commands for automated data ingestion.
• Knowledge of data sharing and role-based access control (RBAC).

AWS Data Engineering

• Amazon S3
• AWS Glue
• AWS Lambda
• Amazon Redshift (good to have)
• AWS IAM
• Amazon CloudWatch
• AWS Secrets Manager
• Amazon EventBridge or Amazon SNS/SQS (preferred)

ETL/ELT

• Design and development of ETL/ELT workflows.
• Data migration from on-premises systems to AWS Cloud.
• Experience with incremental loads and Change Data Capture (CDC).
• Experience scheduling and orchestrating workflows using AWS Glue Workflows or Apache Airflow (preferred).

Databases

• SQL Server
• Oracle
• PostgreSQL
• MySQL

Programming Languages

• SQL (Mandatory)
• Python (Mandatory)
• PySpark (Preferred)

Version Control & CI/CD

• Git
• Jenkins, GitHub Actions, or AWS CodePipeline
• AWS CodeCommit (preferred)

Data Formats

• CSV
• JSON
• XML
• Parquet
• Avro, * Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• 2-4 years of experience in Data Engineering.
• Hands-on experience with Snowflake and AWS Cloud services.
• Strong SQL and Python programming skills.
• Experience developing cloud-based ETL pipelines.
• Good understanding of data warehousing concepts and dimensional modeling.
• Familiarity with Agile/Scrum development methodologies.

Preferred Skills

• Experience with Apache Airflow or Managed Workflows for Apache Airflow (MWAA).
• Experience with dbt (Data Build Tool).
• Exposure to Apache Spark or PySpark.
• Knowledge of Amazon Redshift.
• Experience integrating REST APIs.
• Familiarity with Power BI, Tableau, or Amazon QuickSight.
• Experience implementing CI/CD pipelines for data engineering solutions.

Soft Skills

• Strong analytical and problem-solving skills.
• Excellent verbal and written communication skills.
• Ability to work independently and within cross-functional teams.
• Strong attention to detail and commitment to delivering high-quality solutions.
• Ability to manage multiple priorities in a fast-paced environment.

Nice to Have

• SnowPro Core Certification.
• AWS Certified Data Engineer - Associate.
• AWS Certified Solutions Architect - Associate.
• Experience with Terraform or AWS CloudFormation.
• Exposure to Kafka or Amazon Kinesis for streaming data.
• Knowledge of DataOps and data governance best practices.

Key Competencies

• Snowflake
• AWS Glue
• Amazon S3
• AWS Lambda
• SQL
• Python
• ETL/ELT Development
• Snowpipe
• Data Warehousing
• Data Modeling
• Performance Optimization
• Git
• Jenkins
• AWS IAM
• CI/CD
• Cloud Data Engineering
• Data Quality
• Agile Development, Bachelor or equivalent Range of Year Experience-Max Year 4