Sr Data Engineer

Hexaware Technologies
McLean, VA, United States
10 days ago
Apply on www.dice.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Working hours
Regular working hours
Job source

Tech stack

Artificial Intelligence Amazon Web Services Amazon Elastic Compute Cloud Amazon S3 ARM Architecture Big Data Code Review Data Governance Extract Transform Load (ETL) Amazon DynamoDB Identity and Access Management Information Retrieval
+21 more
Job Scheduling Python (Programming Language) Metadata Repositories Performance Tuning Power BI Standard Sql Search Technologies Tableau (Software) Data Ingestion Retrieval-Augmented Generation Snowflake Apache Spark State Machines Agentic-AI Pyspark AWS Glue Tools for Reporting Data Pipelines Human in the Loop AWS EKS Control M

Job description

We are looking for a Senior Data Engineer to lead the design and development of cloud-native data pipelines using AWS, Snowflake, IICS, and PySpark. The role involves leading Spark-on-EKS implementations, advanced Snowflake development, and AI/Agentic AI proof-of-concept initiatives while mentoring engineers and collaborating with stakeholders., * Architect end-to-end data pipelines using AWS, Snowflake, IICS, and PySpark.

  • Design and deploy containerized PySpark workloads on AWS EKS.
  • Build and optimize EMR clusters on EC2 or EKS for large-scale data processing.
  • Orchestrate workflows using AWS Step Functions and Control-M.
  • Manage metadata using AWS Glue Data Catalog and integrate with Snowflake Iceberg.
  • Use DynamoDB for pipeline state management, checkpoints, and configuration.
  • Lead the implementation of Snowflake capabilities, including Iceberg Tables, Dynamic Tables, Views, Materialized Views, Tasks, Streams, and data governance.
  • Work with Snowflake Horizon Catalog, Semantic Views, Cortex, Snowflake Agents, and token-usage optimization strategies.
  • Design and develop RAG and Agentic AI PoCs covering vector search, BM25 hybrid retrieval, evaluation frameworks, human-in-the-loop controls, and security.
  • Develop PySpark and Snowpark pipelines for data ingestion and transformation.
  • Lead IICS file-based ingestion design and implementation.
  • Conduct code reviews, optimize performance, and mentor engineers.

Requirements

  • Strong hands-on experience with AWS EKS, EMR, EC2, Step Functions, DynamoDB, AWS Glue Data Catalog, S3, and IAM.
  • Advanced Snowflake development experience, including Iceberg Tables, Dynamic Tables, Views, Materialized Views, Tasks, Streams, governance, Horizon Catalog, Cortex, and Agents.
  • Expertise in PySpark, including containerized Spark workloads on EKS.
  • Experience with Snowpark for Python.
  • Hands-on experience with IICS for file-based ETL ingestion.
  • Strong SQL, data modeling, and performance-tuning skills.
  • Experience building RAG or Agentic AI proof-of-concept solutions.
  • Experience with Control-M job scheduling and orchestration.
  • Familiarity with reporting tools such as Power BI or Tableau.
  • Strong technical leadership, mentoring, and stakeholder communication skills.

About the company

Hexaware is a dynamic and innovative IT organization committed to delivering cutting-edge solutions to our clients worldwide. We pride ourselves on fostering a collaborative and inclusive work environment where every team member is valued and empowered to succeed.

Hexaware provides access to a vast array of tools that enhance, revolutionize, and advance professional profile. We complete the circle with excellent growth opportunities, chances to collaborate with highly visible customers, chances to work alongside bright brains, and the perfect work-life balance.

With an ever-expanding portfolio of capabilities, we delve deep into and identify the source of our motivation. Although technology is at the core of our solutions, it is still the people and their passion that fuel Hexaware’s commitment towards creating smiles.

“At Hexaware we encourage to challenge oneself to achieve full potential and propel growth. We trust and empower to disrupt the status quo and innovate for a better future. We encourage an open and inspiring culture that fosters learning and brings talented, passionate, and caring people together.”

We are always interested in, and want to support, the professional and personal you. We offer a wide array of programs to help expand skills and supercharge careers. We help discover passion-the driving force that makes one smile and innovate, create, and make a difference every day.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com
Prepare application

Good distractions

Loading talks and stories from around this role…