Data Engineer / Lead Data Engineer

V-Work Infotech Solutions INC
yesterday

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Tech stack

Airflow
Amazon Web Services (AWS)
Azure
Big Data
Google BigQuery
Cloud Computing
Continuous Integration
Data Architecture
Information Engineering
Data Integration
ETL
Data Mart
Data Warehousing
DevOps
Elasticsearch
Hadoop
Hive
Python
Machine Learning
Operational Databases
Scrum
Redis
SQL Databases
Data Streaming
Talend
Enterprise Data Management
Google Cloud Platform
Azure
Informatica Powercenter
Snowflake
Spark
Database Performance
Infrastructure as Code (IaC)
GIT
Data Lake
PySpark
Kubernetes
Apache Flink
Kafka
Apache Nifi
Spark Streaming
Data Management
Machine Learning Operations
Terraform
Stream Processing
Azure
Data Pipelines
Docker
Redshift
Databricks

Job description

We are seeking a highly experienced Senior Data Engineer with 12+ years of experience in designing, developing, and optimizing enterprise-scale data platforms, ETL/ELT pipelines, data warehouses, and cloud-based analytics solutions. The ideal candidate should be comfortable working on a W2 basis and possess strong expertise in modern data engineering technologies and cloud platforms., * Design, develop, and maintain scalable ETL/ELT data pipelines.

  • Build and optimize enterprise data lakes, data warehouses, and data marts.
  • Develop batch and real-time data processing solutions.
  • Implement data integration solutions across multiple platforms and applications.
  • Collaborate with Data Scientists, Analysts, Product Owners, and Business Stakeholders.
  • Ensure data quality, governance, security, and compliance standards.
  • Optimize database performance and large-scale data processing workflows.
  • Lead technical design discussions and mentor junior engineers.
  • Monitor, troubleshoot, and resolve production data issues.
  • Participate in Agile/Scrum development processes., * Apache Flink
  • Apache NiFi
  • Elasticsearch
  • Redis
  • Data Mesh Architecture
  • GenAI Data Pipelines
  • Master Data Management (MDM)

Requirements

  • 12+ years of overall IT experience with strong Data Engineering expertise.
  • Advanced proficiency in Python, SQL, and/or Scala.
  • Extensive experience with Apache Spark, PySpark, Hadoop, Hive, and Kafka.
  • Strong expertise in ETL/ELT development and data pipeline architecture.
  • Hands-on experience with Databricks, Snowflake, Redshift, BigQuery, or Azure Synapse.
  • Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform.
  • Strong understanding of Data Warehousing, Data Modeling, and Data Lake concepts.
  • Experience with Airflow, DBT, Azure Data Factory, Informatica, or Talend.
  • Experience with real-time streaming technologies such as Kafka, Spark Streaming, or Flink.
  • Knowledge of CI/CD, Docker, Kubernetes, Git, and DevOps practices.
  • Strong analytical, troubleshooting, and communication skills., * Experience with Delta Lake and Lakehouse Architecture.
  • Knowledge of Machine Learning data pipelines and MLOps.
  • Experience with Terraform and Infrastructure as Code (IaC).
  • Cloud certifications (AWS, Azure, or Google Cloud Platform).
  • Experience working in Healthcare, Finance, Retail, or E-commerce domains.

Apply for this position