Senior Data Engineer - INDIA

VYTWO TECHNOLOGIES INC.

Dallas, TX, United States

2 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

5 years minimum

Working hours

Regular working hours

Job source

indeed.com

Tech stack

Airflow Amazon Web Services Data Analysis Application Integration Architecture Automation of Tests Microsoft Azure Cloud Computing Code Review Continuous Integration Information Engineering Data Governance Extract Transform Load (ETL)

+34 more

Data Security Data Systems Data Warehousing DevOps Distributed Systems Python (Programming Language) Metadata Standards Online Analytical Processing Online Transaction Processing Operational Databases Performance Tuning Standard Sql Software Engineering SQL Stored Procedures Workflow Management Systems Enterprise Data Management Cloud Platform System Sql Optimization Fast Healthcare Interoperability Resources Snowflake Containerization Data Lakes Pyspark Kubernetes Data Lineage Collibra Health Level Seven International Integration Frameworks Machine Learning Operations Software Version Control Data Pipelines Docker Databricks Programming Languages

Job description

Design, develop, and maintain scalable data pipelines using Python, PySpark, and other modern programming languages to support both batch and streaming workloads
Build and optimize data processing frameworks on cloud platforms such as Databricks or Snowflake, ensuring performance, reliability, and cost efficiency
Design and implement robust data models, including transactional (OLTP) and dimensional (OLAP) schemas, to support analytics, reporting, and application integration
Develop high quality SQL code including complex queries, stored procedures, and views, with a focus on performance tuning and efficient data access patterns
Create and manage workflow orchestration using Apache Airflow or similar tools, ensuring reliable scheduling, dependency management, and monitoring
Implement and enforce data governance and metadata standards through tools such as Microsoft Purview, including data lineage, classification, cataloging, and security policies
Build automated data quality and validation frameworks to ensure accuracy, completeness, and reliability of production datasets
Collaborate with cross functional teams including data architects, analysts, scientists, and business stakeholders to understand requirements and deliver scalable, well designed data solutions
Lead technical design sessions and code reviews, promoting engineering best practices, reusability, and maintainability
Support cloud infrastructure and DevOps practices, including CI/CD pipelines, version control, testing automation, and environment management
Monitor and troubleshoot production data pipelines, proactively addressing issues, performance bottlenecks, and system failures
Contribute to the evolution of the enterprise data platform, recommending tools, frameworks, and architectures to improve scalability and efficiency

Requirements

Do you have experience in Software engineering?, * 5+ years of experience in data engineering, software engineering, or similar disciplines

Hands-on experience with Databricks or Snowflake
Experience with orchestration tools such as Apache Airflow
Experience working with cloud ecosystems (Azure preferred; AWS/GCP acceptable)
Advanced SQL skills and experience with OLTP and OLAP data modeling
Solid understanding of modern data warehousing, data lake, and ELT/ETL design patterns
Familiarity with data governance tools, especially Microsoft Purview
Solid programming expertise in Python, PySpark, or similar languages, * Healthcare industry experience, including claims, clinical, FHIR, HL7, or provider data
Experience with containerization (Docker, Kubernetes) for data workloads
Experience supporting machine learning workflows or analytical data science pipelines
Knowledge of distributed computing concepts and performance tuning