Data & AI Engineer

Tata Consultancy Services Limited
New York, NY, United States
3 months ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
4 years minimum
Compensation
$100,000.0 - $120,000.0
Working hours
Regular working hours
Job source

Tech stack

A/B Testing Application Programming Interfaces (APIs) Agile Methodology Artificial Intelligence Airflow Data Analysis Automation of Tests Big Data BigQuery Cloud Computing Cloud Storage Databases
+43 more
Continuous Integration Directed Acyclic Graph (Directed Graphs) Data Control Information Engineering Data Governance Extract Transform Load (ETL) Data Transformation Data Warehousing Data Flow Control Monitoring of Systems Python (Programming Language) PostgreSQL Machine Learning Meta-Data Management MongoDB Rapid Application Development Power BI Cloud Services Prometheus Azure Machine Learning Software Engineering SQL Databases Tableau (Software) Unstructured Data Datadog Data Logging Google Cloud Data Storage Technologies Data Classification Data Ingestion Large Language Models Generative AI Backend Git Information Technology Data Lineage Real Time Data Machine Learning Operations Cloudwatch Stream Analytics Data Pipelines Jenkins Programming Languages

Job description

  • Design, build, and maintain scalable data pipelines (batch and streaming) supporting real-time / near-real-time data integration for Lounge Services Analytics use cases.

  • Develop and optimize ETL/ELT processes for structured and unstructured data from various backend DBs and sources including MongoDB, PostgreSQL, APIs, and third-party systems.

  • Architect and manage cloud-native data infrastructure/warehouses within Google Cloud Platform (BigQuery) environments.

  • Orchestrate Data Transformations with Cloud Composer / Airflow DAGs.

  • Enable trusted, analytics-ready datasets to support Tableau dashboards and analytics.

  • Experience with observability and monitoring tools such as Datadog, CloudWatch, Prometheus, or similar platforms.

  • Implement data monitoring, logging, and observability frameworks to ensure data quality, integrity, governance and security standards are met.

  • Build AI / ML models and integrations to support predictive & personalized use cases including lounge recommendations, capacity forecasting, and lounge optimization

  • Collaborate with cross-functional teams including Analytics, Product & Business to define data requirements and deliver solutions.

  • Optimize data storage and processing for performance and cost-efficiency (partitioning and indexing solutions).

  • Implement and enforce data governance standards, including data classification, access controls, and retention policies.

  • Develop and maintain metadata management practices, including data lineage, catalog documentation, and business glossary alignment.

Requirements

  • 4-6 years of Data Engineering experience with proven hands-on experience in Big Data technologies and Business Intelligence tools.

  • Understanding of Agile or other rapid application development methods. Exposure to design and coding across one or more platforms and languages as appropriate.

  • Exposure to application design, software development, and automated testing methodologies.

  • Experience with CI/CD practices using Git, Jenkins, and code integration/review processes.

  • Backend platform: Experience building data connectors with relational & nonrelational backend databases including MongoDB, PostgreSQL, and custom API Connectors.

  • Google Cloud Platform Expertise: Hands-on experience with GCP services like BigQuery, Dataflow, Cloud Storage, Pub/Sub, Cloud Composer (Airflow DAGs), and Cloud Functions.

  • Programming Languages: Strong proficiency in Python and SQL for data ingestion, processing, and manipulation.

  • ETL/ELT: Experience with ETL/ELT processes and data warehousing concepts.

  • Data Modeling: Understanding of data modeling principles and techniques.

  • Cloud Technologies: Knowledge of cloud computing concepts and standard processes.

  • AI / ML: Experience with Statistical / Predictive Modeling, Machine Learning, A/B testing, and designing feature experiments

  • AI Frameworks: *Experience supporting AI integrations ML pipelines including BigQuery ML, or other GCP AI/ML services
  • BI Tools: Experience working with BI tools including Tableau and PowerBI

  • Problem-Solving: Excellent analytical and problem-solving skills.

  • Experience working with large datasets, ODL, identifying ingestion issues and proactively collaborating to resolve data related issues.

  • Passion for travel/airports a plus.

  • Experience with LLMs, Generative AI, or NLP systems.

  • Familiarity with feature stores and model registries.

  • Knowledge of data governance and compliance standards.

  • Experience with real-time analytics and streaming architectures.

  • Certifications in cloud or AI technologies., Qualifications : BACHELOR OF COMPUTER SCIENCE You must create an Indeed account before continuing to the company website to apply

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on indeed.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · WWC Europe 2026

1:02 min

Applying an ETL methodology to infrastructure configuration management

Axel Barbier · WWC 2023

6:21 min

Investigating push inefficiencies with upstream Git experts

Jonathan Creamer · Coffee With Developers

3:09 min

Balancing data science skillings alongside systems engineering rigor

Nico Schmidt · LIVE

3:05 min

Audience questions on AI agents and pipeline vectorization

Joy Joy · WWC 2024

Videos

See all

Related articles

See all