Data Engineer

This is a highly hands-on engineering role focused on translating evolving R&D requirements into secure, reliable, scalable, and production-ready data platforms and pipelines. You will collaborate with AI Engineers, Data Scientists, Researchers, Software Engineers, and Product Managers to accelerate experimentation and production delivery.

• Design and build scalable batch, streaming, and event-driven data pipelines and platforms. * Develop data ingestion, transformation, enrichment, processing, and consumption capabilities using modern lakehouse and cloud-native architectures. * Build reusable data products, APIs, services, and pipelines supporting analytics, machine learning, generative AI, and agentic AI workloads. * Optimize solutions for performance, scalability, reliability, resiliency, observability, and cost efficiency. * Implement engineering practices including data modeling, automated testing, CI/CD, Infrastructure as Code, and deployment automation. * Implement data quality, lineage, metadata, cataloging, security, privacy, governance, and access controls. * Evaluate emerging data and AI technologies through prototypes and proof-of-concepts and transition successful solutions toward production. * Troubleshoot complex data, performance, scalability, and production issues. * Translate evolving product and R&D requirements into practical technical designs and incremental delivery plans. * Collaborate with cross-functional teams and communicate technical decisions, dependencies, risks, and trade-offs.

• Extensive experience designing and building enterprise-scale data engineering solutions. * Strong hands-on experience with Python, PySpark, SQL, Apache Spark and distributed data processing. * Experience designing and operating large-scale batch and real-time data pipelines and modern lakehouse platforms like databricks. * Experience building cloud-native data solutions using AWS, Azure, or comparable enterprise cloud platforms. * Experience with modern data architecture patterns including data lakes, lakehouses, data products, and event-driven architectures. * Experience with orchestration technologies such as Apache Airflow, Databricks Workflows equivalent. * Strong understanding of data governance, data quality, lineage, metadata management, security, privacy, and access controls. * Ability to diagnose and resolve complex performance, scalability, reliability, and operational issues. * Strong problem-solving, communication, and cross-functional collaboration skills. * Experience integrating structured, semi-structured, unstructured, and streaming data. * Ability to work effectively in an R&D environment with evolving requirements and rapid experimentation. * Experience supporting machine learning, generative AI, or agentic AI applications. * Experience with CI/CD, automated testing, version control, Infrastructure as Code, observability, and platform automation. * Experience with MLflow, Azure Machine Learning, Azure AI Foundry, or Microsoft Fabric. * Experience with Kafka or other event-streaming technologies. * Experience in payments, financial services, or another regulated industry.