Senior Data Engineer

We are seeking an experienced Senior Data Engineer to architect, build, and scale our core data platform. You will take ownership of end-to-end batch and streaming data pipelines, ensuring high data quality, security, and performance across our cloud infrastructure. Working closely with data scientists, analysts, and software engineers, you will translate complex business requirements into robust data models and production-ready data products., * Pipeline Architecture & Development. Design, build, and maintain scalable, fault-tolerant ETL/ELT pipelines for both batch and real-time streaming workloads.

• Data Modeling & Warehousing. Architect efficient data models, data lakes, and cloud data warehouses to support downstream business intelligence and machine learning workflows.
• Platform Optimization & Cost Management. Monitor system performance, tune complex queries, and manage compute resources to balance latency, throughput, and cloud infrastructure costs.
• Data Quality & Observability. Implement automated testing, validation frameworks, and alerting systems to ensure data accuracy, completeness, and reliability.
• Security & Governance. Enforce role-based access controls, data privacy standards, and compliance policies across all data storage and processing layers.
• AI/ML & GenAI Enablement. Partner with ML engineers to prepare structured and unstructured datasets, feature stores, or retrieval-augmented generation (RAG) pipelines for production AI applications.
• Technical Leadership & Mentorship. Lead architectural design reviews, establish coding best practices, and mentor junior and mid-level engineers.

• Experience. 10+ years of hands-on experience in data engineering, building and operating large-scale production data platforms.
• Programming & Querying. Advanced proficiency in Python (or Scala/Java) and expert-level SQL skills, including query optimization and schema design.
• Big Data & Streaming. Practical experience with distributed processing frameworks such as Apache Spark (PySpark) and event-streaming platforms like Apache Kafka or Flink.
• Cloud & Data Warehousing. Deep hands-on experience with major cloud providers (AWS, Google Cloud Platform, or Azure) and modern data platforms such as Snowflake, Databricks, BigQuery, or Redshift.
• Orchestration & Transformation. Strong experience with workflow orchestration tools (e.g., Apache Airflow, Prefect, Dagster) and transformation frameworks like dbt.
• DevOps & CI/CD. Familiarity with containerization (Docker, Kubernetes), Infrastructure as Code (Terraform), and automated CI/CD deployment pipelines