Data Platform Engineer

We are seeking an experienced Data Engineer to join our growing team and play a central role in shaping our data strategy. This individual will be responsible for building, optimizing, and governing our Databricks-based data platform hosted on AWS connecting disparate data sources, enabling reliable data pipelines, and ensuring that high-quality data is available to support business intelligence, analytics, and machine learning initiatives., Data Strategy & Platform Governance

• Partner with leadership to define and execute a comprehensive data strategy aligned with business goals.
• Establish standards and best practices for data ingestion, transformation, storage, and access across the organization.
• Drive adoption of data governance frameworks including data cataloging, lineage tracking, and quality monitoring.
• Champion a culture of data-driven decision-making by enabling self-service analytics capabilities.

Databricks Platform Administration

• Architect, configure, and maintain the Databricks environment hosted on AWS (EMR, S3, IAM, VPC, etc.).
• Manage Databricks workspaces, clusters, Unity Catalog, and access controls.
• Optimize compute and storage costs through cluster sizing, auto-scaling, and lifecycle management.
• Implement and maintain CI/CD pipelines for data workflows using tools

Data Integration & Pipeline Development

• Design and build scalable ETL/ELT pipelines to ingest data from a variety of sources including SaaS applications, databases, APIs, and streaming platforms.
• Leverage Delta Lake, Delta Live Tables, and Databricks Workflows to build reliable, incremental data processing solutions.
• Integrate cloud-native AWS services (Glue, Kinesis, EventBridge, Lambda) with the Databricks ecosystem.
• Ensure data pipeline reliability through monitoring, alerting, and automated recovery mechanisms.

Collaboration & Data Enablement

• Partner with data analysts, data scientists, and business stakeholders to understand data requirements and deliver fit-for-purpose datasets.
• Develop and maintain data documentation, including data dictionaries, runbooks, and architecture diagrams.
• Mentor junior team members on best practices in data engineering and cloud-native development.

Do you have experience in Tooling?, Technical Skills

• 5+ years of hands-on data engineering experience in a cloud environment.

• Strong proficiency with Databricks (Notebooks, Workflows, Delta Lake, Unity Catalog, SQL Warehouses).

• Deep expertise with AWS services: S3, IAM, VPC, Glue, Redshift, RDS, Lambda, Kinesis, or EventBridge.

• Proficiency in Python for data engineering workloads; SQL expertise required.

• Experience designing and building data lake / lakehouse architectures.

Familiarity with data transformation tooling such as dbt, Apache Spark, or similar frameworks.

• Knowledge of data modeling concepts (dimensional modeling, data vault, or medallion architecture)., * Bachelor's degree in Computer Science, Information Systems, Engineering, or equivalent practical experience.

• Proven track record of delivering end-to-end data platform solutions at scale.

• Experience with infrastructure-as-code tools such as Terraform is a strong plus.

• Databricks Certified Associate or Professional certification preferred.

• AWS Certified Data Analytics or Solutions Architect certification preferred