Big Data / AWS Data Engineer

• Design, develop, and maintain scalable data pipelines using Spark (Scala/Python), AWS Glue, AWS Lambda, and Shell scripting.
• Build and support ETL/ELT processes across AWS platforms, including S3, Hive, Apache Iceberg, and Amazon Redshift.
• Develop and maintain Redshift SQL/PLSQL code, stored procedures, and data-processing frameworks.
• Perform query tuning, performance optimization, and data-sharing implementations.
• Support ingestion and processing of structured and semi-structured data formats including JSON, CSV, XML, ORC, Parquet, and Avro.
• Design, enhance, and support solutions that provide a unified view of customer data across systems.
• Integrate multiple data sources, applications, and business processes to support reporting, analytics, and customer engagement initiatives.
• Collaborate with business and technical teams to define and implement scalable data solutions.
• Manage and support Data Subject Requests (DSRs), including customer data erasure requests.
• Work closely with the DPO team to ensure compliance with privacy regulations and internal policies.
• Maintain and support the MicroStrategy Privacy Compliance module. (Very infrequent once in year or 2 years)
• Contribute to data governance initiatives that improve data quality, consistency, and compliance.
• Analyze and troubleshoot complex data issues using Redshift SQL, MySQL, Python/PySpark, Shell scripting, and Excel.
• Investigate data mismatches, reporting issues, and business data anomalies.
• Support business analysis related to customer tiers, loyalty programs, and email marketability.
• Provide production support and root-cause analysis for critical data platform issues.
• Technical Leadership & Collaboration
• Participate in Customer 360, Privacy, Data Governance, and Phaedon initiatives.
• Conduct code reviews and design reviews.
• Provide cross-team technical support and problem resolution.
• Research, design, and develop next-generation data capabilities and products.
• Act as a subject matter expert for customer data platforms.
• Primary Technologies
• AWS: S3, EC2, Redshift (RA3/Serverless), S3 Tables, S3 Hive, AWS Glue, Lambda (Python), Kafka, Airflow, Apache Iceberg
• Spark (Scala/Python)
• Redshift SQL/PLSQL, MySQL
• Data Modeling and Query Optimization
• Key Strengths
• Strong ownership and accountability
• Strong AWS Data Engineering stack awareness
• Ability to bridge business and technical teams
• Expertise across engineering, analytics, compliance, architecture, and operational support
• Adaptability and willingness to take on diverse data-related challenges, Lead Data Engineer (Python, AWS, Spark, Kafka, SQL, Snowflake, Databricks, GenAI) Do you love building and pioneering in the technology space? Do you enjoy solving complex busine…
• 16 hours ago, Lead Data Engineer (Python, AWS, Spark, Kafka, SQL, Snowflake, Databricks, GenAI) Do you love building and pioneering in the technology space? Do you enjoy solving complex busine…
• 19 hours ago, Lead Data Engineer (Python, AWS, Spark, Kafka, SQL, Snowflake, Databricks, GenAI) Do you love building and pioneering in the technology space? Do you enjoy solving complex busine…
• 19 hours ago +

• 12+ years of experience in Data Warehouse / Big Data ecosystems.
• 5+ years in a lead /SME role

Strong understanding of:

• Data architecture & dimensional modeling
• ETL/ELT frameworks
• Distributed data processing
• Cloud data platforms
• Data governance and security
• Experience managing large-scale enterprise data environments.
• Proven track record of delivering complex data programs.
• Excellent communication and stakeholder management abilities., * Experience in large enterprise or global delivery environments.
• Exposure to data modernization and cloud transformation programs.
• Certifications in cloud platforms (AWS/Azure/GCP) or data engineering.
• Experience in regulated industries (Hospitality, Finance, Healthcare, Telecom, etc.).

Key Competencies:

• Strategic thinking with hands-on technical depth
• Executive presence and client-facing confidence
• Problem-solving and decision-making ability
• Operational excellence mindset