Senior Data Engineer

We are looking for a Senior Data Engineer with strong hands-on experience in building and supporting modern data platforms using Databricks, Delta Lake, SQL, Python, and PySpark. The candidate will be responsible for developing scalable and reliable data pipelines, implementing data quality and reconciliation processes, supporting production workloads, and contributing to cloud-based data engineering solutions. The role also requires an understanding of DevOps, CI/CD, data governance, metadata management, and security fundamentals. The ideal candidate should be able to understand business requirements, translate them into effective data solutions, and work closely with business and technical stakeholders. Exposure to Databricks Genie Spaces, AI agents, and operational bots is highly desirable as part of our evolving Data & AI engineering landscape., * Design, develop, and maintain scalable data pipelines and ETL/ELT solutions using Databricks, Delta Lake, SQL, Python, and PySpark.

• Implement Bronze, Silver, and Gold data processing, including batch, incremental, and metadata-driven data transformations.
• Develop and optimize Databricks workflows and orchestration, including scheduling, dependencies, retries, error handling, and logging.
• Implement data quality, validation, reconciliation, and completeness checks across source, transformation, and target layers.
• Provide production support and troubleshooting for data pipelines and Databricks workloads, including monitoring, root-cause analysis, incident resolution, and preventive actions.
• Optimize Spark jobs, SQL queries, Delta tables, and data workloads for performance, scalability, reliability, and cost efficiency.
• Apply cloud security and access-control practices, including RBAC, identities/service principals, secrets, encryption, and environment-level controls.
• Develop and maintain DevOps and CI/CD processes covering source control, code reviews, testing, automated deployment, and promotion across environments.
• Follow data governance and metadata standards, including lineage, cataloging, data classification, documentation, ownership, and stewardship.
• Leverage Databricks Genie Spaces, AI agents, and operational bots for governed data self-service and automation of engineering, monitoring, validation, and support activities.
• Understand business processes, data flows, KPIs, and domain requirements and translate them into reliable, scalable data solutions.
• Openness and willingness to work night shifts is mandatory., We are looking for a Senior Data Engineer to design, develop, and optimize scalable, reliable, and secure data platforms and pipelines. The role requires strong expertise in modern data engineering, cloud data platforms, data architecture, data modeling, and database technologies, with the ability to translate business requirements into high-quality, production-ready data solutions. The candidate will work closely with cross-functional teams to build and evolve data products, optimize performance and cost, ensure data quality and governance, and contribute to engineering standards and best practices across the organization., * Design, develop, and maintain end-to-end data pipelines, data platforms, and integration solutions across batch and streaming workloads.
• Translate business, functional, and technical requirements into maintainable, production-ready data solutions.
• Build data flows across structured, semi-structured, and unstructured sources using appropriate ingestion and integration patterns.
• Develop data transformation and processing workflows to support large data volumes and evolving business requirements.
• Implement data quality, validation, reconciliation, testing, and observability practices across data pipelines.
• Optimize data pipelines and processing workloads for performance, scalability, and cost efficiency.
• Implement security, privacy, access control, and data governance throughout the data lifecycle.
• Monitor pipelines and platform components, troubleshoot failures, perform root-cause analysis, and implement corrective actions.
• Collaborate with architects, stakeholders, application teams, and QA/UAT teams to deliver solutions aligned with business objectives.
• Contribute to CI/CD, automation, reusable frameworks, development standards, mentoring, and continuous improvement initiatives.

• Strong proficiency in SQL, including complex queries, analytical functions, stored procedures, and performance optimization.
• Strong understanding and hands-on experience with modern data architecture, including Data Lake, Lakehouse, Cloud Data Warehouse, Medallion Architecture, and ELT patterns.
• Strong understanding of data warehousing and data modeling, including dimensional modeling, fact/dimension design, and Slowly Changing Dimensions (SCD).
• Hands-on experience with cloud data platforms and modern data engineering technologies.
• Experience working with relational and non-relational databases, data ingestion, and streaming technologies.
• Understanding of data quality, governance, security, and observability practices.
• Proficiency in Python, Shell, PowerShell, or equivalent scripting and automation technologies.
• Experience with Git, CI/CD, and Agile delivery practices.
• Strong analytical, troubleshooting, and problem-solving skills with the ability to independently diagnose and resolve complex technical issues.
• Excellent communication, documentation, collaboration, and stakeholder management skills, with a strong sense of ownership and accountability.

Systech is a modern Data and Analytics consulting firm helping clients embed data-driven capabilities into operations.