Senior Data Engineer : Databricks & Delta

This is an exciting role for a Senior Data Engineer with strong Databricks and Delta to join a world-leading Energy Trading company. Role Description: We are looking for a highly skilledSenior Data Engineerwith strong analytical capabilities and hands-on experience in building scalable data solutions. The ideal candidate will have expertise inDatabricks,streaming architectures, data ingestion frameworks, anddashboarding, along with experience working withSAP data sources. You will be responsible for designing and implementing scalable data pipelines and enabling real-time and batch analytics across the organization. Key Responsibilities: Design and implement scalable data pipelines usingDatabricks and Delta Live Tables (DLT) Develop and managestreaming data solutionsfor real-time analytics and decision-making Ingest data from various sourcesincludingSAP systems using Qlik ReplicateandADFto ingest data from various sources ensuring data quality and consistency Performdata modellingand optimize data structures for analytics and reporting Collaborate with cross-functional teams to understand data requirements and deliver robust solutions Monitor, troubleshoot, and enhance data workflows for performance and reliability Maintain technical documentation and contribute to data engineering best practices Required Skills & Qualifications: Strong experience withDatabricks- DLT, Delta Lake concepts, UC governance Solid understanding ofstreaming technologies(e.g., Spark Structured Streaming, Autoloader) Programming skills inPython,SQL, orScala Proficiency indata modelling, ETL/ELT processes, and data architecture Experience in MS Copilot, AI Foundry , Databricks (MosiacAI, MLflow, Agentbricks, Genie) Strong analytical background with problem-solving skills Performance tuning concepts like watermarking, late data handling, parallelism & checkpointing Hands-on expertise inADF, and Qlik Replicatefor data ingestion and replication Experience working inAzure cloud environments #J-18808-Ljbffr