Principal Data Engineer

Eckoh is on a transformational journey to enhance its data capabilities, leveraging Databricks and AWS to create a scalable and resilient data platform. This platform will underpin real-time client dashboards, operational reporting, advanced AI and data science initiatives, and robust compliance and data management processes across multiple regions. As Principal Data Engineer, you are the most senior engineer on the team. You will help shape the architecture, contribute to engineering standards, and work on the most demanding parts of the system, guiding the team where needed. This is a role with technical depth and influence over a greenfield platform, but one that operates within a shared technical direction set collaboratively.

During the initial build phase, an external delivery partner will be embedded within the team. You will work alongside their engineers, providing technical direction, setting shared standards, and ensuring the architecture and engineering practices being established are ones Eckoh will be able to own and build on confidently after handover.

Your work will span the full depth of the platform, from ingestion and pipeline design through to the data products that serve client dashboards, operational reporting, and AI use cases.

Key Functional Responsibilities:

Architecture:

· Contribute to the definition of the end-to-end architecture, covering ingestion, storage, processing, and serving layers, and the integration patterns connecting source systems to downstream consumers.

• Play an active role in establishing the target-state architecture from a greenfield position, contributing to foundational decisions on data modelling approach, medallion layering, batch vs streaming, and multi-region design, and helping to document rationale and trade-offs.

Engineering:

· Build pipelines, integration patterns, and platform components. This is a hands-on role, and the principal is part of delivery, not just directing it.

• Work with the Team Lead to define and uphold engineering standards across the team, covering code quality, testing, documentation, code review, and deployment practices.
• Contribute to and champion the CI/CD and infrastructure-as-code practices that underpin reliable, repeatable platform delivery.
• Act as a senior technical escalation point for production data issues, pipeline failures, and platform incidents post-handover.

Data:

· Define datasets, semantic models, and data contracts that power client dashboards, operational reporting, billing assurance, and AI analytics.

• Work closely with the analytics engineering and BI development functions to ensure modelling and Tableau patterns are consistent, performant, and maintainable.
• Ensure reporting data products meet the accuracy, latency, and availability expectations of client-facing and internal consumers.

Delivery:

· Contribute to sprint planning, estimation, and prioritisation alongside the Team Lead and Technical Product Owner, bringing engineering depth to delivery decisions.

• Identify and surface technical risk early, propose mitigation approaches, and help the team maintain momentum without accumulating avoidable technical debt.
• Support and mentor senior and mid-level engineers, helping them grow their technical capability and raise the overall standard of engineering across the team.

Partner & Stakeholder Management:

• During the initial build phase, work alongside the delivery partner on architecture alignment, code review, and knowledge transfer, ensuring a smooth transition to Eckoh ownership.
• Engage with Databricks on platform capabilities, roadmap, and technical support as usage scales.
• Communicate technical decisions and trade-offs clearly to non-technical stakeholders, including the Head of Product and commercial teams.

Data Governance & Quality

• Embed governance into the platform from day one, covering data classification, access control, lineage, cataloguing, retention, and deletion across all data domains.
• Set and maintain data quality standards, ensuring pipelines deliver accurate, complete, and timely data to all downstream consumers.
• Ensure sensitive data, including payment and personal data, is handled appropriately throughout ingestion, processing, storage, and consumption.
• Ensure the platform meets data sovereignty, PCI DSS, SOC2, and GDPR obligations across all regions in which Eckoh operates.

Platform Operations & Cost

• Define SLAs for data products and pipelines. Drive reliability, performance, and cost efficiency across the platform.
• Monitor cloud and spend, identify optimisation opportunities, and ensure the platform operates within a responsible run rate without compromising quality or reliability.
• Partner with DevOps and Cloud Ops teams on deployment pipelines, observability tooling, and the rollout of additional data processing regions.

• Demonstrable hands-on data engineering background, with significant experience designing and operating cloud-native data platforms in production.
• Deep, hands-on experience including Spark and PySpark, Delta Lake, Unity Catalog, and medallion architecture, with at least one deployment of meaningful scale.
• Strong proficiency in Python and SQL, used in production data engineering contexts.
• Strong AWS data stack knowledge, including S3, IAM, and networking, with sufficient familiarity with Glue, EMR, or Athena to make informed build-vs-buy decisions.
• Experience designing systems that serve both low-latency operational queries and large-scale analytical workloads from a shared lakehouse foundation.
• Strong grounding in data modelling, including medallion architecture, dimensional modelling, data contracts, schema versioning, and serving layer design.
• Experience with infrastructure-as-code and CI/CD for data pipelines, using tools such as Terraform, GitHub Actions, or equivalent.
• Experience implementing data observability, covering data quality, lineage, freshness, and pipeline health monitoring.
• Familiarity with data governance concepts as implemented in modern Lakehouse platforms, covering cataloguing, lineage, classification, and access control.
• Comfortable operating in a regulated environment with PCI DSS, GDPR, SOC2, or similar compliance obligations.
• Demonstrated senior individual contributor leadership, with the credibility and communication skills to set technical direction for engineers and partners alike.

Desirable Skills & Experience

• Hands-on experience with Databricks Unity Catalog, Delta Live Tables, or Databricks Workflows.
• Experience with real-time or streaming data pipelines, using tools such as AWS Kinesis, Apache Kafka.
• Experience working alongside a systems integrator or delivery partner on a build-and-handover engagement.
• Familiarity with Tableau or similar BI and embedded analytics tools, particularly from a data platform and serving layer perspective.
• Exposure to payments, contact centre, conversational AI, or SaaS platform domains.

As trusted advisors, we use our knowledge to solve challenges and deliver the best for our clients

We take personal ownership to strive for excellence in whatever we do

We are welcoming, embrace diversity and respect each other in a spirit of true humanity

At Eckoh we value creativity, collaboration and innovation and look forward to welcoming a new team member who shares these values.

Join Us!

So, if you are an innovator and love to find creative solutions to challenges, are passionate about helping customers provide exceptional service then you should consider working here at Eckoh.

Pay: From £75,000.00 per year

Benefits:

• Bereavement leave
• Casual dress
• Company events
• Company pension
• Cycle to work scheme
• Free flu jabs
• Free parking
• Life insurance
• On-site parking
• Referral programme
• Work from home

Eckoh is a market leader of Customer Engagement Security Solutions, supporting an international client base from our offices in the U.S. and the U.K. Eckoh is owned by Bridgepoint one of the world’s leading private asset growth investors, with over $75 billion of assets under management.

As a PCI DSS Level 1 Service Provider, our products and expanding digital and conversational AI capabilities help enterprises protect sensitive data, improve customer experience, and modernise the way they operate. Our mission is to set the standard for secure interactions between consumers and the world’s leading brands. With a strong heritage in secure payments and an expanding portfolio of cloud native, omni-channel engagement solutions our vocation is simple: to make every customer interaction secure whether it is handled by a human agent or an AI agent, without ever compromising the customer experience.

Our large portfolio of clients, which includes many of the Fortune 250, come from a broad range of vertical markets including healthcare, retail, hospitality, financial services, and utilities.