Sr Data Engineer - Databricks
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+14 more
Job description
- Design, build, and maintain scalable batch and streaming pipelines on Databricks using PySpark and Spark SQL within a medallion (Bronze/Silver/Gold) architecture
- Ingest and unify data from operational databases, cloud storage, APIs, and marketing platforms such as Braze
- Model and optimize Delta Lake tables for performance, reliability, and cost efficiency using techniques such as partitioning, liquid clustering, OPTIMIZE, and VACUUM
- Deliver analytics-ready data for reporting on sales, shipments, orders, and other key business metrics, supporting Power BI dashboards and downstream analytics
- Orchestrate jobs using Databricks Workflows, managing dependencies, retries, alerting, and SLAs
- Apply Unity Catalog for data governance, lineage, and access control
- Work to a high engineering standard using Git, CI/CD, testing, code reviews, and clear documentation
- Monitor pipeline health, troubleshoot issues, and tune Spark jobs and cluster configurations for speed and cost optimization
- Partner with business and technical stakeholders to deliver reliable, high-quality data products
Requirements
- Bachelor’s degree in Computer Science, Information Technology, Engineering, Mathematics, or a related field
- 5+ years of data engineering experience building and operating production data pipelines
- Hands-on expertise with the Databricks Lakehouse Platform, including Delta Lake, Databricks Workflows, and Unity Catalog
- Advanced experience with PySpark and Spark SQL
- Strong SQL development and performance tuning skills
- Solid Python development skills and experience with modern engineering practices
- Experience using Git, CI/CD pipelines, testing frameworks, and code review processes
- Experience working with AWS, Azure, or GCP and their core compute and storage services
- Strong understanding of data modeling concepts, including dimensional and medallion architectures
- Experience designing ELT/ETL solutions and integrating data from databases, APIs, files, and event streams
- Strong communication skills and the ability to work independently within a client-facing environment
NICE TO HAVE
- Databricks Certified Data Engineer Associate or Professional certification
- Experience with Structured Streaming, Auto Loader, Kafka, or Kinesis
- Experience working with marketing or customer data platforms such as Braze
- Retail, grocery, or consumer packaged goods (CPG) industry experience
- Experience supporting Power BI reporting and self-service analytics initiatives
Benefits & conditions
- Opportunity for advancement and career progression
- Competitive compensation package
- Comprehensive benefits plan
- Successful referral program
- The opportunity to work with one of Canada’s 50 Best Managed Companies
- Satisfaction of working for a reputable company
- A flexible, dynamic, and diverse workplace
EQUAL OPPORTUNITY EMPLOYER
In our commitment to promote fair and equitable treatment of all employees and applicants, Adastra Corporation provides equal employment opportunities for all individuals regardless of age, sex, disability, race, ethnic origin, citizenship, creed, sexual orientation, marital status, or any other ground as described in the Ontario Human Rights Code.
In addition, accommodation will be provided during the hiring process. Adastra Corporation’s implementation and support of employment initiatives, encourage diversified labour force participation and equal access to opportunities based on merit and performance.
AI Usage - Our hiring process includes the use of AI-enabled tools to screen applications (e.g., keyword matching, qualification ranking). A human recruiter reviews all AI-generated shortlists to make informed hiring decisions.
About the company
Adastra is looking for a hands-on Data Engineer to join our team supporting a modern enterprise data platform. You’ll design and run Databricks pipelines that turn raw operational, marketing, and retail data into trusted, analytics-ready data products the business relies on every day.
Working as an embedded member of the data team, you’ll own pipelines end to end on the Databricks Lakehouse Platform. If you love clean data, well-tuned Spark jobs, and shipping solutions that people actually use, this role is for you., Adastra is a global leader in AI and data-driven transformation, helping organizations lead with artificial intelligence-responsibly, strategically, and at scale. With over 25 years of experience, Adastra empowers enterprise clients to unlock business value through data innovation, operational excellence, and smart customer engagement.
Trusted by some of the world’s most prominent brands, Adastra delivers end-to-end solutions grounded in thoughtful strategy, robust governance, and deep technical expertise. From defining vision to ensuring execution, Adastra guides organizations through every stage of their AI, data and cloud journey-building future-ready capabilities and delivering measurable, lasting impact.
Adastra serves clients across key industries including financial services, automotive, manufacturing, technology, media and telecom (TMT), healthcare, retail, and professional services. The company employs more than 2,000 professionals across North America, Europe, and Asia.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.techcareers.comGood distractions
Talks and stories from around this role — technically off-topic, practically not.
Moments
Explore playlistsVideos
See allRelated articles
See all
Top Big Data Technologies That You Need to Know
Highest Paying Tech Companies for Developers
Data Engineer Salary UK
Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production