Data Engineer
Role details
Job location
Tech stack
Job description
You'll build and scale the data platform that powers decisions across Salmon - owning pipelines end to end so data flows reliably from source to insight.
What you'll do
- Design and run batch and streaming pipelines on Databricks and Apache Spark over AWS, ingesting from databases, APIs, and event streams.
- Shape our lakehouse architecture on S3 and Delta Lake, including medallion (Bronze / Silver / Gold) layering and storage optimization.
- Work directly with analytics, engineering, and product teams to turn raw data into reliable, usable inputs for the business.
What you'll own
- Build and maintain scalable ETL/ELT pipelines that are fault-tolerant, cost-optimized, and performant on AWS.
- Optimize Spark jobs end to end - cluster sizing, caching, joins, shuffles, partitioning, and storage formats (Parquet, Delta).
- Manage Databricks Workflows, jobs, and cluster configurations, and respond to failures and performance issues.
- Implement data governance and access control with Unity Catalog across accounts and environments.
- Protect data correctness and reliability across the platform.
Requirements
- Hands-on experience with Databricks and strong proficiency in Apache Spark (PySpark and/or Scala).
- Strong Python for data engineering, plus SQL and Spark SQL for transformation and analytics.
- Experience with Delta Lake and lakehouse architectures, including performance optimization.
- Experience writing clean, testable, maintainable code with Git-based version control.
- Ability to work independently and own pipelines end to end.
Skills: Access Control, Amazon Web Services (AWS), Apache Spark, Application Programming Interface (API), Banking Services, Caching, Cisco Unity, Cost Control, Database Extract Transform and Load (ETL), Finance, Git, Information/Data Security (InfoSec), Loans, Multiplatform/Cross-Platform, Performance Tuning/Optimization, Problem Solving Skills, Python Programming/Scripting Language, SQL (Structured Query Language), Scalable System Development, Software Engineering, Source Code/Configuration Management (SCM)