Senior Data Engineer

This is an opportunity to join a high growth business as the Data Engineer. The position will work with the Analytics & Insights team at Allworth Financial to optimize the company’s existing data infrastructure to support a robust data analytics and reporting structure.

We are looking for a Senior Data Engineer, or similarly experienced data modernization professional, to help guide the next stage of our data platform evolution. This role will support the modernization of legacy SQL-based processes into scalable, maintainable cloud data pipelines, while also helping us identify architectural gaps, improve engineering standards, and mature our data governance practices.

The ideal candidate has hands-on experience across data engineering, cloud data platforms, ETL/ELT design, data modeling, orchestration, performance tuning, data quality, and governance. We are looking for someone who can both build and advise: someone who can write production-quality pipelines, review existing architecture, mentor team members, and point out the risks, patterns, and opportunities we may not yet know to look for.

This person will play a key role in helping us move from legacy stored procedures and fragmented reporting processes toward a more reliable, transparent, and well-governed modern data architecture., · Design, build, and optimize data pipelines using PySpark, Delta Lake, SQL, and cloud-based data engineering tools.

· Improve data pipeline reliability, observability, logging, error handling, and restartability.

· Review existing notebooks, SQL scripts, data models, and orchestration workflows for maintainability and performance.

· Guide best practices for Azure Synapse, Spark, Delta Lake, data lake storage, and related cloud data services.

· Identify architectural gaps, technical debt, and modernization risks.

· Help design and implement data quality checks, reconciliation processes, and validation frameworks.

· Support development of canonical IDs, master data patterns, and entity resolution processes.

· Assist with data cataloging, lineage, metadata management, and governance practices.

· Partner with business stakeholders to understand reporting, analytics, and data product requirements.

· Help structure scalable data models for BI tools such as ThoughtSpot, Power BI, or similar platforms.

· Mentor data team members and help raise the overall engineering maturity of the team.

· Provide guidance on what we may be overlooking in areas such as security, performance, cost, governance, orchestration, testing, and long-term maintainability.

Do you have experience in SQL databases?, · 5+ years of professional experience in data engineering, analytics engineering, data architecture, or a closely related role.

· Strong SQL skills, including stored procedures, joins, window functions, CTEs, merge logic, and performance tuning.

· Hands-on experience with PySpark or distributed data processing frameworks.

· Experience designing and maintaining ETL/ELT pipelines in a cloud environment.

· Experience with data lake or lakehouse architecture, including Delta Lake or similar table formats.

· Strong understanding of data modeling concepts, including dimensional modeling, staging layers, curated layers, and semantic/reporting models.

· Experience with pipeline orchestration, scheduling, dependency management, and failure recovery.

· Ability to troubleshoot data quality issues, pipeline failures, schema drift, performance bottlenecks, and source system inconsistencies.

· Familiarity with modern data governance concepts, including cataloging, lineage, ownership, access control, data definitions, and data quality monitoring.

· Ability to communicate clearly with both technical and non-technical stakeholders.

· Experience reviewing existing systems and recommending practical, incremental improvements.

· Experience with Azure Synapse Analytics, Azure Data Lake Storage, Microsoft Fabric, Azure Data Factory, or related Azure data services.