Data & BI Engineer

• Designs, develops, and maintains ETL/ELT pipelines using Azure Data Factory and Microsoft Fabric

• Develops Fabric notebooks using PySpark, Python, and Spark SQL.

• Writes, optimizes, and supports complex T-SQL queries, stored procedures, functions, and views, including performance tuning and advanced query optimization

• Designs and implements efficient, scalable, and secure database schemas to support modernization initiatives

• Builds and manages Azure SQL and related cloud-based data services

• Monitors and improves data pipeline performance, reliability, and cost efficiency

• Manages source control and deployments for data solutions using Git and Azure DevOps, including Fabric deployment pipelines

• Collaborates with subject matter experts, discipline and job function leaders, software engineers, business analysts, and data scientists to meet data requirements

• Supports data integration from external providers and internal applications

• Designs and optimizes data models, schemas, and integration patterns across multiple platforms

• Partners with leadership to align data architecture initiatives with business objectives and future growth

• Evaluates and recommends emerging technologies, tools, and platforms for data integration, storage, and analytics

• Implements, establishes and enforces best practices for data governance, quality, security, and compliance

• Establishes medallion architecture in Fabric to ensure consistent report ready data

• Establishes logic and governance patterns for business data calculations (schedule gates, architectural and job cost calculations, etc)

• Collaborates with business units, analysts, and data scientists to ensure architecture supports analytics and reporting needs

• Supports and mentors team members in data engineering and architecture practices

• Develops timely, professional documentation for data flows, schemas, pipelines, and architectural decisions

• Supports Power BI users and leads visualization best practice discussions

• Leads efforts to migrate on-premises data infrastructure to cloud

• Performs other duties as needed or required

Results/Accountability:

• Delivers a comprehensive and forward-looking data architecture strategy

• Ensures data solutions are scalable, secure, and aligned with organizational goals

• Maintains reliable and performant data pipelines that meet business requirements

• Provides architecture documentation that guides implementation and supports knowledge transfer

• Proactively identifies opportunities for modernization and efficiency improvements

• Achieves high levels of internal client satisfaction through delivery of robust, future-ready solutions

• Maintains good backlog and active sprint work item hygiene in Azure DevOps

• Delivers solutions according to specifications and plan

• Ensures that first time production release quality is feature complete and defect free

• Manages deployment of business intelligence solutions to ensure the application of best practices and data integrity

• Analyzes data environments and vision for long term data delivery performance proactively

• Bachelor of Science in Computer Science, Bachelor of Arts in Information Systems, or relevant work experience required

• Seven or more years of experience in data architecture and engineering roles

• Advanced proficiency in T-SQL

• Effective at Azure SQL database design and optimization

• Proven experience developing and maintaining pipelines with Azure Data Factory

• Expertise with Microsoft Fabric for advanced data integration and analytics

• Strong understanding of data modeling, schema design, and warehousing techniques

• Experience with ETL/ELT processes and performance tuning

• Working proficiency in Python, including PySpark, for data transformation in Fabric notebooks

• Knowledge of Power BI (semantic models, DAX) or similar BI tools preferred

• Demonstrated ability to set and enforce data governance and architecture standards

Microsoft Certified: Fabric Data Engineer Associate (DP-700), Fabric Analytics Engineer Associate (DP-600), or comparable Azure data certification preferred

General Requirements:

• Excellent attention to detail

• Strong written and verbal communication skills

• Strong organizational skills

• Ability to work in a fast-paced environment

• Comply with all company policies and procedures

• Demonstrate the qualities and character traits as defined in the True Difference

Physical Requirements:

• Must be able to remain in a stationary position 75% of the time

• Associate needs to occasionally move about the office to access file cabinets, office equipment, etc.

• Constantly operates a computer and other office equipment