Data Engineer - SQL

R+L Carriers

Ocala, FL, United States

23 days ago

Apply on www.jobmonkeyjobs.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

3 years minimum

Working hours

Regular working hours

Languages

English

Job source

www.jobmonkeyjobs.com

Tech stack

Query Performance Application Programming Interfaces (APIs) Data Analysis Microsoft Azure Big Data Business Software Business Systems Databases Data Architecture Data Validation Information Engineering Data Governance

+41 more

Data Integration Data Integrity Extract Transform Load (ETL) Data Retention Data Security Data Structures Data Synchronization Data Systems Data Warehousing Relational Databases Database Development Database Queries Dimensional Modeling File Transfer JSON Software Architecture Query Optimization Role-Based Access Control Power BI Azure Data Lake Software Engineering SQL Stored Procedures SQL Databases Data Streaming Systems Integration Technical Data Management Systems Enterprise Data Management Data Logging Enterprise Software Applications Azure Data Factory Change Data Capture Indexer Git Data Lakes Semi-structured Data Information Technology Data Management Restful APIs Software Version Control Data Pipelines Databricks

Job description

Develop, maintain, monitor, and optimize data pipelines that move information from operational business systems into the company’s enterprise data platform and data lake.
Design and maintain ETL and ELT processes using Azure Data Factory and related Azure data technologies.
Manage and support the company’s Azure Data Lake environment, including data organization, storage structures, processing workflows, access, and overall platform reliability.
Architect new data feeds and integrations from enterprise applications, APIs, databases, files, and other internal and external data sources.
Develop processes for ingesting structured and semi-structured data including SQL data, APIs, JSON, CSV, flat files, and other common data formats.
Troubleshoot data feed failures, pipeline errors, processing issues, data discrepancies, and other problems affecting the availability or accuracy of business data.
Monitor and improve data pipeline performance, processing times, query performance, resource utilization, and overall data platform efficiency.
Develop and maintain SQL queries, views, stored procedures, transformations, and reusable data structures used by reporting and business applications.
Develop curated datasets and reporting views that provide Power BI and other analytics tools with consistent, reliable, and understandable business data.
Work with Power BI developers and data analysts to understand reporting requirements and translate business needs into appropriate data structures and models.
Design and maintain data models that support enterprise reporting, analytics, operational reporting, historical analysis, and KPI measurement.
Apply dimensional modeling principles where appropriate, including fact tables, dimension tables, relationships, measures, historical data, and common business entities.
Improve business visibility into data by identifying opportunities to make information easier to access, understand, analyze, and use for decision-making.
Establish and maintain appropriate data quality controls, validation processes, reconciliation procedures, and monitoring to identify missing, inaccurate, duplicated, or inconsistent data.
Develop processes for incremental data loading, change detection, historical data retention, and efficient processing of large data sets.
Establish standards for data naming, definitions, structure, documentation, ownership, lineage, and appropriate use across enterprise reporting.
Support data governance initiatives that create consistent definitions and trusted sources for customers, employees, shipments, financial information, operational metrics, and other key business entities.
Help establish consistent enterprise KPIs and reporting definitions to reduce conflicting calculations and different versions of the same business metric.
Document data sources, pipelines, transformations, dependencies, business rules, data models, and reporting structures.
Maintain visibility into dependencies between source systems, data pipelines, transformations, reporting datasets, and downstream applications.
Participate in the evaluation and implementation of new data technologies, tools, integrations, and architectural improvements.
Work with application developers, infrastructure teams, data analysts, project managers, and business stakeholders to support new projects and data requirements.
Follow appropriate security, access control, data privacy, change management, testing, and development practices when working with enterprise data.
Proactively identify opportunities to improve data reliability, processing performance, automation, scalability, maintainability, and overall data architecture.

Requirements

3+ years of experience in data engineering, data integration, database development, business intelligence engineering, or a related technical role.
Strong experience with Microsoft Azure data technologies, particularly Azure Data Factory, Azure Data Lake Storage, and DataBricks.
Strong SQL skills with experience developing complex queries, views, stored procedures, transformations, and reporting datasets.
Experience designing, developing, and maintaining ETL and ELT pipelines that integrate multiple enterprise data sources.
Experience integrating data through REST APIs, databases, file transfers, JSON, CSV, flat files, and other common integration methods.
Strong understanding of relational databases, data structures, database relationships, data types, indexing, and query optimization.
Understanding of data warehousing concepts, dimensional modeling, fact and dimension tables, star schemas, and data structures designed for analytics.
Experience preparing and modeling data for Power BI or similar business intelligence and visualization platforms.
Understanding of Power BI data requirements, semantic models, relationships, refresh processes, and reporting performance considerations.
Experience troubleshooting and optimizing data pipelines, SQL queries, transformations, and large data processing workloads.
Knowledge of incremental loading, change data capture concepts, data synchronization, data dependencies, and historical data management.
Experience implementing data validation, reconciliation, monitoring, logging, error handling, and data quality processes.
Understanding of data governance concepts including data ownership, business definitions, lineage, documentation, access controls, quality, and trusted data sources.
Ability to understand business processes and translate business reporting requirements into scalable technical data solutions.
Ability to analyze data discrepancies and determine whether problems originate within source systems, integrations, transformation logic, data models, or reporting layers.
Experience with Python for data processing, automation, integration, or data engineering is required.
Familiarity with Git or other source control and development lifecycle practices.
Familiarity with Azure security, role-based access control, service accounts, credentials, and secure data integration practices.
Experience working with data originating from ERP, CRM, transportation, logistics, financial, warehouse, customer service, or other enterprise business systems is preferred.
Strong analytical, troubleshooting, communication, organization, documentation, and problem-solving skills.
Ability to explain technical data concepts, issues, dependencies, and recommendations clearly to both technical and non-technical stakeholders.
Ability to work independently while collaborating cross-functionally with application development, infrastructure, analytics, project management, and business teams.
Ability to take ownership of critical data processes and proactively identify potential failures, performance issues, data quality concerns, and architectural improvements.
Ability to manage multiple priorities in a fast-paced environment with changing business and reporting needs.
Ability to read, write, and speak English fluently., * Bachelor’s degree in Computer Science, Software Engineering, or a related field; or equivalent practical experience

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.jobmonkeyjobs.com

Prepare application

Good distractions

Loading talks and stories from around this role…