Data Engineer job

Tech Brains Solutions Inc
Atlanta, GA, United States
10 days ago
Apply on jobs.diversity.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
2 years minimum
Working hours
Regular working hours

Tech stack

LangGraph Framework Microsoft Azure Information Systems Continuous Integration Information Engineering Data Governance Extract Transform Load (ETL) Data Transformation Data Systems Data Warehousing Python (Programming Language) Microsoft SQL Server
+12 more
Power BI SQL Databases SQL Server Integration Services Data Logging Retrieval-Augmented Generation Apache Spark Microsoft Fabric Information Technology Collibra Data Pipelines Legacy Systems Databricks

Job description

We are seeking a motivated Data Engineer to join our team and support the modernization of our data estate. This role focuses on assisting with data pipeline development, migration of legacy systems, and maintaining scalable, secure, and efficient data solutions using modern technologies, particularly Microsoft Fabric and Azure-based platforms., &bull Assist in designing, building, and maintaining ETL/ELT data pipelines using Microsoft Fabric and Azure Databricks. &bull Support migration and maintenance of SSIS packages from legacy systems. &bull Implement medallion architecture (Bronze, Silver, Gold) for data lifecycle and quality. &bull Create and manage notebooks (Fabric Notebooks, Databricks) for data transformation using Python, SQL, and Spark. &bull Build curated datasets to support Power BI dashboards. &bull Collaborate with data analysts and business stakeholders to deliver fit-for-purpose data assets. &bull Apply data governance policies in line with Microsoft Purview or Unity Catalog. &bull Support monitoring, logging, and CI/CD automation using Azure DevOps.

Technical Stack &bull Microsoft Fabric (Dataflows, Pipelines, Notebooks, OneLake) &bull Azure Databricks &bull SQL Server / SQL Managed Instances &bull Power BI &bull SSIS (migration and maintenance) &bull LangGraph and RAG DB (for advanced data workflows)

Requirements

&bull Bachelor’s degree in computer science, Information Systems, or related field. &bull 2-3 years of experience in data engineering or related roles. &bull Proficiency in SQL, Python, Spark. &bull Familiarity with LangGraph and RAG DB concepts. &bull Hands-on experience with Microsoft Fabric and Power BI. &bull Understanding of ETL/ELT pipelines and data warehousing concepts.

Preferred &bull Knowledge of CI/CD automation with Azure DevOps. &bull Familiarity with data governance tools (Microsoft Purview, Unity Catalog). &bull Experience with SSIS package migration and maintenance.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on jobs.diversity.com
Prepare application

Good distractions

Loading talks and stories from around this role…