Senior Data Engineer (Azure and Databricks)

OMSAI TECH LLC
New Brunswick, United States of America
4 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior
Compensation
$ 125K

Job location

New Brunswick, United States of America

Tech stack

Clean Code Principles
Agile Methodologies
Azure
Bash
Big Data
Cloud Computing
Data Integration
ETL
Data Security
Database Design
Linux
Python
Linked Data
Shell
Microsoft SQL Server
SQL Azure
Oracle Applications
Azure
Shell Script
SQL Databases
Systems Integration
Talend
Azure
System Availability
Spark
Reliability of Systems
Data Management
Dynamic Data
REST
Data Pipelines
Databricks

Job description

We are seeking a highly skilled and energetic Senior Data Engineer with expertise in Azure cloud platform and Databricks to join our dynamic data team. In this role, you will lead the design, development, and implementation of scalable data solutions that empower data-driven decision-making across the organization., * Design, develop, and maintain scalable ETL (Extract, Transform, Load) processes using Azure Data Lake, Databricks, Spark, and other big data tools to support complex analytics and reporting needs.

  • Collaborate with cross-functional teams to gather requirements and translate them into efficient data models and pipelines that facilitate seamless data access and analysis.
  • Implement cloud-based solutions utilizing Azure services such as Azure Data Factory, Azure SQL Database, and Azure Data Lake Storage to ensure high availability and security of enterprise data assets.
  • Write clean, efficient code in Python, Bash (Unix shell), or Shell Scripting to automate workflows and improve system reliability.
  • Apply Agile methodologies to manage project tasks efficiently while continuously improving processes through collaboration and feedback.
  • Support database design efforts by creating schemas that promote efficient storage, retrieval, and analysis of large datasets including linked data concepts for semantic interoperability.

Requirements

Do you have experience in Schema design?, * Proven experience as a Data Engineer working with Azure cloud services including Azure Data Lake, Databricks, Data Factory, and related tools.

  • Hands-on experience with SQL databases like Microsoft SQL Server and Oracle; knowledge of database design principles is essential.
  • Proficiency in programming languages such as Python
  • Experience with ETL tools like Talend or Informatica for complex data integration workflows.
  • Demonstrated ability to develop RESTful APIs for seamless system integration.
  • Familiarity with shell scripting (Bash/Unix shell) for automation purposes in a Linux environment.

Apply for this position