Analytics Platform Engineer (Azure & AWS Databricks

Dcode Talent LLC

31 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Languages

English

Job location

Tech stack

Amazon Web Services (AWS)

Data analysis

Azure

Bash

Big Data

Computer Programming

Information Engineering

ETL

Data Warehousing

DevOps

Github

Identity and Access Management

Python

Log Analysis

Nagios

Performance Tuning

Powershell

SQL Databases

Data Logging

Scripting (Bash/Python/Go/Ruby)

Azure

Gitlab

GIT

Git Flow

Kubernetes

Infrastructure Automation Frameworks

Bicep

Terraform

Data Pipelines

Docker

Databricks

Job description

Design, implement, and maintain CI/CD pipelines for data pipelines and infrastructure using Gitlab, Azure DevOps, or GitHub Actions.

Develop automation using Terraform modules or other IaC tools to deploy and manage Databricks Workspaces and other cloud resources.
Provision and manage Azure Analytics services (Databricks, Synapse Analytics, Data Factory, ADLS, Azure Monitor) using Infrastructure as Code (Terraform, ARM, Bicep).
Deploy, operate, and optimize Azure data services, including performance tuning, monitoring, and troubleshooting.
Automate routine operational tasks and deployments using scripting languages (Python, PowerShell, Bash).
Implement security best practices, including IAM, encryption, and data governance compliance (e.g., GDPR).
Collaborate with data engineers, data scientists, and business analysts to deliver robust technical solutions and foster a DevOps culture.

Requirements

Deep expertise in Microsoft Azure and/or AWS cloud services, especially Databricks.

Proficiency with DevOps tools: Gitlab, Azure DevOps, GitHub Actions, Git, and branching strategies.
Strong experience with Infrastructure as Code (Terraform, ARM, Bicep).
Experience with Docker and container orchestration (AKS) highly beneficial.
Proficiency in programming and scripting languages (Python, SQL, PowerShell, Bash).
Understanding of Data Engineering fundamentals (ETL/ELT, data modeling, data warehousing, big data).
Experience with monitoring, logging, and alerting tools (Azure Monitor, Log Analytics, App Insights).
Excellent problem-solving, analytical, and communication skills; ability to work in agile environments.
Preferred certifications: Azure Data Engineer Associate (DP-203) or Azure DevOps Engineer Expert (AZ-400).