Azure Data Engineer

Antal International

Municipality of Madrid, Spain

19 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Languages

English

Job location

Remote

Municipality of Madrid, Spain

Tech stack

Java

Agile Methodologies

Data analysis

Azure

Big Data

Cloud Computing

Continuous Integration

Distributed Systems

Hadoop

Hadoop Distributed File System

HBase

Hive

Python

Shell

Machine Learning

NoSQL

Scala

Spark

GIT

Information Technology

Spark Streaming

Software Version Control

Data Pipelines

Jenkins

Databricks

Job description

Data Engineer/Big Data Developer to support all NBx use cases and foundation products, with the end-to-end responsibility, to retrieve, store and model data, develop and maintain data pipelines, and develop solutions to support the use cases. DUTIES

Data reporting and visualization to support business operations and decisions.
Design data structures and write code to combine and transform datasets to meet business needs.
Carry out quality controls on output data for accuracy.
Perform statistical analyses on a wide range of datasets.
Maintain a working knowledge of algorithmic structures and tactics, as well as programming languages and methodologies.
Devise and apply quality control testing in project work.
Determine the best way to complete workstreams within assigned schedules and project objectives.
Build new datasets proactively; seek out, develop, and share new algorithms and best practices.
Evaluate projects, make revisions to suggested project plans when resources and unanticipated events become available, and explain decisions and rationales to clients.
Perform other duties as assigned.

Requirements

Bachelor/Master´s in Computer Sc./App or Information Technology
Technical understanding of big data Cloud platforms such as MS Azure.
Desirable experience on version control (Git), Data Modelling, Data Pipelines, Solution Development, Product Industrialization, CI/CD (Jenkins or Git hub)
Experience using analytics programming languages (Python, Spark, Scala, Java…) to manipulate data and draw insights from large data sets.
Knowledge in machine learning techniques.
Valued Skills: o Experience in Unix Shell scripting. o Experience in agile software development. o Experience with Spark, SQL, NoSQL, Databricks o Knowledge in distributed systems and frameworks such as Hadoop, HBase, NoSQL. Spark SQL, Spark Streaming, HDFS, Hive.
Strong critical thinking and problem-solving skills.
Ability to work individually without much direction while also working as part of a team towards a common goal.
Time management skills.
English level: Fluent reading/writing/conversation