Data Engineer
Role details
Job location
Tech stack
Job description
- The candidate does not have the needed experience with Kubernetes which is required from the role.
- We interviewed this candidate and he could not pass the technical evaluation (could not answer a single technical question).
- The candidate does not have the required on-prem Kubernetes experience., A Data Engineer is responsible for implementing, building, testing, and maintaining the infrastructure and architecture required for the efficient and reliable processing of various types of data. Selected candidate works with Data Analysts, Data Scientists, and other stakeholders to understand the data requirements, identify data sources, and implement the necessary processes to transform and load the data into the target systems. They are also responsible for ensuring the integrity, availability and security of the data throughout its lifecycle, and for continuously optimizing the data pipelines to meet the evolving business needs.
Requirements
-
Based on the CV, the candidate does not have enough experience and knowledge in some for the tools required for the role.
-
Based on the CV, the candidate lacks relevant data engineering experience.
-
Based on the CV, the candidate does not have the needed experience and background in data engineering.
-
Based on the CV, the candidate does not have enough experience in the domain of data engineering, especially in the tools required for the role.
-
Based on the CV and a video interview, the candidate does not have knowledge in tools required for the role.
-
Based on the CV, the candidate lacks experience in some of the tools required for this role, such as on-prem Kubernetes.
-
The candidate does not have experience in some of the tools required for the role, such as Kubernetes (on prem).
-
Based on the CV and a video interview, the candidate lacks experience in some of the tools required for the role, such as on-prem Kubernetes., * NATO Secret security clearance
-
A minimum requirement of a Bachelor's degree at a nationally recognised/certified University in a related discipline, such as Computer Science, Data Science and Engineering, and post related experience.
-
Or exceptionally, the lack of a university degree may be compensated by the demonstration of a candidate's particular abilities or experience that is/are of interest to NCIA, that is, at least 6 years extensive and progressive expertise in duties related to the function of the post.
-
Strong understanding of data engineering concepts, including data warehousing, ETL/ELT processes and data governance
-
Experience in planning and maintaining data lakes and pipeline processes, both batch-wise and (near) real-time;
-
Experience in using of data engineering tools to support development of data science, data analytics and AI applications for both, structured and unstructured, data
-
Experience with monitoring and logging tools to ensure the reliability and performance of data pipelines and AI systems.
-
Solid understanding of containerization and orchestration tools such as Docker and Kubernetes.
-
Experience with cloud platforms such as AWS, Google Cloud or Azure
-
Solid understanding of good software development principles such as version control, CI/CD, unit/functional testing.
-
Strong proficiency in Python for building and integrating various data engineering components
-
Solid understanding of data security best practices, including data encryption, access controls, authentication/authorization mechanisms, and compliance requirements.
