Job Title: Data Engineer

The Meta Game, Inc.
2 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Tech stack

Artificial Intelligence
Airflow
Amazon Web Services (AWS)
Azure
Cloud Computing
Cloud Engineering
Databases
Information Engineering
Data Governance
ETL
Data Systems
Python
Machine Learning
Meta-Data Management
Software Tools
Cloud Services
Software Engineering
Data Streaming
Systems Integration
Workflow Management Systems
Data Storage Technologies
Data Ingestion
FastAPI
Pandas
Build Management
Containerization
PySpark
Kubernetes
Storage Technologies
Information Technology
Kafka
Machine Learning Operations
Data Pipelines
Docker

Job description

We are looking for a skilled Data Engineer to join our team. The Data Engineer will be responsible for designing, building, and maintaining robust data pipelines that enable scalable, secure, and intelligent data processing across cloud environments. The ideal candidate will have hands-on experience in data acquisition from diverse sources, modern data storage paradigms, and a passion for building pipelines that support advanced analytics and AI/ML use cases., * Data Ingestion & Acquisition: Collect and integrate data from a wide variety of sources.

  • Pipeline Development: Design and implement scalable ETL/ELT pipelines.
  • Cloud Deployment: Build and deploy data pipelines on AWS or Azure.
  • Database Architecture: Optimize for different storage engines.
  • Streaming Data Processing: Handle high-volume data streams.
  • Workflow Orchestration: Schedule and monitor data workflows.
  • AI/ML Integration: Collaborate with data scientists to integrate ML models into pipelines.

Requirements

  • Bachelor''s or master's degree in computer science, Engineering, or related field.
  • 10+ years of experience in data engineering or software development roles.
  • Strong proficiency in Python, including experience with libraries like pandas, PySpark, FastAPI, or similar.
  • Experience with cloud services (AWS or Azure) and Cloud native data engineering tools.
  • Experience in building and maintaining data pipelines using Kafka, Airflow, and other open-source frameworks.
  • Strong grasp of database internals and trade-offs between different storage technologies.
  • Familiarity with data governance, lineage, and metadata management concepts.
  • Experience or strong interest in integrating ML models into production-grade data systems., * Knowledge of data cataloging tools and semantic layer design.
  • Experience with containerization (Docker) and orchestration (Kubernetes).
  • Familiarity with MLOps tools or platforms (e.g., SageMaker, MLflow).
  • Prior experience working in regulated or secure environments.

Apply for this position