Senior Data Engineer
Role details
Job location
Tech stack
Job description
We are looking for a highly skilled Senior Data Engineer for our Data and Cloud Engineering team with expertise in AWS and Azure. The ideal candidate will have a strong technical background in designing, building, developing, and implementing data pipelines and cloud solutions along with great technical guidance and communication skills. Your daily tasks...
-
Design and implement robust, scalable, and secure cloud-based data pipelines and architectures in AWS (later maybe migrating to MS Azure).
-
Ensure best practices in code quality, architecture, and design.
-
Design and implement secure, scalable, and high-performance cloud infrastructure.
-
Manage cloud resources, optimize costs, ensure high availability and disaster recovery.
-
Automate infrastructure provisioning and deployment processes using Infrastructure as Code (IaC) tools such as Terraform, CloudFormation, and ARM templates.
-
Collaborate with cross-functional teams to understand data needs and deliver comprehensive cloud solutions.
-
Oversee cloud infrastructure management, including monitoring, maintenance, and scaling of cloud resources.
-
Ensure compliance with industry standards and regulatory requirements.
-
Implement data governance policies and practices and ensure high data quality, integrity, and security across all cloud platforms.
-
Identify and implement process improvements to enhance efficiency, quality, and scalability of data engineering and cloud operations.
-
Stay current with emerging technologies and industry trends to drive innovation. Utilize AI to increase our efficiency. Our tech-stack...
-
Infrastructure: Glue, Lambda, Step Function, Batch, ECS, Quicksight, Machine Learning, Sagemaker, Dagster
-
DevOps: Cloudformation, Terraform, Git, CodeBuild
-
Database: Redshift, PostgreSQL, DynamoDB, Athena (Trino), Snowflake, Databricks
-
Language: Bash, Python (PySpark, Pydantic, PyArrow), SQL
Requirements
- Min. 5 years of proven experience as a Data Engineer with proven track record of delivering production-grade data pipelines.
- Hands-on experience with AWS cloud platform and its data services (, AWS Redshift, AWS Glue, AWS S3, Azure Data Lake, Azure Synapse, Snowflake, Databricks).
- Strong understanding of ETL processes, data warehousing, and big data technologies.
- Proficiency in SQL and Python, comfortable with Spark jobs.
- Experience with infrastructure as code (IaC) tools such as Terraform, CloudFormation, or ARM templates.
- Knowledge of containerization and orchestration (, Docker, Kubernetes).
- Understanding of cloud cost management and optimization strategies.
- Familiarity with CI/CD pipelines and DevOps practices.
- Strong problem-solving and analytical skills.
- Familiarity with data visualization tools (, Power BI, QuickSight) is a plus.
- Openness to use AI, in our case Cursor, as your daily tool.