Data Engineer
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+32 more
Job description
We are seeking a skilled Data Engineer to design, build, and optimize modern data solutions that support business operations and decision-making. This role combines data engineering, software development, and emerging AI technologies to create scalable, reliable, and efficient data systems., * Design, build, and maintain scalable data pipelines and ETL/ELT processes.
- Develop automated data collection, integration, and web scraping solutions.
- Create and manage data models, validation frameworks, and monitoring processes.
- Build APIs and internal applications that improve data accessibility and usability.
- Collaborate with technical and business stakeholders to deliver data-driven solutions.
- Leverage AI and automation tools to improve engineering workflows and productivity.
- Support architectural improvements and the ongoing evolution of the data platform.
- Ensure data quality, integrity, performance, and security across all systems.
Requirements
The ideal candidate has strong experience with data pipelines, cloud platforms, automation, and Python development, along with a passion for solving complex technical challenges., * 5+ years of experience in Data Engineering, Software Engineering, or a related field.
- Strong proficiency in Python and SQL.
- Experience with data processing libraries such as Pandas and NumPy.
- Hands-on experience designing and supporting production ETL/ELT workflows.
- Experience with workflow orchestration tools such as Airflow, Dagster, Prefect, or similar platforms.
- Experience with cloud-based data platforms such as Snowflake, Databricks, BigQuery, or Redshift.
- Strong understanding of data modeling, transformation, validation, and quality management.
- Experience building automated data acquisition or web scraping solutions.
- Familiarity with AI-powered development tools and machine learning or LLM integrations.
- Strong problem-solving skills and the ability to work independently in a fast-paced environment., * Experience with TypeScript or JavaScript.
- Experience with backend frameworks such as FastAPI, Django, Express, or NestJS.
- Familiarity with React, Next.js, or other modern web frameworks.
- Knowledge of AI applications, RAG architectures, and LLM-powered solutions.
- Experience with AWS or other cloud environments.
- Familiarity with Docker, Terraform, CI/CD pipelines, and Infrastructure as Code.
- Experience working with semi-structured data such as PDFs, spreadsheets, and contracts.
- Experience with vector databases such as Pinecone, Weaviate, or pgvector.
- Previous experience in a high-growth or startup environment.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦