Data Engineer

Optomi LLC
Atlanta, United States of America
3 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Remote
Atlanta, United States of America

Tech stack

API
Artificial Intelligence
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Big Data
Cloud Computing
Cloud Engineering
Continuous Integration
Data Retrieval
Distributed Systems
Amazon DynamoDB
Elasticsearch
Github
Identity and Access Management
Python
Machine Learning
Search Technologies
Software Deployment
Software Engineering
Systems Integration
Retrieval-Augmented Generation
Large Language Models
Reliability of Systems
Amazon Web Services (AWS)
Documentation System
Event Driven Architecture
Pytest
Collibra
Kafka
Data Management
Machine Learning Operations
Api Gateway
Terraform
Marketplace
Software Version Control
Data Pipelines
Serverless Computing

Job description

  • We are seeking a Senior Software Engineer - Data Marketplace to help build a modern, enterprise-scale data platform that makes data easy to discover, understand, and consume. This role focuses on developing scalable services, search and catalog capabilities, and AI-powered data experiences.
  • This is a hands-on individual contributor role working across cloud infrastructure, data platforms, and advanced search/ML systems. You'll collaborate with cross-functional teams to deliver high-impact solutions that improve how users find and use data., * Design, develop, and maintain Python-based services, APIs, and data pipelines
  • Build and manage AWS infrastructure using Terraform
  • Develop and enhance search and discovery capabilities using OpenSearch
  • Implement AI/ML-powered features, including semantic search and data retrieval
  • Collaborate with engineering, data, and security teams to deliver end-to-end solutions
  • Work across AWS services such as Lambda, API Gateway, Glue, S3, DynamoDB, IAM, and VPC
  • Improve system reliability through testing, CI/CD, and observability best practices
  • Contribute to architectural decisions and technical standards
  • Mentor junior engineers and share best practices
  • Solve complex problems with ambiguous requirements and propose scalable solutions

Requirements

  • 5+ years of professional software development experience
  • Strong proficiency in Python (including testing frameworks like pytest/unittest)
  • Experience building and deploying applications on AWS
  • Hands-on experience with Terraform for Infrastructure as Code
  • Experience with Amazon OpenSearch or Elasticsearch, including search/query capabilities
  • Familiarity with AI/ML concepts such as embeddings, semantic search, or model integration
  • Experience working with serverless and cloud-native architectures (e.g., Lambda, API Gateway)
  • Strong understanding of data pipelines and distributed systems
  • Experience with version control (e.g., GitHub) and documentation tools

Nice to Haves

  • Experience with vector search, RAG (Retrieval-Augmented Generation), or LLM-based systems
  • Exposure to data catalog or governance platforms (e.g., Collibra)
  • Experience with event-driven architectures (e.g., Kafka)
  • Familiarity with additional AWS services such as Lake Formation
  • Experience working in large-scale data platform or marketplace environments

Apply for this position