> Markdown version of [/jobs/ext/1613671-opensearch-engineer-aws-data-pipelines](https://www.wearedevelopers.com/jobs/ext/1613671-opensearch-engineer-aws-data-pipelines). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # OpenSearch Engineer / AWS & Data Pipelines - **Company:** Ruby Labs - **Location:** France (Remote available) - **Contract:** Permanent contract - **Skills:** Java (Programming Language), Amazon Web Services, Amazon S3, Big Data, Databases, Data Deduplication, Digital Architecture, Disaster Recovery, Elasticsearch, Identity and Access Management, Java Virtual Machine (JVM), Python (Programming Language), PostgreSQL, Role-Based Access Control, Data Streaming, Management of Software Versions, Backend, Vertica, Functional Programming, Cloudwatch, Amazon Simple Queue Service (SQS), Terraform, Data Pipelines - **Published:** July 31, 2026 - **Apply:** https://fr.indeed.com/viewjob?jk=2bf1c7e606830cbe ## About the Role * Strong production experience with OpenSearch or Elasticsearch. * Hands-on experience with AWS OpenSearch Service. * Experience building data pipelines between PostgreSQL and OpenSearch. * Strong understanding of: + Index and shard design + Mappings and analyzers + OpenSearch Query DSL + Aggregations, sorting, filtering, and pagination + Bulk indexing and cluster performance + Data consistency, retries, idempotency, and replay * Experience with large datasets and high indexing or query volumes. * Experience performing backfills and reindexing safely. * Strong troubleshooting and production-operations skills., * Experience with AWS DMS or OpenSearch Ingestion. * Experience with Lambda, SQS, S3, ECS, EKS, or Terraform. * Experience building custom ingestion services in Go, Java, or Python. * Experience in payments, billing, fintech, or subscription systems. * Experience with multi-tenant or high-volume search platforms. * Familiarity with ClickHouse, Tinybird, or similar analytical systems. * AWS certifications. ## Description We are building a high-scale payment orchestration platform that processes millions of financial transactions every month across multiple providers, regions, and products., OpenSearch * Own and operate AWS OpenSearch Service in production. * Design indices, mappings, analyzers, templates, aliases, shards, and retention policies. * Optimize search queries, aggregations, filtering, pagination, and bulk indexing. * Investigate slow queries, indexing bottlenecks, hot shards, memory pressure, and rejected requests. * Plan safe upgrades, migrations, reindexing, and capacity scaling. PostgreSQL-to-OpenSearch pipelines * Build and maintain reliable pipelines that synchronize PostgreSQL data with OpenSearch. * Support inserts, updates, deletes, backfills, and full reindexing. * Use AWS-native services such as AWS DMS, OpenSearch Ingestion, Lambda, SQS, S3, ECS, or EKS where appropriate. * Implement retries, idempotency, deduplication, replay, dead-letter handling, and failure recovery. * Monitor ingestion lag, throughput, failed records, and data freshness. * Build reconciliation processes to detect and repair missing, stale, or inconsistent documents. * Ensure ingestion does not negatively affect the transactional PostgreSQL workload. Reliability and operations * Monitor cluster health, search latency, indexing latency, storage, CPU, JVM pressure, and shard distribution. * Build CloudWatch dashboards, alerts, and operational runbooks. * Design snapshot, restore, backup, and disaster-recovery procedures. * Participate in production incident response and root-cause analysis. * Implement secure access using IAM, role-based access control, encryption, and least-privilege principles. Collaboration * Work closely with backend, database, infrastructure, and product teams. * Help design efficient OpenSearch document models for payments, customers, subscriptions, and transactions. * Define standards for searchable fields, schema changes, and index versioning. * Document architecture, data flows, and recovery procedures. ## Related Videos - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [Optimizing Discovery: PostgreSQL's Role in Transforming GetYourGuide's Search](https://www.wearedevelopers.com/videos/1647-optimizing-discovery-postgresql-s-role-in-transforming-getyourguide-s-search) - [Developing the Backend with Stefan Lingler, CTO at Shpock](https://www.wearedevelopers.com/videos/100360-developing-the-backend-with-stefan-lingler-cto-at-shpock) - [Search and aggregations made easy with OpenSearch and NodeJS](https://www.wearedevelopers.com/videos/490-search-and-aggregations-made-easy-with-opensearch-and-nodejs) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Nest.js - TypeScript in the backend can also be clean](https://www.wearedevelopers.com/videos/1033-nest-js-typescript-in-the-backend-can-also-be-clean) ## Related Articles - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Find a Developer Job: 12 Best Job Sites For Developers](https://www.wearedevelopers.com/magazine/165-find-a-developer-job-12-best-job-sites-for-developers) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline)