TELECOMMUTE Data Engineer AWS OpenSearch

Propertyvalue Quantum Technologies Llc

United States

20 days ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Temporary contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

5 years minimum

Compensation

$92,000.0

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Query Performance Amazon Web Services Amazon Elastic Compute Cloud Amazon S3 Bioinformatics Cloud Engineering Continuous Integration Data as a Services Extract Transform Load (ETL) Distributed Systems Elasticsearch Identity and Access Management

+26 more

Python (Programming Language) Performance Tuning Scrum Methodology Query Optimization Prometheus Standard Sql Search Technologies SQL Databases Snowflake Grafana Database Optimization Apache Spark Indexer Git Cloudformation Pyspark Semi-structured Data Infrastructure Automation Frameworks Data Analytics Apache Kafka Cloudwatch Terraform Splunk Data Pipelines Docker Databricks

Job description

We are seeking an experienced Data Engineer with strong expertise in AWS OpenSearch to design, develop, and maintain scalable data pipelines and search solutions. The ideal candidate will have hands-on experience with AWS data services, OpenSearch, Python, SQL, and cloud-native architectures to support analytics and real-time search capabilities., * Design, develop, and maintain scalable data pipelines using AWS services.

Build and optimize AWS OpenSearch clusters for indexing, search, and analytics.
Develop ETL/ELT pipelines using Python, SQL, and AWS services such as Glue, Lambda, and EMR.
Design and maintain OpenSearch indexes, mappings, analyzers, and query performance.
Integrate OpenSearch with upstream and downstream applications for real-time search and analytics.
Work with structured and semi-structured data from multiple data sources.
Optimize OpenSearch cluster performance, shard allocation, indexing, and query execution.
Implement Infrastructure as Code (Terraform or CloudFormation) for AWS resources.
Build monitoring and alerting using CloudWatch, Grafana, or Prometheus.
Collaborate with Data Scientists, Data Analysts, and application teams to support data and search requirements.
Implement security best practices, IAM policies, encryption, and access controls.
Troubleshoot production issues and optimize system performance.

Requirements

5+ years of experience as a Data Engineer.
Strong experience with AWS OpenSearch (formerly Amazon Elasticsearch Service).
Strong SQL and Python programming skills.
Experience with AWS services:
OpenSearch
S3
Glue
Lambda
IAM
CloudWatch
EC2
EMR
Step Functions
Kinesis (preferred)
Experience building ETL/ELT pipelines.
Experience with Docker and Kubernetes (preferred).
Knowledge of distributed search architecture, indexing strategies, and query optimization.
Experience with Git, CI/CD, and Infrastructure as Code (Terraform or CloudFormation).
Familiarity with Agile/Scrum methodologies.

Preferred Qualifications

Experience with Apache Spark or PySpark.
Experience with Kafka or Amazon MSK.
Experience with Snowflake, Redshift, or Databricks.
Experience working with large-scale distributed systems.
AWS Certification (Solutions Architect, Data Analytics, or Developer) is a plus.
Experience with observability tools such as Grafana, Prometheus, or Splunk. Equal Opportunity Employer: We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.