> Markdown version of [/jobs/ext/3961418-performance-engineer](https://www.wearedevelopers.com/jobs/ext/3961418-performance-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Performance Engineer

- **Company:** ConglomerateIT LLC
- **Location:** Plano, TX, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Amazon Web Services, Amazon S3, Business Analytics Applications, Data Analysis, Automation of Tests, Batch Processing, Big Data, Computer Programming, Continuous Integration, Data as a Services, Data Validation, Information Engineering, Data Integrity, Extract Transform Load (ETL), Data Transformation, Data Security, Serialization, Query Languages, Software Debugging, DevOps, Distributed Computing Environment, Distributed Data Store, Distributed Systems, Amazon DynamoDB, Identity and Access Management, Python (Programming Language), Prometheus, Scala (Programming Language), SQL Databases, Enterprise Data Management, Data Processing, Grafana, Apache Spark, Software Troubleshooting, Reliability of Systems, Event Driven Architecture, Pyspark, Data Lineage, AWS Data Analytics, Apache Kafka, Video Streaming, Cloudwatch, Data Pipelines, SDET, Databricks
- **Published:** October 27, 2026
- **Apply:** https://www.careerjet.com/jobad/us55dceb2cc88eeb0476152ff498a3b216

## About the Role

7+ years of experience in Quality Assurance, Software Development Engineer in Test (SDET), Data Quality Engineering, or related testing roles.
Strong hands-on expertise in SQL for complex data analysis, reconciliation, and validation.
Proficiency in Python for test automation, data processing, and quality assurance.
Proven experience testing data pipelines, ETL/ELT processes, and distributed data processing workflows.
Practical experience with Apache Kafka or other event-streaming technologies.
Strong understanding of AWS data services, including S3, Glue, Redshift, Lambda, Athena, or equivalent platforms.
Experience with Databricks and Apache Spark-based data processing is highly desirable.
Knowledge of batch and streaming architectures, data transformations, schema validation, and end-to-end data lineage.
Experience testing large-scale datasets and distributed systems.
Strong troubleshooting, analytical thinking, debugging, and problem-solving capabilities.
Ability to collaborate effectively with cross-functional engineering and analytics teams. Technical Environment
Programming and Query Languages: Python, SQL, Scala
Data Processing: Databricks, Apache Spark, PySpark
Streaming Technologies: Apache Kafka
AWS Services: S3, Glue, Lambda, EMR, Step Functions, Redshift, Athena, Kinesis, DynamoDB
Automation and Delivery: Python-based test frameworks, CI/CD pipelines

## Description

We are looking for an experienced Data Quality Engineer to ensure the accuracy, integrity, reliability, and performance of enterprise data platforms and pipelines. This role will focus on validating data ingestion, transformation, processing, streaming, and analytics workflows across AWS-based environments.
The ideal candidate will have strong expertise in SQL, Python, Apache Kafka, Databricks, and AWS data services, with experience developing automated data validation frameworks and testing distributed data systems. You will collaborate with data engineering, platform, analytics, and DevOps teams to maintain high-quality data and ensure pipeline reliability across the data lifecycle. Key Responsibilities
Design and execute comprehensive test strategies for batch processing, streaming pipelines, ETL/ELT workflows, and data analytics solutions.
Develop automated data quality checks to identify missing records, duplicate entries, schema changes, inconsistencies, and referential integrity issues.
Validate business rules, data transformations, aggregations, and processing logic using SQL and Python.
Test Databricks-based data engineering workflows, including Apache Spark applications developed with PySpark or Scala.
Verify data accuracy and traceability across source systems, intermediate processing layers, and downstream data consumers.
Validate AWS-based data pipelines built using services such as S3, Glue, Lambda, EMR, Step Functions, Redshift, Athena, Kinesis, and DynamoDB.
Test Kafka streaming applications to ensure data integrity, event ordering, and appropriate message delivery guarantees.
Validate Kafka producers, consumers, serialization formats, topic configurations, partitioning, offsets, retention settings, and schema evolution.
Assess pipeline behavior under duplicate events, delayed messages, consumer failures, reprocessing scenarios, backfills, and historical data loads.
Develop and maintain reusable Python-based test automation frameworks, validation utilities, test fixtures, and synthetic datasets.
Integrate automated data quality tests into CI/CD workflows for pre-merge checks, scheduled executions, and post-deployment verification.
Implement automated notifications and reporting mechanisms to identify data quality failures and pipeline anomalies.
Evaluate data pipeline scalability and performance under high-volume workloads, including throughput, latency, concurrency, and resource utilization.
Test failure recovery, retry mechanisms, idempotency, error handling, regression scenarios, and system resilience.
Monitor pipeline health using CloudWatch, Prometheus, Grafana, or comparable monitoring and observability platforms.
Validate data security controls, IAM permissions, encryption configurations, storage lifecycle policies, and access restrictions.
Collaborate with engineering teams to establish data quality SLAs and SLOs and support incident investigations, root-cause analysis, and post-incident reviews. Required Qualifications