> Markdown version of [/jobs/ext/3820956-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/3820956-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** Delta Air Lines, Inc. - **Location:** Atlanta, GA, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Training Data, Application Programming Interfaces (APIs), Artificial Intelligence, Amazon Web Services, Amazon S3, Data Analysis, Application Integration Architecture, Automation of Tests, Cloud Database, Configuration Management Databases, Configuration Management, Code Review, Encodings, Databases, Continuous Integration, Data as a Services, Data Deduplication, Data Governance, Extract Transform Load (ETL), Data Mapping, Data Transformation, Data Warehousing, Database Schema, Database Testing, Federated Identity Management, Identity and Access Management, Information Retrieval, Python (Programming Language), PostgreSQL, Machine Learning, OAuth, Parsing, Performance Tuning, Openid Connect, Bedrock Knowledge Bases, Cloud Services, Runbook, Search Technologies, Software Deployment, SQL Stored Procedures, SQL Databases, Systems Integration, Unstructured Data, Management of Software Versions, Enterprise Data Management, Parquet, AWS Cdk, Data Logging, Data Classification, Sql Optimization, Great Expectations (Foster Youth College-readiness and Support Program), Retrieval-Augmented Generation, Large Language Models, Prompt Engineering, Hallucination Detection, Model Validation, AgentCore, Generative AI, AWS Lambda, Indexer, Git, Cloudformation, Servicebus, Pgvector, Infrastructure Automation Frameworks, Information Technology, Amazon Bedrock, AWS Lake Formation, AWS Glue, AWS Data Analytics, Data Management, Sumo Logic (Software), Cloud Migration, Cloudwatch, Amazon OpenSearch, Software Version Control, Data Pipelines, Dynatrace, Pagerduty, Servicenow, Amazon Redshift - **Published:** October 18, 2026 - **Apply:** https://dejobs.org/x/x/38AB098788DF4A2D8741EAFE71909D5E/job/ ## About the Role * Is at least 18 years of age and has authorization to work in the United States. * Possesses a high school diploma, GED, or high school equivalency. * Embraces diverse people, thinking, and styles. * Consistently prioritizes safety and security of self, others, and personal data. * Strong analytical and interpersonal skills, with the ability to develop effective relationships and influence without direct authority. * Superior written and verbal communication skills, as well as executive presence to be an effective representative of the business across the organization. * 7+ years of IT experience, including 4+ years of hands-on experience designing, developing, and supporting enterprise data pipelines, data platforms, lakehouses, or analytical data solutions. * Demonstrated experience designing and implementing cloud-based data lake, data warehouse, or lakehouse architectures using AWS services. * Hands-on experience with Amazon S3, AWS Glue, Amazon Athena, Amazon Redshift, AWS Lambda, Amazon EventBridge, AWS Lake Formation, AWS IAM, and related data services. * Advanced proficiency in Python and SQL for ingestion, transformation, automation, analysis, data services, and performance optimization. * Experience designing scalable ETL and ELT pipelines integrating APIs, databases, structured files, semi-structured content, and cloud services. * Strong experience with dimensional, relational, normalized, denormalized, and analytics-oriented data modeling, including layered raw, curated, published, and analytics architectures. * Strong understanding of incremental ingestion, change-data capture, watermarking, deduplication, record versioning, historical tracking, late-arriving data, schema evolution, and reconciliation * Experience implementing metadata, lineage, cataloging, source-to-target mapping, data-quality testing, freshness monitoring, logging, alerting, and exception handling. * Working knowledge of Amazon SageMaker, Amazon Bedrock, foundation models, LLM application architectures, RAG, embeddings, vector search, semantic retrieval, and AI-ready data preparation. * Experience preparing structured and unstructured data through parsing, chunking, enrichment, embedding generation, indexing, access filtering, and context management. * Strong understanding of data governance, data classification, privacy, encryption, retention, identity, authorization, and secure handling of personal and sensitive information. * Experience with Amazon Redshift development concepts, including SQL optimization, workload performance, distribution and sort strategies, materialized views, stored procedures, access controls, and columnar formats such as Parquet. * Experience with Git, automated testing, CI/CD, Infrastructure as Code such as AWS CDK, or CloudFormation. * Ability to communicate complex technical concepts, data limitations, architecture decisions, AI data requirements, and implementation plans to technical and business audiences. * Demonstrated ability to lead technical design, mentor engineers, conduct code and architecture reviews, establish reusable standards, and manage work across concurrent Agile initiatives. * Consistent commitment to safety, security, responsible AI, operational reliability, and protection of personal and sensitive data. What will give you a competitive edge (preferred qualifications) * AWS Certified Data Engineer, AWS Certified Solutions Architect, AWS Certified Machine Learning Engineer, or another relevant AWS certification. * Hands-on experience with Amazon SageMaker, Amazon Bedrock, Bedrock Knowledge Bases, Bedrock Agents or AgentCore, Amazon OpenSearch Service, foundation models, embedding models, and production generative AI architectures. * Experience implementing enterprise RAG solutions, semantic search, hybrid retrieval, reranking, retrieval evaluation, grounding, citations, and AI-powered knowledge repositories. * Experience designing or integrating vector databases and vector-capable platforms such as Amazon OpenSearch Service, PostgreSQL with pgvector, or comparable technologies. * Experience with prompt engineering, model evaluation, guardrails, hallucination reduction, responsible AI controls, observability, and secure AI deployment patterns. * Deep knowledge of ServiceNow data, APIs, table structures, reference fields, ITSM processes, configuration management data, and CMDB relationships. * Experience integrating observability and operational platforms such as Dynatrace, PagerDuty, Amazon CloudWatch, ServiceNow, Sumo Logic, or similar enterprise services * Experience implementing OAuth 2.0, OpenID Connect, identity federation, role-based access controls, and document- or row-level authorization for data and AI retrieval * Experience with automated data testing and quality frameworks such as AWS Glue Data Quality, Great Expectations, dbt tests, or comparable technologies * Demonstrated history of leading data modernization, cloud migration, operational intelligence, platform engineering, generative AI, or AI-enablement initiatives ## Description The Senior Data Engineer role supports IT Service Assurance by leading the design and delivery of secure, scalable, and AI-ready data solutions that strengthen operational intelligence, service reliability, automation, and decision-making. Operating with a high degree of autonomy, this position translates IT operational priorities into trusted data products, reusable integration patterns, and modern engineering capabilities that connect signals across ITSM, CMDB, observability, application, and infrastructure platforms. This role partners across technology, operations, analytics, security, and architecture teams to improve data quality, enable self-service access, advance automation, and create the foundation for reporting, predictive insights, semantic search, and intelligent enterprise solutions * Lead independently across complex data initiatives, proactively identifying improvement opportunities, shaping solutions, and driving delivery from concept through production. * Design and modernize secure, scalable AWS data platforms, pipelines, Lakehouse patterns, and reusable data products aligned to enterprise architecture and governance standards. * Provide senior technical leadership for ingestion, transformation, modeling, metadata, quality, lineage, performance optimization, and production reliability. * Build and optimize batch, incremental, event-driven, and near-real-time integrations across APIs, databases, files, ServiceNow, CMDB, observability, telemetry, and enterprise platforms. * Develop efficient, maintainable Python and SQL for data pipelines, orchestration, transformations, analytics layers, APIs, automation, and reusable engineering frameworks. * Establish trusted data products through automated controls for completeness, accuracy, freshness, reconciliation, monitoring, alerting, and exception handling. * Advance AI-ready architectures that prepare structured and unstructured data for Amazon SageMaker, Amazon Bedrock, LLMs, RAG, vector databases, semantic search, agents, and MCP-enabled solutions. * Design retrieval, embedding, indexing, enrichment, and context-management patterns that improve data usability, discoverability, and responsible AI outcomes. * Drive operational excellence through CI/CD, Infrastructure as Code, automated testing, source control, code reviews, runbooks, production monitoring, and resilient deployment practices. * Apply security, privacy, identity, access control, encryption, retention, data classification, and responsible AI requirements across the data lifecycle. * Influence stakeholders, mentor engineers, promote reusable standards, evaluate emerging technologies, and guide modernization efforts that improve scalability, automation, cost, and business value.