Sr. AWS Data Engineer

• Design, build, and support scalable data pipelines using AWS Glue, Lambda, and EventBridge for event-driven and batch processing.
• Build and maintain AWS Glue ETL jobs using PySpark/Python for ingestion, transformation, and curation across data lake layers.
• Develop and manage Glue Workflows and AWS orchestration to coordinate workflows, triggers, and dependencies across AWS services.
• Implement near real-time data processing patterns using AWS Lambda and EventBridge.
• Write, optimize, and maintain complex Postgres SQL for validation, transformation, reporting, and performance tuning.
• Manage metadata and schema definitions in AWS Glue Data Catalog to support governance and discoverability.
• Monitor and troubleshoot pipeline performance using CloudWatch, logs, alerts, and AWS monitoring tools.
• Contribute to system reliability, scalability, and performance optimization of the data platform.

• 10+ years of data engineering experience with production pipelines, ETL/ELT, and large-scale data migrations.
• Strong hands-on Python and PySpark/Spark development experience, including AWS Glue ETL jobs for ingestion, transformation, and curation across data lake layers.
• Production experience with AWS data services such as Glue, S3, IAM, Lambda, EventBridge, CloudWatch, logs/alerts, Secrets Manager, and Glue Data Catalog.
• Experience with batch and event-driven data architectures, including near real-time processing patterns and pipeline design.
• Glue Workflow and AWS Orchestrator experience, including workflow orchestration, triggers, dependencies, and cross-service orchestration across AWS services.
• Advanced SQL skills with Postgres/PostgreSQL and/or Amazon Aurora, including complex transformations, performance tuning, validation, reporting, loading, and schema management.
• Ability to manage metadata and schema definitions using AWS Glue Data Catalog to support governance and discoverability.
• Strong understanding of migration reliability practices, including delta loads, reconciliation, error handling, restart/checkpointing, rollback planning, and idempotency.
• Knowledge of data security, PII handling, encryption, tokenization, least-privilege access, secrets management, and metadata governance.
• Hands-on CI/CD and Git experience using GitHub Actions, Octopus Deploy, or similar tools for deployment automation.
• Infrastructure as Code experience with Terraform or similar tools.
• Ability to monitor, troubleshoot, support, and remediate production data pipeline issues independently using AWS monitoring tools.
• Ability to contribute to system reliability, scalability, and performance optimization of the data platform.
• Experience using AI-assisted development tools such as GitHub Copilot, Microsoft Copilot, or Cursor AI to improve coding productivity, documentation, testing, and troubleshooting.
• Strong collaboration skills with BI, analytics, Snowflake, and downstream data consumers.

Preferred Skills

• Snowflake experience, including data consumption, warehouse integration, and collaboration with downstream analytics or BI teams.
• Experience with Athena Federated Query, Glue crawlers, Lake Formation, SNS, CloudTrail, Parquet, Iceberg, or Delta.
• OAuth 2.0, secure API integration, tokenization, or regulated-data experience.
• Financial services background; ServiceNow or formal change-management experience preferred.
• Data modeling, partitioning, schema evolution, test-driven development, or static-analysis experience.
• AWS certification is a plus.