Lead AWS Glue Developer

We are seeking a Lead AWS Glue Developer to design, build, and maintain scalable cloud-native data integration and ETL solutions within AWS. This role will lead the development of serverless data pipelines, automate cloud infrastructure deployment, and build robust data processing frameworks leveraging AWS Glue, AWS Lambda, DynamoDB, S3, and related AWS services. The ideal candidate will have deep expertise in AWS data engineering, Infrastructure as Code (IaC), Python development, and modern DevOps practices. Experience with AWS Glue, Lambda, CloudFormation, S3, and DynamoDB aligns closely with capabilities referenced in internal AWS engineering profiles., AWS Data Engineering & ETL Development Design, develop, and optimize AWS Glue ETL jobs using Python and Apache Spark. Build scalable batch and event-driven data processing solutions. Develop and maintain data pipelines that ingest, transform, validate, and load data into AWS data stores. Create and manage AWS Glue Data Catalogs, crawlers, and metadata management processes. Implement data quality controls, monitoring, and exception handling throughout ETL workflows. Serverless Application Development Develop AWS Lambda functions to automate file processing and event-driven workflows. Build solutions that process files dropped into S3 buckets and orchestrate movement of data into DynamoDB tables and downstream systems. Implement integrations between S3, Lambda, DynamoDB, and other AWS services. Troubleshoot, optimize, and enhance serverless applications for performance and reliability. Cloud Infrastructure & DevOps Create and maintain AWS CloudFormation templates to automate infrastructure provisioning and deployment. Configure and manage AWS CodeCommit repositories and CI/CD pipelines. Establish and maintain AWS environments following Infrastructure-as-Code best practices. Set up and administer S3 buckets, SFTP services, access controls, encryption, and lifecycle management policies. Collaborate with DevOps teams to automate deployments and improve platform reliability. Technical Leadership Provide technical leadership and architectural guidance to development teams. Lead code reviews and establish development standards and best practices. Mentor junior developers and data engineers. Collaborate with architects, business stakeholders, and product teams to define technical solutions. Drive performance tuning, scalability improvements, and cloud modernization initiatives. Testing & Quality Engineering Write comprehensive unit, integration, and regression tests for AWS Lambda functions and AWS Glue jobs. Implement automated testing strategies within CI/CD pipelines. Ensure high code quality, security compliance, and operational excellence.

8+ years of software engineering or data engineering experience. 5+ years of hands-on AWS development experience. Strong expertise with: AWS Glue AWS Lambda Amazon S3 Amazon DynamoDB AWS CloudFormation AWS CodeCommit Python Apache Spark / PySpark Experience building serverless and event-driven architectures. Strong understanding of JSON data structures and APIs. Experience developing and deploying Infrastructure-as-Code solutions. Experience building and supporting ETL and data integration pipelines. Strong troubleshooting, debugging, and performance optimization skills. Experience writing automated unit and integration tests. Preferred Qualifications AWS Certified Data Engineer, Solutions Architect, or Developer certification. Experience with additional AWS services such as Step Functions, EventBridge, SNS, SQS, CloudWatch, or Redshift. Experience working in Agile/Scrum environments. Familiarity with GitHub and modern source control practices. Knowledge of data governance, security, and compliance requirements. Experience leading development teams and technical projects. AI & Productivity Expectations Comfortable using GitHub Copilot to accelerate software development and improve developer productivity. Ability to leverage AI-assisted coding tools while maintaining code quality, security, and performance standards. Interest in applying emerging AI-enabled development practices to cloud engineering and data engineering workflows.

Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.