> Markdown version of [/jobs/ext/2026616-aws-big-data-lead-software-engineer](https://www.wearedevelopers.com/jobs/ext/2026616-aws-big-data-lead-software-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AWS Big Data Lead Software Engineer - **Company:** JPMorgan Chase & Co. - **Location:** Columbus, OH, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Agile Methodology, Artificial Intelligence, Airflow, Amazon Web Services, Amazon Elastic Compute Cloud, Amazon S3, Software Applications, Bash Shell, Big Data, Cloud Computing, Continuous Delivery, Continuous Integration, Extract Transform Load (ETL), Data Stores, Python (Programming Language), Korn Shell, Machine Learning, Performance Tuning, Scrum Methodology, Shell Script, Software Engineering, Data Streaming, Workflow Management Systems, Apache Spark, Electronic Medical Records, Build Management, Kubernetes, Apache Flink, Cassandra, Apache Kafka, Terraform, Data Pipelines - **Published:** August 11, 2026 - **Apply:** https://jpmc.fa.oraclecloud.com/hcmUI/CandidateExperience/en/sites/CX_1001/requisitions/preview/210711641 ## About the Role * Formal training or certification on software engineering concepts and 5+ years applied experience * Hands-on practical experience in Frameworks, system design, application development, testing, and operational stability * Experience with Apache Spark, Apache Flink or similar large-scale data processing engines * Experience with Distributed Datastores (e.g. Cassandra, Red Shift) * Experience designing, developing and deploying software components on AWS using common compute and storage services such as EC2, EKS, Lambda, S3 * Experience with Big Data / Distributed / cloud technology (AWS Big data services like lambda, glue, glue emr, Performance tuning, Streaming, KAFKA, Entitlements etc., ) * Experience with Apache Spark, Ray or similar large-scale data processing engines * Proficiency in automation and continuous delivery methods * Proficient in all aspects of the Software Development Life Cycle * Advanced understanding of agile methodologies such as CI/CD, Application Resiliency, and Security * Demonstrated knowledge of software applications and technical processes within a technical discipline (e.g., cloud, BigData, artificial intelligence, machine learning, mobile, etc.) Preferred qualifications, capabilities, and skills * Experience building ETL/Feature processing pipelines * Experience using workflow orchestration tools-Airflow, Kubeflow etc. * Experience using Terraform to deploy infrastructure-as-code to public cloud * Experience with Linux scripting such as Bash, KSH, or Python * Certified AWS Cloud Practitioner, Developer or Solutions Architect strongly preferred ## Description * Design & build new applications utilizing leading edge technologies and modernize existing applications * Implement batch & real-time software components consistent with architectural best-practices of reliability, security, operational efficiency, cost-effectiveness and performance * Ensure quality of deployed code via automated unit, integration & acceptance testing * Collaborate with multi-national agile development, support and business teams to meet sprint objectives * Participate in all agile meetings & rituals, including daily standups, sprint planning, backlog reviews, demos, and retrospectives * Provide level 2 support for production systems * Learn and applies system processes, methodologies, and skills for the development of secure, stable code and systems * Hands on applicaitn development leveraging distributed compute such as Apache flink or Spark on very large datasets * Design and development of applications that leverage the AWS infrastructure deploying software components on AWS using common compute and storage services such as EC2, EKS, Lambda, S3 * Lead and deliver projects from concept to production across PNI (Personalization and Insights) platform ## Related Videos - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Infrastructure as Code: The Developer's Secret Weapon](https://www.wearedevelopers.com/videos/1221-infrastructure-as-code-the-developer-s-secret-weapon) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [Why and when should we consider Stream Processing frameworks in our solutions](https://www.wearedevelopers.com/videos/1085-why-and-when-should-we-consider-stream-processing-frameworks-in-our-solutions) ## Related Articles - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [7 Cloud Computing Trends Coming in 2025 for Developers](https://www.wearedevelopers.com/magazine/412-7-cloud-computing-trends-coming-in-2025-for-developers)