> Markdown version of [/jobs/ext/2572694-aws-data-solution-engineer](https://www.wearedevelopers.com/jobs/ext/2572694-aws-data-solution-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AWS Data & Solution Engineer - **Company:** Acunor Infotech - **Location:** New York, NY, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Amazon Web Services, Amazon S3, Data Analysis, Apache HTTP Server, Unit Testing, Code Review, Codecs, Extract Transform Load (ETL), Data Security, Relational Databases, Software Debugging, File Systems, Amazon DynamoDB, Apache Hadoop, Hadoop Distributed File System, JSON, Python (Programming Language), MongoDB, NoSQL, OAuth, Oracle (Applications), Swagger, Software Deployment, Systems Integration, Web Services, Extensible Markup Language (XML), Enterprise Data Management, Parquet, Datadog, Freeform SQL, Flask (Web Framework), Snowflake, Apache Spark, Boto3, Electronic Medical Records, AWS Lambda, Fastapi, Pandas, Build Management, Data Lakes, Pyspark, Information Technology, Avro, AWS Glue, Data Analytics, AWS Data Analytics, Apache Kafka, Virtual Agents, Cloudwatch, Restful APIs, GPT, Data Pipelines, Docker, Confluent, Amazon Redshift, Oracledb - **Published:** August 28, 2026 - **Apply:** https://www.dice.com/job-detail/c97b6fd2-335d-45b3-94a0-25bd2fc63666 ## About the Role * Bachelor's Degree or equivalent in computer science or related and minimum 10+ years of experience * Certified on one of - Solution Architect, Data Engineer or Data Analytics Specialty by AWS * Require hand-on experience on Python and PySpark programming * Require hands-on experience on AWS S3, Glue ETL & Catalog, Lamba Functions, EventBridge, Step Functions, Athena * Require hands-on experience on Kafka integrations * Require hands-on experience working on different file formats i.e. avro, parquet, orc, json, xml * Require hands-on experience on Python pandas, requests, boto3 module * Require hands-on experience in writing complex SQL queries * Require hands-on experience using REST APIs using FastAPI or Flask * Require hands-on experience building Agentic AI workflows * Preferred expertise on Snowflake, AWS Redshift & DynamoDB * Ability to use AWS services, predict application issues and design proactive resolutions * Require Technical Coordination skills to drive requirements and technical design * Requires aptitude to help build skillset within organization Knowledge, Skills & Abilities * Data pipelines using Python and PySpark on AWS Glue, EMR and lambda functions. * Develop and secure RESTful APIs (FastAPI) on Docker/EKS containers and implement OAuth2/JWT authentication for protected endpoints * Hands-on experience with Apache Iceberg tables for cdc and latest snapshots * Event based pipelines for consuming/publishing to/from Apache Kafka/MSK * Lead and communicate complex technical designs and leverage copilot/GPT for agentic coding of above tech stack ## Description We are looking for an Sr AWS Data & Solutions Engineer with primary skills on Python & PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. You will be responsible for building API services using FastAPI or Flask frameworks. Key Responsibilities * Design, build and unit test applications on Spark framework on Python. * Build Python and PySpark based applications based on data in both Relational databases (e.g. Oracle), NoSQL databases (e.g. DynamoDB, MongoDB) and filesystems (e.g. S3, HDFS) * Build AWS Lambda functions on Python runtime leveraging awswrangler, pandas, json, requests * Build PySpark based data pipeline jobs on AWS Glue ETL or EMR Clusters * Build Python based event-driven integration with Kafka Topics, leveraging Confluent libs * Leveraged Apache Iceberg to manage schema evolution and ACID-compliant CDC merges within the data lake * Design and Build API services using FastAPI, understand the swagger metadata files and implement OAuth2/JWT authentication for protected endpoints * Build the process orchestration pipelines using AWS Step Functions and Eventbridge rules. * Optimize performance for data access requirements by choosing the appropriate native Hadoop file formats (Avro, Parquet, ORC etc) and compression codec respectively. * Deploy applications on Docker and Kubernetes containers * Leverage copilot/GPT for agentic coding of above tech stack * Optimize performance of Spark applications in Hadoop using configurations around Spark Context, Spark-SQL, Data Frame, and Pair RDD's * Setup the Glue crawlers to catalog OracleDB tables, MongoDB collections and S3 objects * Ability to monitor, troubleshoot and debug failures using AWS CloudWatch and Datadog * Ability to solve complex data-driven scenarios and triage towards defects and production issues * Participate in code release and production deployment. * Create documentation for user adoption, deployments, runbook, and support client users for enablement or for any issues encountered. * Perform code reviews with the team and enable them to develop code for complex scenarios * Participate in the agile development process, and document and communicate issues and bugs relative to data standards in scrum meetings * Work collaboratively with onsite and offshore team. * Voice the opinions to multiple teams and thus driving the entire initiative with strong leadership ## Related Videos - [From event streaming to event sourcing 101](https://www.wearedevelopers.com/videos/91-from-event-streaming-to-event-sourcing-101) - [Tips and Tricks for Working with JSON](https://www.wearedevelopers.com/videos/1229-tips-and-tricks-for-working-with-json) - [ Evaluating AI models for code comprehension](https://www.wearedevelopers.com/videos/1462-evaluating-ai-models-for-code-comprehension) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [Introducing JSON Structure](https://www.wearedevelopers.com/videos/100219-introducing-json-structure) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)