IT - Technology Architect | Cloud Platform | Amazon Webservices Architecture

Design and develop scalable ETLELT data pipelines on AWS.Build and maintain data lakes and data warehouse solutions.Develop data processing solutions using AWS Glue, Amazon S3, AWS Lambda, Amazon Athena, and Amazon Redshift.Develop and optimize complex SQL queries for analytics and reporting.Use PythonPySpark for data processing, transformation, and automation.Work with large structured and semistructured datasets.Implement data quality checks, monitoring, logging, and errorhandling mechanisms.Optimize data pipelines for performance, scalability, reliability, and cost.Implement appropriate AWS IAM roles and access controls for data solutions.Collaborate with Data Analysts, Data Scientists, BI teams, and application teams to understand data requirements.Support production issues and perform rootcause analysis.Strong handson experience with AWS Data Analytics servicesAWS GlueAmazon S3Amazon RedshiftAmazon AthenaAWS LambdaSQLPython PySparkAmazon EMR Apache SparkAmazon Kinesis or KafkaAmazon

Cloud Computing, Web Services, Amazon Web Services (AWS), Python Programming/Scripting Language, Data Management, Scalable System Development, Data Warehousing, Apache, Data Formats, Data Sets, QuickSightAirflow AWS Managed Workflows for Apache Airflow MWAAExperience with APIs and JSONParquet data formats

Minimum years of experience >10 years

Certifications Needed :No

Top 3 responsibilities you would expect the Subcon to shoulder and execute Design and develop scalable ETLELT data pipelines on AWS. Develop and optimize complex SQL queries for analytics and reporting. Use PythonPySpark for data processing, transformation, and automation.