Data Engineer

Developer experience specifically focusing on Data Engineering.

-Hands-on experience in Development using Python and Pyspark as an ETL tool.

-Experience in AWS services like Glue, Lambda, MSK (Kafka), S3, Step functions, RDS, EKS etc.

-Experience in Databases like Postgres, SQL Server, Oracle, Sybase etc.

-Experience with SQL database programming, SQL performance tuning, relational model analysis, queries, stored procedures, views, functions, and triggers.

-Strong technical experience in Design (Mapping specifications, HLD, LLD), Development (Coding, Unit testing).

-Knowledge in developing UNIX scripts, Oracle SQL/PL-SQL.

-Experience with data models, data mining, data analysis and data profiling.

-Experience in working with REST API’s.

-Experience in workload automation tools like Control-M, Autosys etc.

-Good knowledge in CI/CD DevOps process and tools like Bitbucket, GitHub, Jenkins.

-Strong experience with Agile/SCRUM methodology.

-Experience with other ETL tools (DataStage, Informatica, Pentaho, etc.).

-Knowledge in MDM, Data warehouse and Data Analytics.

qualifications:

• Core Stack: Hands-on development using Python and PySpark for scalable ETL/ELT pipelines.

• AWS Data Services: Production experience with AWS Glue, Lambda, MSK (Kafka), S3, Step Functions, and RDS.

• Databases & Advanced SQL: Deep experience with relational databases (PostgreSQL, SQL Server, Oracle) and advanced SQL programming (performance tuning, stored procedures, complex queries, data modeling).

• System Design & Linux: Solid background in High-Level/Low-Level Design (HLD/LLD), data mapping specs, and shell/UNIX scripting.

skills:

AWS tools,batch processing,large data,data mining,data extraction,data manipulation,Pyspark,Python,design and development,business value