Data Integration Engineer

Do you have experience in Text mining?, * Snowflake Data Engineering -

o Design and implement enterprise-grade data pipelines using Snowflake, including ingestion and transformation o Must be strong in both Core and Semantic aspects o Develop complex SQL transformations, stored procedures and Dynamic tables inside Snowflake to enable near real-time and batch processing o Implement Snowflake data sharing, data marketplace integrations o Engineer Snowpipe and Kafka-to-Snowflake streaming ingestion pipelines also handling high throughput event data at scale o Optimize Snowflake cluster performance - virtual warehouse sizing, query profiling, clustering keys o Architecture, design aspects, performance tuning, time travel, warehouse concepts - scaling, clustering, micro-partitioning o Experience with SnowSQL, Snowpipe

• Data Integration aspects -

o Design and maintain end-to-end ETL/ELT pipelines using Apache Airflow o Experience in building reusable parameterized data ingestion pipelines/frameworks is beneficial. o Thorough on data quality checks

• AI and Data Science -

o Integrate AI/LLMs with data pipelines via Python UDFs or API callouts - enabling text analytics, semantic search and GEN-AI augmented workflows o Experience with Python based frameworks - scikit learn, PyTorch, TensorFlow o Experience with NLP and text-mining techniques on unstructured data to identify actionable information o Time-series forecasting, anomaly detection and propensity modeling

• Experience with Data Visualization aspects

• Hands-on experience with writing Complex queries using - Joins, Self Joins, Views, Materialized Views, Cursor also Recursive, use of GROUP BY, PARTITION BY functions / SQL Performance tuning

• Hands-on experience with ETL and Dimensional Data Modelling - Slowly Changing Dimensions (SCD - Type 1, 2, 3)

o Good understanding of concepts like schema types, table types - fact-dimension etc. like how to design a dimension vs fact, design considerations factored etc.

• Proficiency in Python scripting/programming - using Pandas, PyParsing, Airflow.

o Pandas, Tableau server modules, Numpy, Datetime, Apache Airflow related modules, APIs o Data Pipeline automation o Strong Python programming skills

• Actively participating in discussions with business to understand requirements, perform thorough impact analysis and provide suitable solutions.