Senior Data Engineer 4 4P/259

We are seeking an experienced Senior Data Engineer with a strong background in designing and developing robust, scalable, and high-performance data infrastructure. The ideal candidate will have 10+ years of experience in manipulating data, working with both relational and NoSQL systems, and building modern data pipelines across data lakes and warehouses. This role demands a deep technical skillset in SQL, data modeling, and data pipeline orchestration tools like Apache Spark, Hive, and Airflow., * Design, develop, and maintain scalable ETL/ELT pipelines to process and integrate data from disparate sources.

• Work with structured and unstructured data to build clean, efficient, and machine-readable datasets.
• Normalize databases and ensure that data structures align with application and business requirements.
• Collaborate with business analysts, data scientists, and software engineers to deliver data solutions aligned with company objectives.
• Design and implement data models that support reporting, analytics, and machine learning use cases.
• Migrate and optimize workloads in cloud-based environments (e.g., Azure, AWS, GCP).
• Monitor and optimize data pipeline performance, data quality, and reliability.
• Implement data governance best practices and ensure compliance with security and privacy regulations.

Apache Spark, Data Management, Data Modeling, SQL (Structured Query Language), NoSQL, Apache Hive, Scalable System Development, Data Warehousing, Database Extract Transform and Load (ETL), Data Processing, * Expert-level proficiency in SQL, including writing complex queries and optimizing performance.

• Deep understanding of data modeling techniques for both OLTP and OLAP systems.
• Hands-on experience with big data frameworks:
• Apache Spark, Hive, Airflow, Databricks
• Experience with modern cloud data platforms:
• Azure Data Lake, Azure Synapse, AWS Redshift, or Google BigQuery
• Strong experience with data orchestration, scheduling, and automation tools.
• Familiarity with Python, Scala, or Java for data processing and scripting tasks.
• Knowledge of NoSQL databases (e.g., MongoDB, Cassandra) and streaming tools (e.g., Kafka) is a plus.
• Experience with data security, lineage, and cataloging tools (e.g., Unity Catalog, Collibra)., * Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
• Microsoft Azure Data Engineer Associate or equivalent certifications preferred.
• Experience in data governance processes and supporting enterprise analytics platforms.
• Knowledge of containerization (e.g., Docker) and CI/CD pipelines in DevOps environments.