Senior Data Engineer

Job Summary: The Senior Data Engineer will be responsible for designing, developing, and maintaining scalable data pipelines and solutions on Google Cloud Platform (Eligible to workP).

The role focuses on data ingestion, transformation, orchestration, and governance across enterprise data systems.

The engineer will work closely with architects, data analysts, and business stakeholders to implement high-quality, performance-Eligible to workimized data solutions in BigQuery and related Eligible to workP services., * Design and implement end-to-end data pipelines using Cloud Dataflow (Python/Apache Beam) for batch and streaming data.

• Develop, Eligible to workimize, and maintain BigQuery stored procedures (SPs), SQL scripts, and user-defined functions (UDFs) for complex transformations and business logic implementation.
• Build and manage data orchestration workflows using Cloud Composer (Airflow) with appropriate operators and dependencies.
• Establish secure and efficient connections to source systems for data ingestion and integration.
• Manage data ingestion workflows from on-premise and cloud sources into Google Cloud Storage (Eligible to workS) and BigQuery.
• Execute history data migration from legacy data warehouses (preferably Snowflake, Teradata, Netezza, Oracle, SQL Server) to BigQuery, ensuring accuracy and performance Eligible to workimization.
• Design and maintain data validation and testing frameworks for ensuring data quality and reliability across pipelines.
• Implement data governance practices, including metadata management, lineage tracking, and access control.
• Collaborate with data analysts and architects to define scalable and reusable data models, views, and semantic layers in BigQuery.
• Troubleshoot data pipeline failures, perform root cause analysis, and implement preventive measures.
• Eligible to workimize cost and performance of Eligible to workP workloads using best practices for BigQuery and Dataflow. Skills

Data Management, Cloud Computing, Scalable System Development, Data Analysis, Architectural Analysis, Python Programming/Scripting Language, Apache, SQL (Structured Query Language), Stored Procedures, Scripting (Scripting Languages), * Strong hands-on experience in Google Cloud Platform (Eligible to workP) with expertise in BigQuery, Dataflow, Cloud Composer, and Eligible to workS.

• Proficient in Python programming for data engineering (Dataflow pipelines, validation scripts, automation).
• Expertise in BigQuery SQL, Stored Procedures, Views, and User Defined Functions (UDFs).
• Experience with data migration from on-premise or cloud data warehouses (preferably Snowflake, Teradata, Netezza, Oracle, SQL Server) to BigQuery.
• Strong understanding of ETL/ELT frameworks, data modeling, and schema design (Star/Snowflake).
• Familiarity with data governance, metadata, and lineage frameworks on Eligible to workP.
• Knowledge of data validation and testing techniques, including reconciliation, rule-based checks, and automation.
• Hands-on experience in workflow orchestration using Cloud Composer (Airflow) with custom operators.
• Strong SQL tuning and BigQuery performance Eligible to workimization skills.
• Good communication and collaboration skills to work with cross-functional teams in Agile environments.
• Proven ability to troubleshoot, Eligible to workimize, and enhance complex data workflows at scale.

Security Notice: Our official website is www.diverselynx.com We do not operate or authorize any other websites representing Diverse Lynx LLC.