Data Engineer

2 MTC Brooklyn/Remote an option with onsite visits for critical meetings as needed Brooklyn,NY,11201

RESPONSIBILITIES

• Be experienced in Data Engineering best practices, technologies, tools and processes.

• Bring sound knowledge of Data Warehouses and LakeHouse concepts and practical implementation experience.

• Build a framework of repeatable solutions and playbooks enabling efficient and predictable data pipelines.
• Have hands-on development experience in the implementation of an agile, cloud centric data warehousing and reporting platform with team members of various experience level.
• Maintain query performance and tuning to ensure cost optimization.
• Participate in joint application development sessions with co-engineers and end users and be willing to brainstorm.
• Complete technical documentation and be willing to transfer knowledge as needed.

• 12+ years developing Data Pipelines / Flows using ETL/ELT tools and technologies.
• 10+ years of strong SQL fluency (query optimization, windowing functions, aggregation, etc.).
• 5+ years building complex Analytics and Reporting solutions.

• 3+ years’ experience with a cloud data lake/warehouse solution (Snowflake, Redshift, Google Cloud Platform etc.).

• Hands on experience working with data integration tools like Informatica Intelligent Cloud Services, Informatica Power Center or SSIS or a similar tool.
• Extensive experience developing production grade, large scale data solutions.
• Experience performing conceptual, logical, and physical data modeling using data modeling tools in complex, large-scale environments.
• Experience working with Microsoft Azure cloud computing platform and services.

• Experience managing data orchestration at scale using tools such as Airflow, Prefect and Dagster.

• Experience with traditional RDMS platforms (Oracle and SQL Server).
• Experience working with version control systems (e.g., Git)
• Good understanding of CI/CD principles.
• Experience developing dashboards and reports in applications such as Oracle Analytics Server (OAS), Microsoft Power BI and Google Looker

DESIRABLE SKILLS/EXPERIENCE:

• Experience using Azure services for Security, Blob Storage, Data Lake, Databricks, Data Factory etc.
• Programming experience with Python or Java
• Experience with Azure Monitoring services
• Microsoft Certified Azure Solutions Architect Expert or a Snowpro Certification or a similar one