Data Engineer - APIGEE

Pyramid Technology Solutions

Los Angeles, CA, United States

3 months ago

Role details

Contract type

Temporary contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

5 years minimum

Working hours

Regular working hours

Job source

dice.com

Tech stack

Application Programming Interfaces (APIs) Artificial Intelligence Amazon Web Services Microsoft Azure Big Data Cloud Computing Continuous Integration Information Engineering Data Governance Information Leak Prevention Extract Transform Load (ETL) Data Security

+31 more

Data Warehousing Database Servers Github Python (Programming Language) Online Analytical Processing Online Transaction Processing Performance Tuning Role-Based Access Control Release Management Power BI Standard Sql Scala (Programming Language) SQL Databases Tableau (Software) YAML Azure Data Factory Apache Spark Generative AI Apigee Data Lakes Pyspark Infrastructure Automation Frameworks Data Lineage Machine Learning Operations Terraform Data Pipelines Api Management Powerapps Azure Resource Manager Databricks Programming Languages

Requirements

Cloud Platforms: Deep understanding of Azure ecosystem, including Azure Data Factory, Data Lake Storage, Blob Storage, power apps, and Functions. Additionally, in-depth understanding and implementation of API management such as APIGEE.

Big Data Technologies: Proficiency in Databricks, Spark, PySpark, Scala, and SQL.

Data Engineering Fundamentals: Expertise in ETL/ELT processes, data pipelines, data modeling, schema design, and data warehousing.

Programming Languages: Strong Python and SQL skills, with knowledge of other languages like Scala or R beneficial.

Data Warehousing and Business Intelligence: Strong ERD concepts, designs, and patterns, Understanding of OLAP/OLTP systems, performance tuning, Database Server concepts, and BI tools (Power BI, Tableau).

Data Governance: Strong understanding of RBAC/ABAC, Data Lineage, Data leak prevention, Data security, and compliance. Deep understanding and implementation knowledge of audit and monitoring in Cloud. Infrastructure Deployment: GitHub version control, CI/CD pipelines, release management, Terraform and YAML templates, and script-based deployments.

Additional Experience Required:

Seven (7) years of applying Enterprise Architecture principles, with at least five (5) years in a lead capacity.

Five (5) years of hands-on experience with Azure Data Factory, Azure Databricks, API implementation and management solution, and managing Azure resources., Five (5) years of experience in the following: developing data models and pipelines using Python; working with Lakehouse platforms; GitHub CI/CD pipelines and infrastructure automation, Terraform scripting; and with data warehousing systems, OLAP/OLTP systems, integration of BI tools and designing, developing, and deploying AI/ML and predictive analytics solutions using Databricks, Apache Spark, MLflow, Delta Lake, Python, and cloud platforms such as AWS or Azure. Proven ability to build and operationalize predictive models, Generative AI solutions, and enterprise-scale analytics pipelines to support forecasting, operational intelligence, and data-driven decision-making.