Data Engineer - Databricks / Customer 360

• Design, develop, and deploy batch and streaming data pipelines using PySpark, Spark SQL, and Lakeflow to ingest data from POS, e-commerce platforms, loyalty systems, and marketing clouds.
• Implement data transformations and business logic within the Medallion Architecture (Bronze, Silver, and Gold layers).
• Build and Eligible to workimize Gold dimension tables as the single source of truth.
• Define governed business KPIs using Unity Catalog Metric Views.
• Implement data quality frameworks and cleansing routines to ensure accurate and trustworthy Customer 360 data.
• Monitor, debug, and Eligible to workimize Databricks jobs and Spark clusters for performance and cost efficiency.
• Implement partitioning, caching, and Delta Lake data layout best practices.
• Manage Databricks environments, clusters, and deployments using Terraform and AWS DevOps/GitHub Actions.
• Implement CI/CD best practices for data pipelines.
• Implement Unity Catalog governance capabilities, including data lineage, access controls, tags, and data masking.
• Collaborate with Functional Consultants, Data Scientists, and Analytics Engineers to deliver consumption-ready datasets.
• Build and govern Unity Catalog Business Semantics, including Metric Views, Domains, Pages, Glossary terms, and certified assets.
• Configure and curate AI/BI Genie Spaces and the Genie Ontology to enable conversational analytics and AI-driven data consumption.

SKILLS_REQUIRED

• PySpark
• Spark SQL
• Lakeflow
• Lakeflow Connect
• Lakeflow Declarative Pipelines
• Terraform
• AWS DevOps
• GitHub Actions

Data Management, Marketing, Cloud Computing, eCommerce, SQL (Structured Query Language), Cisco Unity, Performance Metrics, Metrics, Data Quality, Debugging Skills

Security Notice: Our official website is www.diverselynx.com We do not operate or authorize any other websites representing Diverse Lynx LLC.