Data Architecture and Modeling

Diverse Lynx LLC
Marlborough, MA, United States
23 days ago
Apply on www.careerbuilder.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
10 years minimum
Compensation
$124,800 - $145,600
Working hours
Regular working hours

Tech stack

Unity 3d Artificial Intelligence Amazon Web Services Data Analysis Business Logic Cloud Computing Continuous Integration Data Architecture Data Governance Data Masking Data Transformation Software Debugging
+12 more
Github Apache Hive SQL Databases Apache Spark Data Lakes Pyspark Data Lineage Data Management Terraform Data Pipelines Cisco Databricks

Job description

  • Design, code, and deploy batch and streaming data pipelines using PySpark, Spark SQL, and Lakeflow.
  • Ingest data from Point-of-Sale (POS), e-commerce platforms, loyalty systems, and marketing clouds.
  • Implement data transformations and business logic within the Medallion Architecture (Bronze, Silver, Gold layers).
  • Build and Eligible to workimize Gold dimension tables as the single source of truth.
  • Define governed business KPIs using Unity Catalog Metric Views.
  • Implement data quality frameworks and cleansing routines to ensure accurate Customer 360 data.
  • Monitor, debug, and tune Databricks jobs and Spark clusters for performance and cost-efficiency.
  • Implement best practices for partitioning, caching, and data layout in Delta Lake.
  • Manage Databricks environments, clusters, and job deployments using Terraform and AWS DevOps/GitHub Actions.
  • Champion and implement CI/CD best practices for data pipelines.
  • Implement data governance features within Databricks Unity Catalog, including data lineage tracking, fine-grained access controls, ABAC policies, tags, and data masking.
  • Partner with Functional Consultants, Data Scientists, and Analytics Engineers to deliver consumption-ready datasets.
  • Build and govern Unity Catalog Business Semantics, including Metric Views, Domains, Pages, Glossary Terms, and trusted assets.
  • Configure and curate AI/BI Genie Spaces and the Genie Ontology to enable conversational analytics for business users.
  • Enable consumption through AI/BI Dashboards and Databricks One.

Required Skills

  • PySpark
  • Spark SQL
  • Lakeflow
  • Lakeflow Connect
  • Lakeflow Declarative Pipelines
  • Terraform
  • AWS DevOps
  • GitHub Actions

Requirements

Data Modeling, Willing to Travel, Data Management, SQL (Structured Query Language), eCommerce, Cloud Computing, Marketing, Sales, Cisco Unity, Performance Metrics

About the company

Security Notice: Our official website is www.diverselynx.com We do not operate or authorize any other websites representing Diverse Lynx LLC.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerbuilder.com
Prepare application

Good distractions

Loading talks and stories from around this role…