Principal Data Engineer

The Intersect Group
Sandy Springs, GA, United States
3 days ago
Apply on www.careerbuilder.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours

Tech stack

Artificial Intelligence Data Analysis Automation of Tests Microsoft Azure Cloud Engineering Continuous Integration Data Governance Data Integration Data Mart Data Security Software Design Patterns DevOps
+35 more
Distributed Computing Environment E-Business Python (Programming Language) Machine Learning Meta-Data Management Performance Tuning Queueing Systems Release Management Reliability Engineering DataOps Runbook Software Engineering SQL Databases Data Streaming Workflow Management Systems Enterprise Data Management Data Logging Data Processing Generative AI Git Data Layers Event Driven Architecture Data Lakes AI Platforms Pyspark Kubernetes Infrastructure Automation Frameworks Real Time Data Data Management Data Lakehouse Video Streaming Api Design Data Pipelines Docker Databricks

Job description

Design and deliver enterprise data platforms and products that support reporting, analytics, machine learning, AI, and digital business capabilities. Define architectural direction, reusable engineering standards, and scalable data solutions across multiple teams and business domains. Partner with technical and business stakeholders to turn complex problems into reliable, governed data products., * Define and evolve enterprise data engineering architectures, design patterns, standards, and best practices.

  • Create reusable engineering frameworks, templates, automation standards, and playbooks.
  • Design and deliver scalable data platforms and products across lake, warehouse, lakehouse, data mart, and semantic layers.
  • Build and modernize batch, streaming, event-driven, and API-based data integration pipelines.
  • Establish production practices for monitoring, logging, incident response, runbooks, platform support, and continuous improvement.
  • Address systemic gaps in data quality, reliability, observability, performance, cost, resiliency, and operational readiness.
  • Ensure data platforms and products meet standards for security, governance, privacy, data quality, and responsible data use.
  • Provide technical mentorship and lead cross-functional alignment without relying on formal reporting authority.

Requirements

Architectural Services, Machine Learning, Artificial Intelligence (AI), Product Support, Design Patterns Programming Methodologies, Best Practices, Architectural Design, Enterprise Architecture, Automation, Data Mart, * Deep expertise in Python, SQL, PySpark and/or Scala, and distributed data processing frameworks.

  • Strong experience with Azure cloud platforms and Azure Databricks, including Delta Lake and platform-scale data processing.
  • Experience designing and operating data lakehouse, warehouse, data mart, semantic layer, and enterprise analytical data products.
  • Experience with CI/CD, workflow orchestration, Git-based development, automated testing, and production release practices.
  • Experience with Docker, Kubernetes, Infrastructure as Code, cloud-native deployment patterns, and modern DevOps or DataOps practices.
  • Strong understanding of observability, monitoring, logging, performance optimization, reliability engineering, and cost management.
  • Knowledge of data governance, data quality, data security, access controls, metadata management, and compliance-sensitive environments.
  • Preferred experience with streaming technologies, event-driven architectures, message queues, real-time data integration, BI tools, generative AI, or enterprise AI platforms.

Apply if you are ready to define enterprise data engineering direction and deliver platforms that support analytics, AI, and business decision-making. The role offers the opportunity to influence standards, architecture, and engineering practices across multiple teams and domains.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerbuilder.com
Prepare application

Good distractions

Loading talks and stories from around this role…