Technical Lead/Architect (Temporal & Apache Kafka required)

Infinitive
McLean, VA, United States
1 day ago
Apply on www.careerbuilder.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
2 years minimum
Compensation
$90,000
Working hours
Regular working hours

Tech stack

Java (Programming Language) Artificial Intelligence Apache HTTP Server Architectural Patterns BigQuery Cloud Computing Continuous Integration Information Engineering Data Governance Data Vault Modeling Data Warehousing Software Design Patterns
+25 more
Disaster Recovery Distributed Systems Fault Tolerance Protocol Buffers JSON Python (Programming Language) Management of Software Versions Enterprise Data Management Great Expectations (Foster Youth College-readiness and Support Program) Snowflake Concurrency Apache Spark State Machines Backend Pyspark Apache Flink Avro AWS Glue Apache Kafka Data Management Stream Processing Data Pipelines Confluent Databricks Microservices

Job description

We are seeking a Staff / Principal Data Engineer & Platform Architect to define the technical vision, architectural strategy, and technical execution for our enterprise-wide, next-generation event-driven data platforms. In this role, you will act as the principal authority bridging ultra-high-throughput streaming topologies, stateful distributed workflow orchestration, and federated data governance.

You will spearhead our platform architecture centered around Apache Kafka for real-time event distribution and Temporal for durable execution and distributed state orchestration. A core mandate of this position is establishing enterprise data contracts, driving schema evolution frameworks, and designing resilient Lakehouse/warehouse architectures capable of serving high-concurrency analytical and operational workloads. As a technical leader, you will set the engineering standards, mentor senior talent, and align multi-team distributed systems initiatives with executive business priorities., * Architecture & Strategy: Define the multi-year technical roadmap and reference architecture for our real-time streaming infrastructure, Lakehouse ingestion engines, and distributed orchestration layers.

  • Distributed Stream Topologies: Architect and govern multi-tenant, enterprise Apache Kafka ecosystems (producers, consumers, Kafka Connect, Schema Registry), ensuring sub-second latencies, exactly-once processing guarantees, and disaster recovery.
  • Data Contracts & Governance: Formulate and enforce company-wide data governance, schema design patterns, and automated contract testing (Avro, Protobuf, JSON Schema) across polyglot microservices, streaming pipelines, and storage boundaries.
  • Durable Orchestration & Transactional Integrity: Architect long-running, fault-tolerant workflow topologies using Temporal, standardizing complex distributed transaction patterns (e.g., Saga patterns, compensation logic, distributed state machines, human-in-the-loop flows).
  • Lakehouse & Analytical Engine Architecture: Direct the design and operational scaling of large-scale distributed computing workloads (Apache Spark / PySpark) and modern cloud Lakehouse platforms (Snowflake, Databricks, BigQuery) using medallion architectures and dimensional/semantic models.
  • Operational Excellence & FinOps: Implement comprehensive observability platforms (tracing, telemetry, automated drift detection, data quality gates) while establishing cloud infrastructure cost optimization and capacity planning frameworks.
  • Technical Leadership & Mentorship: Act as the technical authority across platform and product teams; establish engineering excellence standards, drive RFC and Architecture Review Board (ARB) processes, and elevate senior engineering talent across the organization.

Requirements

Technical Leadership, Apache Kafka, Return on Investment (ROI), Customer Support/Service, Artificial Intelligence (AI), Consulting, Topology, High Throughput, Architectural Services, Technical Strategy, * 8+ years of professional experience in data engineering, distributed systems, or backend platform architecture, with 2+ years acting in a Lead, Staff, or Principal/Architect capacity.

  • Durable Workflow & State Machine Architecture: Deep architectural experience with Temporal (or Cadence), including deterministic execution modeling, activity orchestration, workflow versioning strategies, signal/query handling, dynamic task routing, and cross-boundary error recovery.
  • Mastery of Streaming Systems: Deep foundational expertise with Apache Kafka internals (broker sizing, partition strategies, consensus mechanisms, custom connector development, consumer group rebalancing, and stateful stream processing via Kafka Streams or Flink).
  • Enterprise Data Contracts & Schema Engineering:
  • Authority in schema specification frameworks (Apache Avro, Protocol Buffers/gRPC, JSON Schema).
  • Hands-on leadership implementing schema registries (Confluent, AWS Glue) with automated CI/CD schema linting, breaking-change prevention, and backward/forward compatibility strategies.
  • Experience standardizing automated data contract verification tooling (e.g., Great Expectations, Pandera, Pydantic, dbt tests, contract-testing frameworks).
  • Distributed Computing & Core Engineering: Advanced engineering capabilities in Python, Go, or Java/Scala, emphasizing distributed design patterns, concurrency models, and resilient microservice design. Extensive scale experience with Apache Spark / PySpark internals and query plan optimization.
  • Modern Lakehouse & Data Modeling: Expert-level knowledge of dimensional data modeling, Data Vault, or semantic layer design, with practical experience scaling multi-terabyte/petabyte systems on Snowflake, Databricks, or BigQuery.
  • Cross-Functional Influence: Demonstrated track record of aligning distributed engineering teams around unified architectural patterns, authoring architectural decision records (ADRs), and influencing executive stakeholders.

Benefits & conditions

Infinitive is required by law in some jurisdictions to include a reasonable estimate of the compensation range for this role. The determination of this range includes various factors not limited to skill set, level, experience, relevant training, and licensure and certifications. Compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range for this role in the U.S. is $90,000 - $154,00.00.Infinitive is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, protected veteran status, or any other characteristic protected by applicable federal, state, or local law.

About the company

About InfinitiveInfinitive is a data and AI consultancy that helps clients modernize, monetize, and operationalize their data to generate lasting value. They pride themselves on their deep industry and technology expertise, ensuring that they drive and sustain the adoption of new capabilities. Infinitive is committed to aligning their team with their clients’ culture, ensuring a successful partnership by bringing the right mix of talent and skills for high return on investment.

Infinitive has earned recognition as one of the β€œBest Small Firms to Work For” by Consulting Magazine, receiving this accolade nine times, most recently in 2026. They have also been honored as a β€œTop Workplace” by the Washington Post, β€œBest Places to Work” by the Washington Business Journal, and β€œBest Places to Work” by Virginia Business.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerbuilder.com
Prepare application

Good distractions

Loading talks and stories from around this role…