Senior Data Engineer Consultant (AI-Forward) - Pharmacy Data, Databricks + Kafka

We need a Senior Data Engineer Consultant (AI-Forward) to build and run the pipelines behind a modern pharmacy platform. You’ll bring together data from pharmacy systems, claims and e-prescribing sources and make it reliable for operations and reporting. You know pharmacy data well enough to spot when something looks wrong. This is a corp-to-corp, 1099 engagement for an independent consultant who works autonomously and uses AI tools daily to multiply their impact., * Daily practitioner with AI coding tools and agents: Claude Code, OpenAI Codex CLI, GitHub Copilot or Cursor.

• Prompt and instruction design: turn data requirements into structured prompts, tools and context.
• Agentic workflows for data operations: pipeline triage, schema-change diffing, test generation and code review with a human in the loop.
• Safety and compliance: PHI redaction, synthetic test data, secrets in Key Vault, nothing sensitive in prompts or logs.
• Consulting & High EQ
• Excellent client-facing communication; comfortable with engineers, analysts and pharmacy operations staff alike.
• Clear trade-off framing and requirements discovery.
• Entrepreneurial Mindset
• Think like an owner; deliver measurable value. We welcome consultants building their own ventures.
• Business Structure
• Must have an active LLC or S-Corp; 1099 only (Corp-to-Corp). 2+ years running your own consulting company strongly preferred.

What You’ll Do

• Build Pipelines: Streaming and batch pipelines on Azure Databricks with PySpark, Delta Lake and declarative pipelines.
• Ingest Change Data: Bring operational database changes in through CDC and Kafka.
• Model Pharmacy Data: Turn prescription, fill, claim, drug and prescriber data into clean, well-tested tables.
• Publish Curated Data: Publish curated datasets for downstream applications and reporting.
• Ship Safely: Automated deployments through CI/CD, with monitoring, alerting and data quality checks.
• Use AI Well: Apply AI tools to speed up delivery, documentation and review without letting PHI leave controlled boundaries.

Preferred Tech & Tooling

• Azure Databricks, PySpark, Delta Lake, Lakeflow / Delta Live Tables, Unity Catalog
• Kafka, CDC (e.g. Debezium), SQL Server
• Python (pyproject, wheels, pytest); CI/CD (GitLab, GitHub or Azure DevOps)
• Agentic IDEs: Claude Code, OpenAI Codex CLI, Copilot/Cursor

Nice to Have

• Pharmacy management system data, or e-prescribing data (NCPDP SCRIPT, medication history).
• Databricks certification.
• Former pharmacy technician or pharmacy-system analyst.

Why Join Proactive Logic Consulting?

• Flexible Engagements: Shape your schedule around your life.
• Top-Tier Peers: Collaborate with senior experts across cloud, data and AI.
• Impactful Projects: Healthcare and pharmacy work with measurable outcomes.
• AI-Forward Culture: Agentic software delivery and practical wins over hype.
• Entrepreneurial Community: Many of our consultants build their own products too., * Your resume or consultant profile highlighting Databricks, Kafka and pharmacy data work.
• A short note on your LLC or S-Corp (name, state).
• Links to relevant work (pipelines, write-ups, demos or repos).
• 3-5 lines on how you use AI tools (Claude Code, Codex CLI, etc.) to accelerate delivery while keeping PHI safe.

Optional (helps us fast-track you): answer in 1-2 sentences each.

• Walk us through a pharmacy claim from submission to paid or rejected.
• A drug’s NDC changes between reference-data updates. What can break in your pipeline, and how do you guard against it?

• 10+ Years in Data Engineering
• Building and operating production streaming and batch pipelines and lakehouse architectures.
• Strong data modeling and testing; you leave behind clean, well-tested, documented tables.
• Databricks & Spark (2+ years)
• Azure Databricks, PySpark, Delta Lake, declarative pipelines (Delta Live Tables / Lakeflow), Structured Streaming.
• Databricks CI/CD and deployment automation; job orchestration, monitoring and alerting.
• Kafka & Change Data Capture
• Production Kafka experience; CDC ingestion from operational databases.
• Idempotent, replayable pipelines with clear failure handling.
• SQL & Python
• Strong SQL, including SQL Server.
• Python packaging and testing (pyproject, wheels, pytest).
• Pharmacy Data (Required, 2+ years)
• NCPDP claims, NDCs and drug reference data, prescriptions and fills, prescriber identifiers (NPI, DEA).
• Good PHI habits: minimum-necessary access, no PHI in logs or test data, HIPAA-aligned practices.

Proactive Logic Consulting INC is a boutique technology consulting firm. We build the systems and capacity behind our clients’ value creation plans, working with private-equity-backed companies and operators in healthcare, insurance and financial services