> Markdown version of [/jobs/ext/200474-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/200474-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** CRITICAL PROPULSION LLC - **Location:** Dallas, TX, United States (Remote available) - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Microsoft Azure, Big Data, Cloud Computing, Information Engineering, Data Infrastructure, Cursor (Graphical User Interface Elements), Software Debugging, Fault Tolerance, Python (Programming Language), Modular Design, Operational Databases, SQL Databases, Azure Data Factory, Pyspark, Virtual Agents, Api Design, Restful APIs, Stream Processing, Databricks - **Published:** May 14, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=741fa245fefc3b95 ## About the Role Do you have experience in Cloud infrastructure?, * Years of experience as a number. If you can build production data pipelines inside a 5-day pulse cycle, the number on your resume is irrelevant. * Specific certifications as prerequisites. We care about engineering judgment and hands-on Azure experience, not badges. * Pedigree. No school or company name substitutes for demonstrated ability to ship data infrastructure that works under real load. Nice to have * Deep hands-on experience with Azure Data Factory and Azure Databricks in production environments. * Strong PySpark skills for large-scale data processing and transformation. * Experience building or consuming RESTful APIs, especially Python-based API architecture. * Familiarity with near-real-time data processing patterns and streaming ingestion. * Hands-on experience with agentic AI tooling: Claude Code, Cursor, Copilot Workspace, or similar. You've used AI agents to accelerate pipeline development, not just code completion. * Comfort with spec-driven development. You've built pipelines from detailed specs in a single pulse cycle without ambiguity or rework loops. * Background in consulting or client-facing delivery. You've built data infrastructure with a client watching, not just an internal stakeholder. * Prior experience on small, autonomous teams. You've operated in a model closer to a 3-4 person crew than a 15-person feature factory. ## Description Databricks pipelines that run in production, not notebooks that run on your laptop. You build, test, and maintain the ingestion infrastructure that moves data from source to consumption. You write resilient, fault-tolerant code that handles real-world data at scale inside 5-day pulse cycles. How we work Critical Propulsion amplifies human delivery. Our swarm model pairs 3-4 senior operators with AI agents to cut cycle times 30-50% and multiply effective capacity 2-5x versus traditional teams. Zero handoffs. Single timezone. No offshore blended teams. No sprint theater. Week 1 is productive delivery, not onboarding. We define outcomes and trust operators to get there. What we expect * You build and maintain data pipelines on Azure. ADF, Databricks, PySpark, SQL. You've done this in production, not just in tutorials. * You write Python at a professional level. Debugging, optimization, modular design. You write code that other operators can read, maintain, and extend in the next pulse cycle. * You embrace agentic development. You don't treat AI as autocomplete. You delegate real work to AI agents, review their output critically, and iterate fast. You see this as the future of how data engineering gets done, not a novelty. * You design for stability, traceability, and fault tolerance. When a pipeline breaks at 3am, your code makes it obvious what went wrong and how to fix it. * You operate as an independent contributor. You don't need someone assigning tickets or walking you through requirements. You see the outcome, pick up the work, and close it. * You communicate directly. Blocked? Say so in hours, not days. Found a better approach? Speak up with a proposal, not a meeting request. * You operate autonomously. No status meetings that could be a message. No decks that could be a decision. ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [Shipping Faster with Less: Render on Cloud Hosting, AI Workloads, and the Future of DevOps](https://www.wearedevelopers.com/videos/1894-shipping-faster-with-less-render-on-cloud-hosting-ai-workloads-and-the-future-of-devops) - [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [OLTP in the Lakehouse: Redefining Data for AI Workloads](https://www.wearedevelopers.com/videos/2038-oltp-in-the-lakehouse-redefining-data-for-ai-workloads) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline) - [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering)