> Markdown version of [/jobs/ext/1921870-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/1921870-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** Brooksource - **Location:** Kansas City, MO, United States - **Experience:** Expert - **Salary:** $114,400.0 - $124,800.0 - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Microsoft Azure, Big Data, BigQuery, Cloud Computing, Computer Programming, Continuous Integration, Data Validation, Data Governance, Extract Transform Load (ETL), Data Transformation, Data Warehousing, Web Development, Data Flow Control, Python (Programming Language), PostgreSQL, Machine Learning, Metadata, Meta-Data Management, Node.Js, OAuth, Performance Tuning, Openid Connect, Cloudera, SQL Databases, TypeScript, Web Applications, Openapi, Data Processing, Enterprise Software Applications, Data Ingestion, Tailwind, ReactJS, Infrastructure as Code (IaC), Rate Limiting, Git, Webpack, Pyspark, Infrastructure Automation Frameworks, Information Technology, Data Lineage, Deployment Automation, Apache Kafka, Front End Software Development, Restful APIs, Pagination, Stream Processing, Webhooks, Software Version Control, Data Pipelines, Confluent, Databricks - **Published:** August 4, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=ab52a5c0f87661c0 ## About the Role * Bachelor Degree in Computer Science or related field and 8 years related professional experience Required or Required * Associate Degree and 10 years related professional experience Required or Required * High School Diploma/GED and 12 years related professional experience Required Required Skills * Strong programming skills in Python and PySpark for large-scale data processing. * Proficiency in SQL for data manipulation, analysis, and performance tuning. * Experience with Dataform, Dataproc, and BigQuery for data pipeline development and orchestration. * Understanding of Data Governance principles and implementation practices. * Experience using Git for version control and CI/CD pipelines for automated deployments. * Experience building and consuming RESTful APIs, including enterprise integrations (OAuth2, OpenID Connect, webhooks, pagination, rate limiting, retry logic, error handling). * Some hands-on web application development experience (full-stack or front-end).Nice to Have : * Experience configuring and building pipelines within Databricks * Working knowledge of Infrastructure as Code (IaC) for cloud resource management and automation * Familiarity with DBT, Machine Learning, and AI concepts is a good advantage * Hands-on experience with Kafka and Confluent for real-time data streaming * Knowledge of Cloud Scheduler and Dataflow for automation and workflow management * Full-stack application development experience using Node.js, TypeScript, React, and Express * Experience designing contract-first APIs (OpenAPI, Orval, Zod) * Proficiency with React, Vite, Tailwind CSS, shadcn/ui, and TanStack Query * Experience with PostgreSQL and ORM frameworks (Drizzle ORM preferred) * Experience integrating engineering, construction, ERP, or project management systems * Experience working with metadata-driven applications, configurable mappings, and schema transformation frameworks * Candidates must be legally authorized to work permanently (i.e., without time limitations, without restrictions or without need for work sponsorship) in the country where this position is located. ## Description We are looking for a data engineer with 8+ years of experience responsible for designing and building reliable data pipelines and transforming raw data into clean, consumable datasets. The role focuses on developing and maintaining ETL and ELT workflows, data modeling, and ensuring data quality and lineage for downstream users. You will collaborate with product and analytics teams to translate business requirements into scalable data products. This position is hands-on and involves coding, optimizing queries, troubleshooting pipelines, and implementing best practices for data governance and reliability. Role & Responsibilities: * Design, develop, and maintain scalable and reliable data pipelines and platforms to support enterprise analytics and reporting. * Collaborate with business and technical teams to understand data requirements and translate them into efficient data engineering solutions. * Build data ingestion frameworks using Kafka Confluent and integrate data from multiple cloud and on-premises sources. * Implement ELT and ETL pipelines using PySpark, SQL, Dataform, DBT, and Dataproc for efficient data transformation and processing. * Design and manage data models, schemas, and data warehousing structures optimized for performance and scalability. * Ensure data quality, consistency, and integrity through effective data validation and monitoring mechanisms. * Implement data governance practices including data lineage, metadata management, privacy, and compliance. * Develop automated monitoring and alerting mechanisms to ensure reliability, availability, and performance of data pipelines. * Collaborate with cross-functional teams to enable downstream reporting, analytics, and AI/ML model consumption. * Follow best practices for version control, documentation, and CI/CD workflows using Git and infrastructure-as-code tools. * Provide operational support for data pipelines and proactively troubleshoot performance issues. * Continuously optimize data processing, query execution, and resource utilization in Azure environments. * Build and maintain RESTful API integrations to support data ingestion from and synchronization with web applications and enterprise systems. * Contribute to lightweight web application development efforts where data pipelines interface directly with front-end or full-stack components. ## Related Videos - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Keeping applications secure by evolving OAuth 2.0 and OpenID Connect](https://www.wearedevelopers.com/videos/100152-keeping-applications-secure-by-evolving-oauth-2-0-and-openid-connect) - [Stop using Node.js like in 2020! What changed and what you can do today with Node.js](https://www.wearedevelopers.com/videos/100011-stop-using-node-js-like-in-2020-what-changed-and-what-you-can-do-today-with-node-js) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [Stop Using Node.js Like It’s 2020! - Alfonso Graziano](https://www.wearedevelopers.com/videos/1863-stop-using-node-js-like-it-s-2020-alfonso-graziano) - [Delay the AI Overlords: How OAuth and OpenFGA Can Keep Your AI Agents from Going Rogue](https://www.wearedevelopers.com/videos/1637-delay-the-ai-overlords-how-oauth-and-openfga-can-keep-your-ai-agents-from-going-rogue) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries) - [Is Software Engineering Over-Saturated?](https://www.wearedevelopers.com/magazine/418-is-software-engineering-over-saturated)