> Markdown version of [/jobs/ext/456764-senior-it-big-data-developer](https://www.wearedevelopers.com/jobs/ext/456764-senior-it-big-data-developer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior IT Big Data Developer - **Company:** Diverse Agile Solutions - **Location:** Washington, DC, United States - **Experience:** Expert - **Salary:** $145,600.0 - $170,560.0 - **Contract:** Permanent contract - **Skills:** Java (Programming Language), JavaScript (Programming Language), Airflow, Amazon Web Services, Microsoft Azure, Big Data, Cloud Computing, Computer Programming, Databases, Continuous Integration, Data Architecture, Information Engineering, Data Governance, Data Infrastructure, Data Integration, Extract Transform Load (ETL), Data Migration, Data Systems, Data Warehousing, Relational Databases, Database Design, Linux, DevOps, Digital Architecture, Distributed Systems, Enterprise Architecture Framework, Perl (Programming Language), Github, Graph Database, Python (Programming Language), PostgreSQL, Machine Learning, Microsoft SQL Server, MySQL, NoSQL, Performance Tuning, DataOps, Unstructured Data, Workflow Management Systems, Enterprise Data Management, Data Processing, Data Storage Technologies, Cloud Platform System, Feature Engineering, Snowflake, Change Data Capture, Gitlab, Data Lakes, Storage Technologies, Information Technology, Deployment Automation, Integration Frameworks, Data Management, Physical Data Models, Software Version Control, Data Pipelines, Programming Languages - **Published:** June 1, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=37fb3c3c32f07e3a ## About the Role Do you have experience in Version control?, Do you have a Master's degree?, * Bachelor's degree in Computer Science, Information Technology, Engineering, Data Science, or a related technical discipline. * Advanced degree (Master's or Ph.D.) preferred. Experience * Minimum of 7 years of experience in data engineering, big data development, data architecture, or related technical roles. * Experience designing and implementing enterprise data platforms supporting large-scale analytical workloads. * Proven experience developing and optimizing enterprise data pipelines and integration frameworks. * Experience supporting complex research, financial, or analytical environments. Required Technical Skills Data Engineering & Database Technologies * Advanced SQL expertise. * Strong experience with relational database platforms: + PostgreSQL + Microsoft SQL Server + MySQL * Database design, administration, and performance optimization. * Data modeling and enterprise information architecture. Programming & Development * Advanced proficiency with: + Python + R * Experience with additional programming languages such as: + Java + Scala + JavaScript + Perl Big Data & Data Processing * Experience working with: + Distributed computing environments + Scalable data processing frameworks + Large-scale data storage architectures + High-volume data workloads, * Experience with: + Amazon Web Services (AWS) + Microsoft Azure + Snowflake * Experience migrating data pipelines between cloud and on-premises environments. DevOps & DataOps * CI/CD pipeline implementation and automation. * DataOps platform development and support. * GitLab and GitHub source control management. * Linux-based development environments. Preferred Qualifications * Experience working with economic, financial, or research-related data environments. * Understanding of time-series data structures and forecasting methodologies. * Experience with NoSQL databases and graph database technologies. * Experience developing, training, deploying, and maintaining machine learning models. * Experience implementing enterprise data warehouses utilizing CDC methodologies. * Knowledge of enterprise architecture frameworks and modern data governance practices. * Experience supporting highly regulated or federal government environments. Desired Competencies * Strong analytical and critical thinking abilities. * Exceptional problem-solving and troubleshooting skills. * Excellent verbal and written communication skills. * Ability to communicate technical concepts to both technical and non-technical stakeholders. * Strong customer-service mindset and collaborative approach. * Self-motivated and capable of managing multiple priorities simultaneously. * Ability to work independently while contributing effectively within cross-functional teams. ## Description The Senior IT Big Data Developer will serve as a key contributor in designing, developing, and optimizing enterprise-scale data architectures and data engineering solutions. This individual will be responsible for building and maintaining scalable data pipelines, modernizing data platforms, integrating diverse data sources, and supporting advanced analytics initiatives. The ideal candidate is a hands-on data engineering professional with deep expertise in database design, large-scale data processing, cloud technologies, ETL/ELT development, and enterprise data architecture. The successful candidate will collaborate with economists, researchers, data scientists, and technical teams to deliver high-performance data solutions that support mission-critical research and policy decisions., * Design, develop, and maintain enterprise-scale data architectures, data warehouses, and data lake solutions. * Build and optimize scalable ETL/ELT pipelines for ingesting, processing, and transforming structured and unstructured data. * Develop and maintain high-performance data integration workflows supporting economic research and analytics initiatives. * Design conceptual, logical, and physical data models aligned with enterprise architecture standards. * Optimize data storage, processing, and delivery frameworks to support large-scale analytical workloads. * Implement workflow orchestration and automation solutions using tools such as Apache Airflow, Prefect, Dagster, or AWS Step Functions. * Develop solutions that support data migration between on-premises and cloud environments. * Collaborate with researchers, economists, and business stakeholders to understand data requirements and deliver scalable solutions. * Perform root cause analysis and troubleshoot complex data quality, integration, and performance issues. * Design and implement Change Data Capture (CDC) processes to support enterprise data warehouse solutions. * Develop, test, deploy, and maintain data applications and services using industry best practices. * Implement DataOps and CI/CD processes to improve deployment automation and operational efficiency. * Support machine learning and advanced analytics initiatives through the development of reliable data pipelines and feature engineering frameworks. * Ensure data governance, security, reliability, and scalability standards are met across enterprise platforms. * Document architecture designs, technical specifications, and operational procedures., ETL/ELT & Pipeline Development * Design and automation of ETL/ELT workflows. * Data integration and transformation solutions. * Workflow orchestration tools including: + Apache Airflow + Prefect + Dagster + AWS Step Functions Cloud & Modern Data Platforms ## Related Videos - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [Docker network without Docker](https://www.wearedevelopers.com/videos/1418-docker-network-without-docker) - [MySQL Protocol Features You Should Be Aware Of](https://www.wearedevelopers.com/videos/100267-mysql-protocol-features-you-should-be-aware-of) - [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Coding for Good: Achieving social change with an app](https://www.wearedevelopers.com/videos/1645-coding-for-good-achieving-social-change-with-an-app) ## Related Articles - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)