Senior IT Big Data Developer

The Senior IT Big Data Developer will serve as a key contributor in designing, developing, and optimizing enterprise-scale data architectures and data engineering solutions. This individual will be responsible for building and maintaining scalable data pipelines, modernizing data platforms, integrating diverse data sources, and supporting advanced analytics initiatives.

The ideal candidate is a hands-on data engineering professional with deep expertise in database design, large-scale data processing, cloud technologies, ETL/ELT development, and enterprise data architecture. The successful candidate will collaborate with economists, researchers, data scientists, and technical teams to deliver high-performance data solutions that support mission-critical research and policy decisions., * Design, develop, and maintain enterprise-scale data architectures, data warehouses, and data lake solutions.

• Build and optimize scalable ETL/ELT pipelines for ingesting, processing, and transforming structured and unstructured data.
• Develop and maintain high-performance data integration workflows supporting economic research and analytics initiatives.
• Design conceptual, logical, and physical data models aligned with enterprise architecture standards.
• Optimize data storage, processing, and delivery frameworks to support large-scale analytical workloads.
• Implement workflow orchestration and automation solutions using tools such as Apache Airflow, Prefect, Dagster, or AWS Step Functions.
• Develop solutions that support data migration between on-premises and cloud environments.
• Collaborate with researchers, economists, and business stakeholders to understand data requirements and deliver scalable solutions.
• Perform root cause analysis and troubleshoot complex data quality, integration, and performance issues.
• Design and implement Change Data Capture (CDC) processes to support enterprise data warehouse solutions.
• Develop, test, deploy, and maintain data applications and services using industry best practices.
• Implement DataOps and CI/CD processes to improve deployment automation and operational efficiency.
• Support machine learning and advanced analytics initiatives through the development of reliable data pipelines and feature engineering frameworks.
• Ensure data governance, security, reliability, and scalability standards are met across enterprise platforms.
• Document architecture designs, technical specifications, and operational procedures., ETL/ELT & Pipeline Development
• Design and automation of ETL/ELT workflows.
• Data integration and transformation solutions.
• Workflow orchestration tools including:

• Apache Airflow
• Prefect
• Dagster
• AWS Step Functions

Cloud & Modern Data Platforms

Do you have experience in Version control?, Do you have a Master's degree?, * Bachelor's degree in Computer Science, Information Technology, Engineering, Data Science, or a related technical discipline.

• Advanced degree (Master's or Ph.D.) preferred.

Experience

• Minimum of 7 years of experience in data engineering, big data development, data architecture, or related technical roles.
• Experience designing and implementing enterprise data platforms supporting large-scale analytical workloads.
• Proven experience developing and optimizing enterprise data pipelines and integration frameworks.
• Experience supporting complex research, financial, or analytical environments.

Required Technical Skills

Data Engineering & Database Technologies

• Advanced SQL expertise.
• Strong experience with relational database platforms:

• PostgreSQL
• Microsoft SQL Server
• MySQL

• Database design, administration, and performance optimization.
• Data modeling and enterprise information architecture.

Programming & Development

• Advanced proficiency with:

• Python
• R

• Experience with additional programming languages such as:

• Java
• Scala
• JavaScript
• Perl

Big Data & Data Processing

• Experience working with:

• Distributed computing environments
• Scalable data processing frameworks
• Large-scale data storage architectures
• High-volume data workloads, * Experience with:
• Amazon Web Services (AWS)
• Microsoft Azure
• Snowflake

• Experience migrating data pipelines between cloud and on-premises environments.

DevOps & DataOps

• CI/CD pipeline implementation and automation.
• DataOps platform development and support.
• GitLab and GitHub source control management.
• Linux-based development environments.

Preferred Qualifications

• Experience working with economic, financial, or research-related data environments.
• Understanding of time-series data structures and forecasting methodologies.
• Experience with NoSQL databases and graph database technologies.
• Experience developing, training, deploying, and maintaining machine learning models.
• Experience implementing enterprise data warehouses utilizing CDC methodologies.
• Knowledge of enterprise architecture frameworks and modern data governance practices.
• Experience supporting highly regulated or federal government environments.

Desired Competencies

• Strong analytical and critical thinking abilities.
• Exceptional problem-solving and troubleshooting skills.
• Excellent verbal and written communication skills.
• Ability to communicate technical concepts to both technical and non-technical stakeholders.
• Strong customer-service mindset and collaborative approach.
• Self-motivated and capable of managing multiple priorities simultaneously.
• Ability to work independently while contributing effectively within cross-functional teams.

Diverse Agile Solutions (DAS) is a technology consulting firm delivering innovative IT, data analytics, cloud, cybersecurity, and digital transformation solutions to federal, state, and commercial clients. We are committed to helping organizations modernize their technology ecosystems through cutting-edge engineering, data-driven decision-making, and agile delivery methodologies. We are seeking a highly skilled Senior IT Big Data Developer to support the Federal Reserve Board's Data Architecture, Technology, and Analytics (DATA) Section within the Division of Research & Statistics (R&S). This role offers an opportunity to help transform how critical economic and financial data is ingested, managed, analyzed, and delivered to support research and policy initiatives., At Diverse Agile Solutions, you will work alongside talented professionals solving complex data and technology challenges for some of the nation's most impactful organizations. We offer opportunities to work with modern cloud technologies, advanced analytics platforms, and enterprise-scale data solutions while supporting critical missions that influence economic policy and research. Equal Opportunity Employer Diverse Agile Solutions (DAS) is an Equal Opportunity Employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability status, protected veteran status, or any other characteristic protected by federal, state, or local law. Apply today to join DAS and help shape the future of enterprise data architecture and analytics.