Data Engineer & Insights Analyst

• Create, maintain, and optimize scalable data pipeline architecture.
• Design and implement cloud-based data solutions using AWS services such as S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.
• Build and maintain data orchestration workflows using Apache Airflow, AWS Step Functions, or similar technologies.
• Assemble and transform large, complex datasets to meet functional and non-functional business requirements.
• Engineer analytical data environments that support statistical analysis, predictive modeling, machine learning, and evidence-building activities.
• Partner with data scientists, research analysts, and policy analysts to support reproducible research and statistical modeling.
• Automate manual processes, optimize data delivery, and redesign infrastructure to improve scalability and efficiency.
• Build ETL processes using SQL, big data technologies, and multiple data sources.
• Develop analytics tools that provide actionable insights into education and workforce outcomes, operational efficiency, and key performance metrics.
• Develop and maintain pipeline monitoring, logging, and alerting frameworks.
• Implement data quality checks and validation frameworks across data pipelines.
• Develop dashboards, reports, analytical models, and other decision-support products.
• Support stakeholders across executive, data, design, and technical teams with data infrastructure and technical needs.
• Ensure compliance with Commonwealth data governance standards, privacy requirements, security controls, and interagency data-sharing agreements.
• Develop tools that help analytics and data science teams build and optimize data products.
• Support data conversions associated with system modernization and replacement initiatives.

Analysis Skills, Operational Improvement, Team Player, Analysis Skills, Large-Scale Systems, Operations Research, Big Data, Product Support, Data Sets, Reporting Dashboards, The role combines data engineering and analytics, with responsibility for designing scalable data pipelines, integrating and transforming data from multiple sources, improving data quality, and developing dashboards, analytical models, and other decision-support products. The ideal candidate will have strong experience with SQL, AWS technologies, big data environments, and complex datasets, along with the ability to support multiple teams and data initiatives., * Advanced SQL knowledge with strong experience working with relational databases and query authoring.

• Experience building and optimizing big data pipelines, architectures, and datasets.
• Advanced experience designing and building dashboards, reports, analytical models, and decision-support products for executive and non-technical audiences.
• Experience working with student-level higher education data and linking it with other sources, such as employment and wage data, to measure outcomes and generate insights.
• Experience performing root-cause analysis across internal and external data and processes.
• Strong analytical skills working with unstructured datasets.
• Experience building processes supporting data transformation, data structures, metadata, dependencies, and workload management.
• Demonstrated experience manipulating, processing, and extracting insights from large, disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable big data stores.
• Strong project management and organizational skills.
• Experience working with cross-functional teams in a dynamic environment., * Experience with AWS data technologies, particularly S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.
• Experience with Apache Airflow, AWS Step Functions, or comparable data orchestration technologies.
• Experience supporting advanced analytics, statistical modeling, predictive modeling, or machine learning environments.
• Experience supporting data modernization, system replacement, or large-scale data conversion initiatives.
• Experience working with government, education, workforce, or longitudinal data.
• Familiarity with data governance, privacy, security controls, and confidential datasets., The strongest candidate will combine hands-on data engineering expertise with strong analytical judgment and the ability to translate complex datasets into meaningful insights. You’ll be highly proficient with SQL, data pipelines, ETL processes, cloud-based AWS technologies, and large-scale data environments while being comfortable developing dashboards and decision-support products for technical and non-technical stakeholders.

You’ll also be self-directed, organized, and comfortable supporting multiple teams, systems, and products. Experience working with higher education, employment, workforce, or longitudinal data will be especially valuable in helping strengthen the Commonwealth’s data architecture and support data-informed decision-making.