Data Engineer

The Data Engineer is responsible for designing, building, and maintaining data pipelines to ensure seamless data flow and integration. This position leverages tool such as Python, SQL, and Delta Lake within the Microsoft Fabric Ecosystem to acquire, transform, and manage data from source systems like Ellucian Banner as well as external APIs. The Data Engineer ensures data is effectively structured, reliable, and optimized to support college-wide data needs and various strategic initiatives. Working extensively with enterprise systems and emerging data technologies, the Data Engineer creates a robust foundation for the college’s data ecosystem and may explore advanced techniques to enhance data reliability and scalability. Utilize expertise in data Lakehouse management, data pipeline creation/support, and analytical reporting skills. This position will acquire and analyze data from various internal and external sources, designing a cloud-based data platform for analytics and business intelligence, providing consistent and reliable data to our Business Intelligence Analysts and the campus community at large. In addition, the Data Engineer will work with the Research, Assessment, and Special Initiatives (RASI) Vice President supporting Project Management efforts., Manage workflows, projects, and provides training as needed. Use Python and SQL to extract and integrate data from Ellucian Banner and external APIs, ensuring seamless data flow into the Microsoft Fabric Lakehouse solution. Design, develop, and deploy high-performance Python-based data pipelines for structured, semi-structured, and unstructured data. Implement performance tuning techniques to improve data engineering workloads. Work with Microsoft Azure and Fabric cloud services to design and implement robust cloud data architectures. Ensure the security and compliance of data products through the implementation of appropriate controls and data governance best practices. Partner with college faculty and staff (data analysts, stakeholders, and other IT-engineers) to identify data needs and deliver tailored data solutions. Design and maintain reporting dashboards and visualizations to track key performance indicators for various college initiatives. Assist the RASI Vice President with Project Management to ensure initiatives are completed in a timely and efficient manner. Create and maintain scripts and reports to facilitate the rapid completion of federally required surveys, such as IPEDS. Perform data collection assignments including data retrieval/collection and reporting for faculty and staff requestors. Other duties as assigned.

Associate’s degree and three years of related experience OR Bachelor’s degree and one year of related experience.

Experience in computer programming with Python and SQL.

Preferred Qualifications: Bachelor’s degree.

Microsoft Fabric Analytics Engineer Associate Certification., CompTIA Project + Certification.

Technical experience with Ellucian Banner.

Experience in higher education.

Knowledge, Skills, and Abilities

Knowledge of various programming languages such as Python and SQL.

Knowledge of data visualization and reporting tools such as Power BI, Tableau, and Looker.

Knowledge of Data Transformation tools such as Polars, Pandas/NumPy, Apache Spark, dbt, etc.

Knowledge of Delta Lake features like time travel, schema enforcement, or ETL optimization.

Knowledge of cloud data ecosystems such as Microsoft Fabric, AWS Data Lake, Databricks, Google Data Lake, and Snowflake.

Knowledge of data orchestration tools such as Apache Airflow, Azure Data Factory/Fabric Data Factory, and Google Cloud Composer.

Strong understanding of REST API concepts.

Familiarity with Java, Scala, and Rust is a plus.

Ability to write shell scripts using languages such as PowerShell or bash to automate data refreshes effectively.

Ability to handle multiple tasks simultaneously and meet deadlines.

Ability to restructure transactional data into meaningful data models, such as star schemas.

Ability to restructure transactional data into meaningful data models, such as star schemas.

Must be detail-oriented, highly organized, and self-motivated.

Pay Rate: $64,257 - $80,320 annual salary depending on experience

Volunteer State offers a comprehensive benefits package, including but not limited to the following:

Insurance: Health, Dental, Vision, Life, Disability (long and short term), FSA/HSA

Wellness Incentive Program (if enrolled in Health)

Educational Assistance: Fee Waiver, Spouse/Dependent Discounts, Tuition Reimbursement

Employee Assistance Program

Longevity Pay

Retirement Options: State of TN Pension (TCRS), 401a, 401K with up to $50/month match, 403b, and 457

14 Paid Holidays/Year

Annual Leave (if applicable)

Sick Leave

Sick Leave Bank

State Employee Discount Program with over 900 merchants