Data Operations Engineer II
Role details
Job location
Tech stack
Job description
Want to be part of a team that builds, monitors, and maintains the data ingestion systems behind the world's most accurate weather source, running reliably around the clock? We are looking for a Data Operations Engineer II who displays a deep interest in and passion for scripting, automation, and operational reliability; an experienced and detail-oriented individual who likes to diagnose systems and help identify areas of improvement. Reporting directly to the Manager, Data Operations, this position will be responsible for building, monitoring, and independently maintaining the data ingestion processes and systems that move, store, and transform data across platforms - including administering Databricks, our Data Marketplace and Ad Platform, and the core weather, locations, and provider data that power AccuWeather's products. You will develop, maintain, and optimize complex scripts to automate data collection and monitoring, build end-to-end monitoring systems and automated alert mechanisms to ensure pipeline health, and troubleshoot issues to ensure continuous operation and data integrity.
You will demonstrate skill across multiple platforms, solve technical problems independently, and collaborate with peers and stakeholders in an Agile-based environment, to deliver reliable data operations that cater to a global audience.
We are open to hiring candidates to work from our global headquarters in State College, PA, USA, or remotely., * Build, monitor, and maintain data ingestion processes and systems that move, store, and transform data across platforms, administering Databricks and our Data Marketplace and Ad Platform.
-
Develop, maintain, and optimize complex scripts using Python, PowerShell, or similar languages to automate data collection and monitoring.
-
Own the operational state of a defined set of data pipelines - monitoring, incident response, maintenance procedures, and escalation for systems in your scope.
-
Own SQL database maintenance (Forecast, current, historical, locations) for data loading and deployment.
-
Develop and maintain unit test coverage for Data Ops code bases, ensuring scripts and pipelines behave as expected before deployment.
-
Administer GoAnywhere and other managed file transfer platforms.
-
Build and manage end-to-end monitoring systems and automated alert mechanisms to optimize the health and performance of data pipelines, ensuring alerts are appropriately calibrated to trigger the right notifications.
-
Own incident diagnosis and resolution within established parameters - investigating alerts, identifying root causes, implementing fixes, validating recovery, and documenting resolutions.
-
Build and maintain CI/CD pipelines (GitHub Actions, Azure DevOps) to support reliable, repeatable deployments
-
Own proactive operational maintenance - executing scheduled maintenance, applying patches and updates, and performing checks that prevent incidents before they occur.
-
Create and maintain comprehensive documentation of scripts, automation and monitoring workflows, data pipelines, and deployment procedures.
-
Contribute to and improve runbooks based on incidents you've worked through that weren't adequately covered before.
-
Collaborate closely with cross-functional teams, providing support for scripting needs and contributing to automation strategy.
-
Independently plan and prioritize work to meet deadlines.
-
Begin sharing knowledge with peers; engage in informal coaching moments.
-
Performs other duties as assigned.
Requirements
o BA/BS in Computer Science, Information Technology, or a related field, with 3+ years of experience in data operations, scripting, and automation.
o Intermediate proficiency in scripting languages such as Python, PowerShell, Bash, or similar.
o Fully competent in Azure, Databricks, Snowflake, and SQL Server.
o Experience managing relational databases such as SQL.
- Preferred Qualifications:
o Proficiency in Object-Oriented Programming (OOP) languages such as C++, Java, or similar.
o Experience with DataDog or similar tools for monitoring and performance tracking.
o Experience with deployment automation (CI/CD) tools and processes.
o Administration experience with managed file transfer tools (e.g. GoAnywhere)
o Experience working with cloud platforms (e.g., AWS, Azure, GCP).
o Familiarity with version control systems (e.g., Git).
- Skills & Competencies:
o Exceptional analytical and problem-solving skills with meticulous attention to detail.
o Demonstrated ability to debug systems, tracing issues back to their source.
o Excellent communication skills with the ability to collaborate effectively within a team environment.
o Ability to adapt to changing priorities and manage multiple tasks simultaneously.
o Provides clear incident updates on a defined cadence during active incidents.