> Markdown version of [/jobs/ext/3390782-automation-test-engineer](https://www.wearedevelopers.com/jobs/ext/3390782-automation-test-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Automation Test Engineer - **Company:** Skyward Inc - **Location:** United States (Remote available) - **Experience:** Expert - **Salary:** $110,000.0 - $130,000.0 - **Contract:** Permanent contract - **Skills:** Testing (Software), Training Data, Application Programming Interfaces (APIs), Agile Methodology, Artificial Intelligence, Artificial Neural Networks, Automation of Tests, Continuous Integration, Data Files, Github, Python (Programming Language), Machine Learning, Azure Machine Learning, Selenium, Software Testing Automation Framework, Strategies of Testing, Management of Software Versions, Web Applications, Scripting, Delivery Pipeline, Model Validation, Cypress (Programming Language), Gitlab, Playwright, Machine Learning Operations, Devsecops, Jenkins - **Published:** September 12, 2026 - **Apply:** https://www.builtincolorado.com/job/automation-test-engineer/11139801?handler=ApplyRedirect ## About the Role * 5+ years of experience in software QA/automation engineering, with at least 1 year focused on AI/ML product testing. * Strong proficiency in at least one automation framework (e.g., Selenium, Cypress, Playwright) and scripting language (Python preferred). * Hands-on experience with testing AI models, including validation metrics, dataset versioning, and model explainability. * Familiarity with tools like MLflow, TensorBoard, or similar for AI observability. * Understanding of model bias, fairness audits, and compliance considerations in government or regulated environments. * Experience working in Agile DevSecOps environments with modern CI/CD tooling (Jenkins, GitHub Actions, GitLab, etc.). * Strong collaboration skills with a knack for clear communication across engineering, data science, and product teams. ## Description Designs and maintains automated test frameworks for web applications, APIs, and AI-driven features. Develops model validation, bias detection, drift monitoring, test datasets, and CI/CD quality gates. Executes functional, regression, performance, and AI-specific testing while partnering with AI/ML, data science, and product teams. The role also promotes responsible AI practices, mentors junior engineers, and supports a government CMS platform. The summary above was generated by AI We are Skyward. That is, a love for people, for improvement, for human advancement through information technology. We are a people-centered business with a desire to serve others. We are diverse and unified; creative and collaborative; a collection of complementary, not competing talents. And though on the surface we remain relaxed, beneath, a torrent of energy links us to our civic tech mission. We stand by our values, and we won't compromise on any of them. Integrity: We're conscientious, intentional, and empathetic. Our words and actions align. That's our character. Please don't ask us to play another part, we're poor actors. Compassionate: If we may borrow a quote from Theodore Roosevelt: "No one cares how much you know until they know how much you care." Because our team is thoughtful and supportive, caring deeply for each other, our clients, and our work, this comes naturally. Inquisitive: We remain students by failing openly and turning lessons into solutions. Unconventional: For us, life isn't what happens outside of work. Work happens inside of life and our culture erases the line often dividing the two. Authentic: Made possible only because we embody the values listed above. We're relaxed and fun yet intensely curious and driven. Team members are placed with thought, care, and precision to ensure that Trust, Truth, and Transparency continue to represent our brand. Because of that, we continue Onward, Upward, and Skyward. We need an Automation Test Engineer. Are you the kind of person who can't look at a machine learning model without wanting to test it six different ways (and then automate all six)? Do your friends sigh when you bring up responsible AI? Maybe they've stopped asking what you do for work because the last time you went on a 20-minute tangent about bias in training data and flaky CI pipelines for neural nets. If that sounds familiar, you might be exactly who we're looking for. Come join us if you're motivated to learn from others, to learn from mistakes, to be part of a future-looking and growth-oriented team. Let's go Skyward together. What you'll do: * Be part of a team supporting the development of an AI/ML platform that will leverage generative AI to relieve user burden across the Centers for Medicare and Medicaid Services (CMS). * Design and maintain robust, scalable automated test frameworks for web apps, APIs, and AI-driven features. * Develop automated pipelines for model validation, bias detection, and drift monitoring in ML-based systems. * Define and execute test plans that include functional, regression, performance, and AI-specific testing strategies. * Partner with AI/ML engineers to create test datasets and simulate real-world user behavior and edge cases. * Implement CI/CD-integrated quality gates to ensure AI releases meet defined reliability and fairness metrics. * Mentor junior engineers and help establish best practices in intelligent automation and AI testing. * Advocate for ethical AI standards and contribute to cross-functional discussions about responsible AI. ## Related Videos - [GitLab CI pipelines for a whole company](https://www.wearedevelopers.com/videos/143-gitlab-ci-pipelines-for-a-whole-company) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [Why Testing Matters in AI - Luise Freese and Elio Struyf](https://www.wearedevelopers.com/videos/1837-why-testing-matters-in-ai-luise-freese-and-elio-struyf) - [Enabling automated 1-click customer deployments with built-in quality and security](https://www.wearedevelopers.com/videos/83-enabling-automated-1-click-customer-deployments-with-built-in-quality-and-security) - [Are Classical Automation Frameworks Dead? How AI Agents Are Transforming QA](https://www.wearedevelopers.com/videos/100243-are-classical-automation-frameworks-dead-how-ai-agents-are-transforming-qa) - [WeAreDevelopers LIVE - Modern DevOps for IoT Devices and More](https://www.wearedevelopers.com/videos/1805-wearedevelopers-live-modern-devops-for-iot-devices-and-more) ## Related Articles - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [What is Software Engineering in the Age of AI?](https://www.wearedevelopers.com/magazine/640-what-is-software-engineering-in-the-age-of-ai) - [Transforming Software Development: The Role of AI and Developer Tools](https://www.wearedevelopers.com/magazine/527-transforming-software-development-the-role-of-ai-and-developer-tools) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)