Brien Colwell

Using non-functional testing to guide user interface, backend services, voice interface, and media development

How do you prevent your app from getting slower with every new feature? Use non-functional testing to build a data-driven safety net against performance regressions.

Using non-functional testing to guide user interface, backend services, voice interface, and media development
#1about 3 minutes

Understanding non-functional vs functional testing

Non-functional testing measures performance, quality, and usability against baselines, moving beyond simple pass/fail criteria to create a five-star user experience.

#2about 7 minutes

Building a data-driven regression safety net

Establish a framework to catch non-functional regressions by collecting data, identifying hotspots, measuring against baselines, and fixing performance drops.

#3about 7 minutes

Methods for collecting raw performance data

Collect raw data for analysis by either instrumenting the application with code bookends or by instrumenting the test environment to capture video, network, and system stats.

#4about 9 minutes

Analyzing raw data to create useful metrics

Transform raw data from instrumented environments into actionable metrics like page load time by defining a region of interest and applying visual analytics.

#5about 3 minutes

Structuring a cross-functional testing team

Integrate non-functional testing into project teams by combining the skills of product engineering, data analysis, and testing to drive improvements.

#6about 4 minutes

Applying non-functional testing to gaming performance

Use non-functional testing in game development to set and track baselines for CPU usage and frames per second (FPS) across different devices.

#7about 3 minutes

Testing the responsiveness of voice interfaces

Measure and improve the time-to-respond for voice inputs by setting a baseline, such as 500 milliseconds, and testing across various locales and devices.

#8about 4 minutes

Ensuring quality in streaming media applications

Monitor streaming media performance by tracking loading animations, buffering events, resolution switches, and perceptual quality factors like blockiness and blurriness.

#9about 9 minutes

Q&A on tools, frameworks, and implementation

Discussion covers recommended tools like Appium, integrating tests into CI/CD pipelines, and the benefits of instrumenting the test environment over the application.

Related jobs
Jobs that call for the skills explored in this talk.

Featured Partners

Related Videos

See all videos
Measure and improve frontend performance by using test automation25:38

Measure and improve frontend performance by using test automation

Ramona Schwering

How not to test 43:07

How not to test

Golo Roden

Implementing smoke testing and proactive monitoring in production1:29:04

Implementing smoke testing and proactive monitoring in production

Liam Hurrell & Jan Kunzmann

How To Test A Ball of Mud42:34

How To Test A Ball of Mud

Ryan Latta

Continuous testing - run automated tests for every change!47:22

Continuous testing - run automated tests for every change!

Christian Kühn

Test Driven Development with JavaScript1:42:16

Test Driven Development with JavaScript

Sascha Depold & Parinaz Roghany

Your Testing Strategy is broken - lets fix it!28:12

Your Testing Strategy is broken - lets fix it!

Luise Freese

Frontend Performance Testing in practice36:50

Frontend Performance Testing in practice

Jonas Kröger

From learning to earning

Jobs that call for the skills explored in this talk.