Every time a new feature was released, the existing playback, download, and purchase flows had to be looked at again. At kids audio platform company K, we took on QA to a recurring release schedule and left test assets for the next round of verification.
Checking only the newly added screens is not enough when verifying an update to an audio service. Users may replay content they bought before the update, or use content they had already downloaded.
Kids audio platform company K, too, had to check the existing experience while feature improvements kept coming. With no dedicated QA staff, it needed a way to keep verification going in step with the development schedule.
In this project, which ran from 2019 to 2020, we repeatedly verified the critical flows of the audio service and shared the results with the development team at every release. Bringing the check of new features and the re-check of existing ones into the same release process was the focus.
The standard for verification was the flow of using content, not the number of screens
That the content list is visible and the play button can be pressed is a starting point. The actual usage process also connects to content purchases, entitlements, and the use of downloaded files.
In the project, playback, download, purchase, and entitlement-based access were the primary verification targets. Testing was organized around the flow in which a user obtains content and uses it.
These flows are hard to look at in isolation. Whether a purchase is reflected in content access and whether the audio plays are separate check items. One of them being fine does not mean the other has been confirmed.
The longer a service is verified, the more important it becomes to record these relationships. Rather than a new person pressing through the screens for the first time and guessing the scope again, they should be able to inherit what has to be checked in connection with what.
With every new release, we re-checked the previous experience as well
An update is not necessarily just an addition of new features. When shared code or an integration changes, existing flows can be affected too.
In the company K project, we ran QA to the ongoing releases and kept regression testing going. It is the work of repeatedly confirming that features that worked before still hold after a change.
The point of regression testing is not to press the same screens as many times as possible. It is to keep the critical flows users continue to rely on as the standard, so that they are not missed in the next change.
A schedule that only looks at whether new features are done is different from one that also confirms the existing experience. In the project, we closed that gap by treating repeated verification as part of the release.
Older devices and operating systems were treated as verification conditions
The devices used in the development environment and the devices actual users have can differ. A playback result confirmed on one particular device cannot describe every usage environment.
The verification scope included combinations of older devices and operating systems. We looked at how the critical flows behaved in the agreed supported environments.
What matters is not having many devices, but connecting which environment was used to confirm what. Only when the environment in which a problem occurred can be described can the development team re-check it under the same conditions.
Supported environments have to be reviewed again as the product changes. When the combinations confirmed in a given release are on record, the scope to keep and the scope to add in later versions can be compared.
Problems found were passed on as information that could be used for the fix
An explanation like "there is no sound" is not enough to narrow down the cause. The reproduction conditions, the sequence of steps, and the observed result have to be there together before it can lead to a fix.
In the project, defects were organized by severity and reproduction information and shared. Verification scheduling, execution, and result reporting were run in step with the recurring release process.
The role of verification does not end with finding a problem. The information has to be handed over so that the development team can re-check and respond. Especially across a series of releases, being able to trace which problem was confirmed at which point keeps the discussion from getting mixed up.
Rather than putting the number of defects forward as a result, what mattered was whether the problems found remained as records that could actually be acted on. Making sure the same problem does not have to be explained from scratch each time is also part of the role of continuous QA.
The repeated verification was left as an asset the next person could take over
During the engagement, we organized the test strategy, the main scenarios, and the records used for repeated verification. Test assets were handed over so that the critical flows could be re-checked after the project ended.
Securing verification volume while a dedicated person is working and having a standard that remains after that person leaves are different outcomes. In the company K project, we left a foundation for subsequent verification alongside the immediate release response.
Test assets help prepare the next release. Inheriting which flows were repeatedly confirmed and under which conditions problems appeared means the starting point of verification does not have to be built anew each time.
A service that keeps shipping new features also needs a process that keeps confirming the existing experience. The company K case is an experience of maintaining the standard of verification, beyond a one-time check, throughout a period of continuous releases.
