Even when an app's features work correctly, whether data and permissions are handled safely has to be confirmed separately. In the pre-release testing for mobile app company R, we examined the device, communication, authentication, and the server API together, and re-verified the defects that were fixed.
When there is no button on the app screen, it can look as though that function cannot be executed. But the requests the server receives have to be checked independently of what the screen restricts.
Testing only the behavior visible to users can miss the permission handling and input validation on the server the app calls. What information the app leaves behind, and how communication and authentication work, are further items to check.
In company R's project, which was preparing a release on two mobile operating systems, IXC placed the app and the server inside the same testing scope. The work was to confirm not only the success paths of features but how access that must not be allowed is handled.
First we defined the targets and the permitted scope
Security testing is not a matter of probing freely wherever one likes. There has to be an agreed scope covering which app and server, and which features and requests, are targeted.
At company R, the inspection scope was fixed in writing before kickoff, and the items were divided between the app and the server for execution. What to confirm before release was made explicit.
Only when the scope is defined first can the results be explained accurately. The result of inspecting some APIs must not be stretched into the security of the entire service, nor the result for one version applied as-is to every version after it.
In this case, too, the items inspected and the items left out of scope were recorded separately.
The device, communication, and the server each faced a different question
On the app, we checked the data left on the device, the communication channel, and the authentication flow. On the server, we examined API permissions and input validation.
- Device
- What data is left behind while the app is used?
- Communication and authentication
- How are data transfer and user verification handled?
- Server API
- Is the request authorized, and is the input validated?
No single test can answer all of these questions. That is why a successful app login does not let you conclude that permissions on every server request are handled correctly.
Whether a feature can be used, and whether the right to use it is checked, are different things to verify.
We also checked requests that bypass the screen
In the server testing, we also ran requests that called the server directly without passing through the app screen. Rather than relying on the screen's restrictions alone, we confirmed the criteria by which the server handles a request.
The purpose of this kind of testing is not to run as many specific attack techniques as possible. It is to examine the boundaries of permission and input handling that can stay hidden in normal usage flows.
For example, even when an action is restricted on the screen, whether that restriction is also enforced on the server has to be confirmed separately. This does not mean a particular breach occurred in this service; it is a question that testing has to ask.
At company R, the problems confirmed by looking at the app and the server together were organized into records with severity and reproduction conditions.
After a fix was reported, the same conditions were run again
After reporting defects, we re-verified the results of the fixes. The same items were run again to confirm whether each had been resolved.
Having fixed code and having resolved the problem originally found are different states. Only by checking the same conditions in an environment where the change is actually deployed can the outcome of a fix be explained.
In this inspection, reporting and fix confirmation were not separated. What was found and how that item was checked again were kept connected.
That said, resolving the defects found does not mean the service has no other problems at all. Security test results have to be interpreted within the target version, the scope, and the conditions used.
We left the boundaries and evidence the release decision needed
The project was left with the app and server inspection results, the record of defects and fix confirmations, and the testing scope with its excluded items. A check list that can be consulted again at later releases was organized as well.
The value of this deliverable does not lie in rating the whole service with a single sentence that it is safe. It lies in being able to explain how far was checked and which fixes were verified again.
Judging an app's quality by on-screen behavior alone leaves areas unseen. The information left on the device, the communication, and the conditions under which the server accepts a request are part of the service too.
Company R's case is a project that did not confine pre-release testing to the app but extended it to the server that processes the data.