We are thrilled to announce a major enhancement to the Ostorlab mobile vulnerability scanner: the AI-powered Monkey Tester. This new feature significantly boosts test coverage and speed, providing a more comprehensive and efficient testing experience.

What is the Monkey Tester?

For those unfamiliar, the Ostorlab Monkey Tester is designed to automate interactions with mobile applications to achieve maximum coverage. It fully automates the testing process, including handling authentication using simple credentials and supporting multi-factor authentication (MFA).

This article will showcase this new capability and give you a behind-the-scenes look at how it works.

How It Works

The Ostorlab Monkey Tester automates interactions and generates a call coverage graph, illustrating the different explored scenarios within the application.

Originally, the Monkey Tester employed three main strategies to achieve high coverage:

1. Pure Randomness: Randomly interacting with the application.
2. Rule-Based: Following predefined rules to navigate the app.
3. Genetic Algorithms: Utilizing testing "minions" with a set of parameters to explore different paths.

While these strategies served well, they required consistent tweaks and improvements and often missed the most complex workflows. Additionally, they struggled to answer the question, "How did I get here?"

The New AI-Powered Approach

The new feature leverages generative models to reason about test scenarios within an application similarly to how a human would. For each page, screen, or activity, the Monkey Tester generates a set of test scenarios and executes them sequentially.

The new AI-powered approach excels in handling complex, multi-step actions such as creating an account, purchasing an item, or answering questionnaires. These actions require several interactions to be executed in a specific sequence, a task where genetic algorithms often take time to converge. With the AI-powered Monkey Tester, these scenarios are tested more efficiently, ensuring comprehensive coverage of intricate workflows.

Behind the scenes, it’s much more complex. Once a scenario is exercised, you often can’t go back and rerun it without starting from scratch. To address this, the Monkey Tester constructs a scenario graph to keep track of executed scenarios and those that still need to be run.

This approach is coupled with the shared persistence of generated scenarios, accelerating testing in future iterations. This has demonstrated a massive performance boost, with over a 1000% increase in application coverage in some scenarios and over a 300% increase on average.

How to Use It

If you are already using Ostorlab’s dynamic analysis, simply enable the AI feature from the organisation settings menu. You should start seeing new scenarios added to your call coverage with clear explanations of what was covered.

If you want to take it for a test drive, please reach out to us.

This enhancement makes the Ostorlab mobile vulnerability scanner more powerful and user-friendly, ensuring your applications are thoroughly tested and secure.