Resource Considerations

Performance of the AI Script Generation process is highly dependent on the resources available to your test node as well as configuration choices that you make. Here are some general guidelines.

Browser Considerations

You can run the AI browsers in headless or non-headless mode. Headless mode is recommended because of increased performance and lower resource usage. As a general rule of scale, if you are able to run five browsers in non-headless mode, you can likely run ten to fifteen headless browsers with the same performance and resource usage.

Non-Headless Browsers

For non-headless mode, the main limiting factor is memory (RAM) on your test node. If you have the following amount of memory available:

  • 16 GB you will max out the memory with five non-headless browsers 

  • 32 GB you will max out the memory with ten non-headless browsers

Headless Browsers

For headless mode, the main limiting factor is your CPU speed and number of threads. You should be able to run ten to fifteen headless browsers with a base processor speed in the 2.4 to 2.6 GHz range. If you are able to increase that speed you will have better performance.

Number of Browsers vs Performance

If you are experiencing slow performance, try reducing the number of browsers and see if performance increases.

For example, if you run too many browsers and max out the machine's resources, the AI Script Generation process will run slower. For example, if you run fifteen browsers, but the machine resources actually max out at ten browsers, you will get poor performance. This is because the machine is being overworked trying to run the fifteen browsers. If you actually ran ten browsers, you would see a significant performance increase because the machine isn't being overworked, even though resources may be maxed out.

Amazon Web Services Test Nodes

AWS servers use a base processor speed of 2.4 to 2.6 GHz. Since the AWS instance is not a fixed machine, you may be able to pay for credits to increase the processor speed.

Delay Between Requests

Delay Between Requests (ms) is an option that you specify for an AI Script Generation execution that determines how much time AIQ waits for the application to load before it reports a failure while navigating and performing actions. The longer the wait time, the longer it will give the application a chance to respond before reporting a failure.

A side effect of the higher a delay you specify is that you may be able to run more browsers. Delays make some browsers go to sleep, and because of this you can you share processor resources with other browser instances during that time.

For example, you find that ten browsers work best with a delay between requests of 0 ms. However, if the delay between requests is 20 ms then you can run fifteen browsers.

The ability to run more browsers if you specify a higher delay between requests is not a direct correlation. It is merely a side effect. Benchmarking and performance testing is encouraged. See Basic Benchmarking for suggestions.

Types of Applications

The types of applications that you are testing will factor into how many browsers you can run.

  • Static - If the web application is very static and very few elements, with a standard 8 core CPU with 32 GB of memory, you may be able to run up to twenty headless browsers.

  • Dynamic - If the application is highly dynamic (moving elements, Shadow DOM, iframes, etc) and the scan type is tags you should do some basic benchmarking.

Basic Benchmarking

You can perform some initial benchmarking to see if you may need to change some execution parameters.

  • Run a quick test AI Script Generation execution. You could even test a mix of headless and non-headless browsers depending on your situation.

    For example if you want to test the difference between headless and non-headless browsers:

    • Run an execution with ten headless browsers.

    • Run a second execution with five non-headless browsers.

  • Monitor how much memory and processor resources are utilized.

  • Use this information to decide if you need to run a new AI Script Generation with more or fewer browsers.

SmartTags and Validations

Running scans with large numbers of SmartTags and Validations will impact performance.

AIQ Limitations

Despite any external resources, how powerful your test node is, configuration settings, number of SmartTags and validations you use, AIQ has its own limits. The absolute maximum number of browsers that you can use is fifty.

However, the optimum number of browsers even if you have abundant computer resources is actually somewhere between thirty-five and fifty browsers.