Test Designer Performance Scalability: Maximizing the number of concurrent browsers per node

For performance testing, you can maximize the number of concurrent browsers that can be opened in a node.

For AWS EC2 nodes running AIQ, the number of concurrent Chrome browsers that can be assigned per node is around 20 to 30, depending on the application.

The main bottleneck to achieving more concurrent browsers is the speed of the storage volume as measured in IOPS (input/output operations per second). A typical 60GB EBS volume is not rated higher than 3000 IOPS.

To avoid the bottleneck of volume speed, you can leverage AWS EC2 instances that include NVME SSD volumes. These types of volumes are very fast, but they are ephemeral storage volumes. Any data stored in these volumes will be lost once the instance is stopped or terminated. Also, the data in the NVME SSD volumes are not saved in the AMI images.

So the process of starting appvance in one of this nodes involves several initial steps which are coded in the provided nvme_setup.sh script recommended for performance nodes. The script does:

  1. Mount the NVME SSD storage as a device visible to the OS

  2. Copy GraalVM and aiq_distribution folder to the NVME SSD

  3. Modify global.json in aiq_distribution/NetworkProxy/config/global.json so Chrome browser has the following option added:

    Copy 
    {
         "name": "experimentalDS3",
         "value": "--disk-cache-dir=cache||--disk-cache-size=1"      
    }


  4. Start AIQ from the NVME SSD drive.

Setting up a Amazon Linux 2 node for performance testing

  1. Add the following file (nvme_setup.sh) to the aiq_distribution folder in the Amazon Linux 2 instance.
    This script will mount the NVME SSD volume, if found, and then copies the Graalvm and aiq_distribution folders to the SSD drive.
    Note: This script assumes the AIQ is located at /opt/aiq/aiq_distribuition. Edit the script if the AIQ location is different.

    Copy

    nvme_setup.sh

    #!/bin/bash
    # This script assumes:
    #  - using a AWS ec2 instance with an ephemeral SSD NVME volume (for example m5ad, c5d instance families)
    #  - AIQ is installed at /opt/aiq/aiq_distribution, modify below if not.
    #  - Linux has NVME volumne drivers (included in Amazon Linux 2 OS)
    #  - graalvm is installed and $JAVA_HOME is set to its location
    if [ -e '/dev/nvme1n1' ] ; then
        echo "Mounting NVME volume as /data"
        sudo file -s /dev/nvme1n1
        sudo mkfs -t xfs /dev/nvme1n1
        if [ ! -d '/data' ] ; then
           sudo mkdir /data
        fi
        sudo mount /dev/nvme1n1 /data
        sudo chown ec2-user:ec2-user /data
        sleep 5
        FOLDER=$(basename $JAVA_HOME)
        sudo cp -R $JAVA_HOME /data
        export JAVA_HOME=/data/${FOLDER}
        export PATH=/data/${FOLDER}/bin:$PATH
        echo "JAVA_HOME: $JAVA_HOME"
        sudo cp -R /opt/aiq/aiq_distribution /data/aiq_distribution
        sudo chown -R $USER:$USER /data
        cd /data/aiq_distribution/TestNode
    else
       echo "Skipping mounting /data as instance do not have an NVME volume (/dev/nvme1n1)"
    fi
    bash startup.sh


2. Modify global.json in aiq_distribution/NetworkProxy/config/global.json so that Chrome browsers have following option added:

Copy 
{
      "name": "experimentalDS3",
      "value": "--disk-cache-dir=cache||--disk-cache-size=1"      
}

3. Start AIQ started using a systemd serviceDo not use /etc/rc.local.
The systemd service can set the maximum number of open files per process. This is needed in linux distros because the default of 1024 files per process is too low.
Note: Staring AIQ as a service in this manner is not compatible with the Lab Management automatic upgrade for linux nodes.

To configure the service create a text file named appvance.service in the /etc/systemd/system folder with the following contents. Edit the paths in file as needed to match your configuration.

Copy 
[Unit]
Description=Start Appvance IQ services
After=syslog.target
[Service]
Environment=PATH=/opt/graalvm-ce-java11-21.3.3.1/bin:/sbin:/bin:/usr/sbin:/usr/bin
Environment=JAVA_HOME=/opt/graalvm-ce-java11-21.3.3.1
User=ec2-user
Type=simple
LimitNOFILE=10000000
LimitNPROC=20000
ExecStart=/bin/bash /opt/aiq/aiq_distribution/TestNode/nvme_setup.sh
WorkingDirectory=/opt/aiq/aiq_distribution/TestNode
TimeoutStartSec=0
[Install]
WantedBy=multi-user.target


After creating above file, execute the following commands in the shell:

systemctl daemon-reload
systemclt enable appvance

With the above configuration, an instance with size of m6id.2xlarge AIQ was able to execute a test designer test case using 40 chrome browsers concurrently. An instance with a size of m6id.4xlarge was able to execute 80 chrome browsers concurrently. Please note that specific values might vary depending of the application under test.