Appvance CICD

Appvance AI – CI Integration

Abstract

The purpose of this document is to give an initial guide about the implementation of Appvance IQ scenarios and blueprints in a CI tool.

Introduction

Appvance IQ can provide integrated smoke-test testing for any web-based application’s builds. All manual and AI-generated UX scripts are written in Test Designer. Test Designer is our technology in UX automation, that includes "state of the art" features like codeless mode, accessor fallback, accessor self-healing, auto-validation with SmartTags, support for all major browsers, and platforms, and support for major frameworks like Angular, KendoJS, ViewJS and others.

The solution involves three main components:

1. Appvance IQ scenarios. These scenario files can contain many test cases with several scripts each, even several iterations.
2. Blueprint templates to have the CICD tool execute an AI blueprint scan (which includes API and UX level validations and traverses the application once the initial AI hinting process is done).
3. A CICD Tool that supports Apache Ant jobs (Jenkins, Bamboo, TeamCity, Circle CI, TFS, gitlab.com, etc)

SOLUTION

Our technology is completely restful, so in order to trigger a build, you can use any CI tool able to trigger an HTTP call. We can provide an ANT-based solution to easily integrate with TeamCity to launch builds, tests, and upload results to an online set of Dashboards showing the results. To easily launch Appvance IQ processes you will need the AppvanceRestClient.

Appvance Rest Client

The Appvance Rest Client enables any CI tool to connect to Appvance. It can be used from any CI Engines that support ANT tasks like GitLab, Jenkins, TeamCity, Circle CI, Bamboo, etc. The Appvance Rest Client is a .jar that is located here https://s3-us-west-2.amazonaws.com/appvanceivy/projects/AppvanceCIClientWithDeps/master/latest/ivy

You can use the following line inside abuild.xmlfile (sample included in a .zip below) target to download and name it as AppvanceRestClientWithDeps.jar:

<get src="https://[aiq-controller-domain]:8443/UI/container/AppvanceCIClientWithDeps.jar" dest="${lib}/RestClient.jar"/>

Find in the next section a zip with a complete setup including the build.xml file.

Triggering AppvanceIQ pipelines using Apache ANT Targets

Providing a CI tool with support for Ant tasks, like Jenkins, whole integration can be accomplished by Ant jobs. The only requirement is to set up a repository with a build.xml that has the targets and that also downloads our RestClient.jar. We currently can provide this data as a starting point, assuming some changes will be requested to adjust to any required workflow.

Here is a.zip that includes all required files, including a sample documented build.xml: CICD_Pipeline_demo.zip

Setting the CI Tool (Jenkins used as example)

Create a free style project in Jenkins. Make sure ANT_HOME is properly set in the system as this is required:

Configure the repository that has the required files (4.3.1) in the project:

Set the Build Environment to use Ant targets:

Set the trigger configuration as you want. In the image, the trigger is set to run once daily, but you can set the Poll SCM option to configure the trigger to happen once the polling of a given repository detects new commits.

Next step is to add an Ant Job in the Build section:

Some available Target Descriptions

REPORTING AND DASHBOARDS

We will have several levels of Summary Dashboard.

1. Global Main Dashboard
   Show global summarized data at environment (stage) level (alpha, beta, staging, qa4, etc) and per product (web player, IOS player, Android Player)
   Product A
   Environment A
   LastExecution "Scenarios", Link to history dashboard and link to execution report
   LastExecution "Blueprints", Link to history dashboard and link to execution report
   Environment B
   LastExecution "Scenarios", Link to history dashboard and link to execution report
   LastExecution "Blueprints", Link to history dashboard and link to execution report
   
   
2. History Dashboard
   This shows all the latest tests and blueprints ran per each environment per product, like this one:
   below will be other environments (stages) if available.
3. Execution Historic Dashboard
   If you click one of the above rows, it will show the historical summaries for results on that specific test/blueprint.
4. Single execution Details
   Clicking on an individual execution in the previous Execution Historical Dashboard we will see specific details on that test successes and failures.

Sample AI Main dashboard report:

In the following link you will be able to see a sample dashboard for an AI Blueprint scanning.
https://appvance-dev-cicd.s3-us-west-2.amazonaws.com/products/main.html

A triggering system using CURL using Appvance Gitlab and Amazon AWS IT

This section is added as an example of other possible ways to interact with Appvance IQ. Currently, some clients do prefer us to manage this integrationusing our Gitlab and Amazon AWS ITand they use CURL to trigger builds when they want. So, to trigger a build in Linux/Windows using CURL the command line would be

curl -X POST -F token=TOKEN \
     -F ref=ENVIRONMENT \
     https://gitlab.com/api/v4/projects/PROJECT_ID/trigger/pipeline

Clearly replacing TOKEN, and PROJECT_ID with values to be defined. The ENVIRONMENT can be QA4, Alpha, Staging, etc.

To trigger the first Smoke Test group, you can call

curl -X POST \
     -F token=TOKEN \
     -F ref=Staging \
     https://gitlab.com/api/v4/projects/9504048/trigger/pipeline

The result of the call is a JSON that looks like

{
   “id”: 39791847,
   “sha”: “c656bbbbc5844f342a677d28c3b21c1ad3c6e10ad”,
   “ref”: “QA4”,
   …
}

The important element is the “id”. Extract theidfrom the JSON file and use it to get the summary about successes and failures after the test ends, with this general format URL:

https://s3-us-west-2.amazonaws.com/ACCOUNT/projects/PROJECT_NAME/ENVIRONMENT/BUILD_ID/report.json

WhereACCOUNTis the client name,PROJECT_NAMEis the Project Name,ENVIRONMENTcan be for example QA4, Alpha, Staging, etc., andBUILD_IDcomes from above JSON response.

So for example, in the case of the above response, the results URL would be

https://s3-us-west-2.amazonaws.com/ACCOUNT/projects/Scenario_01to10/Staging/39789664/report.json

The result of thecall if the scenario has not yet finished executingis an HTTP 403. Forbidden Response

The result of the call when the scenario has already finished is a valid JSON file that indicates the count and name of successes and failures like:

{
   “nottested”: 1,
   “workingClasses”: [“MyAccont_1_QA4_DeepLinkTest”],
   “failures”: 0,
   “successes”: 1,
   “skippedClasses”: [],
   “failingClasses”: [],
   “project”: “MyAccount_Validations”,
   “buildNumber”: “40791847”,
   “branch”: “QA4”,
   “hash”: “2be843acd5622293087a32496c67d1bb”,
   “skipped”: 0
}

Accessing the HTML Dashboards from AWS S3

Fully detailed results about each project can be accessed using this URL

For blueprints: https:/BUCKET/s3-us-west-2.amazonaws.com//products/PRODUCT/ENVIRONMENT/EXECUTIONTYPE/BUILD_ID/blueprintExecution.html

For scenarios: https:/BUCKET.s3-us-west-2.amazonaws.com/products/PRODUCT/ENVIRONMENT/EXECUTIONTYPE/ELEMENTNAME/BUILD_ID/report.html