eng_

Test Orchestration

Concepts

Appvance designed Appvance IQ around several testing concepts: Test orchestration, repurposing, record/playback, Repository support, script-driven testing, data-driven testing, and cloud and grid testing.

Test Orchestration

Many organizations need a reliable way to deliver Internet-based applications. They provide software development services and hire testers to build and operate tests. It is common to see the following services in place:

  • Repository – to store and version control software assets, including source code, data, and tests.

  • Bug Tracker – to manage changes to the application software

  • Test Management – to manage requirements, identify test coverage, manage test efforts, and provide management with business risk assessment inputs

  • Continuous Integration (CI) – to automate software builds and testing as developers commit changes to the repository

  • Monitoring – to close the loop by identifying an operating software applications compliance with its original requirements and operation

With these services in place, the organization adds the most appropriate test authoring tool to the project. For example, soapUI works well for building test suites of SOAP and REST-based Web services, Sahi for building automated test scripts for Rich Internet Applications (RIA, using Ajax, Flex, Flash), and JUnit tests written in Java for building reusable software test objects of Java and BPM applications.

It does not take long for the organization to move from manual testing to automated testing, to move on to data-driven automated testing, and repurpose their tests to be load and performance tests and production monitors. Organizations accomplish test automation by adopting a test orchestration platform

A Test Orchestration solution delivers the following tasks:

  • Test Operation Console – provides automation to the repository to store and version control test assets, including test scripts, operational test data, and test use case definitions. The Appvance Designer component provides record/playback capability to build test objects for Web, Ajax, and Flex-based applications. The Console is a test runtime that deploys tests to one or more Test Injectors. The Console retrieves the test results and identifies the root cause of performance bottlenecks and functional issues. The Console delivers actionable knowledge in a standardized set of results analysis reports.

  • Test Injector – is a distributed test runner environment. The Test Injector provides test ScriptRunners to operate the tests in their native format, including Selenium, soapUI, Sahi, and unit tests written in Java, Ruby, Groovy, .NET, C#, and command-line shell scripts. The Test Injector Data Production Library (DPL) injects operational test data from relational databases, comma-separated values (CSV,) and data generation objects into tests. Appvance IQ calls a test injector a TestNode.

SoapUI support was removed in release 5.2.0 of AIQ.

  • Appvance Monitor - is an agent-based resource monitor. It observes, at a minimum, CPU, Network, and Memory usage for the server running the application under test (AUT) while the TestNode operates the test against the front-end of the app. Appvance Monitor logs the monitored values to the Repository. Results Analysis correlates the monitored values to the test operation to forecast server bandwidth to forecasted user levels.

The Console and Injector environment is lightweight enough to run on developer desktop development machines, test lab environments, virtual environments, cloud environments, and geographically distributed test environments. Test orchestration components provide test automation to the complete software development lifecycle: Test Authoring, Test Orchestration, and Test Management.