Calibration Testing into the Test Plan
Recommended
When guidancedefining the test scenario a set of assumptions is calculated into the Test Plan. For instance, the test scenario in The Test Scenario speculates a test case can achieve satisfactory throughput (in TPS) with a message payload request size of 500,000 bytes and 200 concurrent requests. A Calibration Test identifies a service agent's optimum throughput – measured in TPS at the consumer – against the given testing hardware and software. For instance, consider the data in Payload Size, Concurrent Agents, and Transactions -Per-Second Results.
Add
calibration
testing
Payload Size, Concurrent Agents, and Transactions-Per-Second Results list the two input values for the test: the message size sent to the performanceservice, and the number of concurrent test planagents. beforeUsing runningthese largevalues, scalabilityXSTest tests.operates Calibrationa confirmstest thatcase by instantiating one thread for each concurrent request. Each thread dynamically generates the script,defined data,payload environment,size load model, monitoring,data and resultsends collectionit are ready for higher-volume execution.
Calibration goals
Validate thatto thescenarioserviceworks reliably withas asmallrequest.userThencount.Confirmthetestthreaddata,receivesDPLs,theaccounts,server's response, validates the response, handles any exceptions, andenvironmentlogsaccess are sufficient.Measure baselinethe responsetimes,aserrors,athroughput,completedandtransaction.resourceTheuse.Identifythreadbottlenecksrepeatsinthe same steps until the testscript,caseTestperiodNodes,is finished. A Bar Chart of the Results Showing the Maximum Throughput Values displays the results from Payload Size, Concurrent Agents, and Transactions -Per-Second Results in a bar chart clearly shows the maximum throughput values.A Bar Chart of the Results Showing the Maximum Throughput Values
Payload Size, Concurrent Agents, and Transactions -Per-Second Results and Scalability Index provide information about the service under test, including:
-
As payload increases, TPS reduces proportionately. The test is not saturating or under-utilizing the server, network, or
applicationconsumer.underIftest.TPS increases, the testing level was not high enough, or if it was flat or dropping sharply, the testing level was not low enough.
The logs,reduction reports,in TPS is not proportional to the increase in request size. When the network and dashboardsconsumer captureare not at high enough activity levels, the requiredservice evidence.
Planning checklist
Runhas asingle-userpoor-performingvalidationrequestbeforeprocessor.addingOneconcurrency.reason this could happen is if the message parsing system is not allocating resources (memory, network socket connections, or message queues) of the correct size for the demands of the test.
TPS takes a significantly larger reduction for test cases above 3000 bytes of payload. In this case, a code profiler is used to find a test experiencing a buffer overflow or an undersized object list.
While A Bar Chart of the Results Showing the Maximum Throughput Values identifies a few things about the service, additional information is needed to conclude. Values, listed in controlledParameter stepsValues Required to Calibrate a Test, are needed for the test parameters to conclude.
Parameter Values Required to Calibrate a Test
In a stateless system, for each request, a service allocates its memory, CPU bandwidth, network bandwidth, and reviewother errorsresources atneeded to generate a response. For a stateless calibration test, resource bottlenecks are identified. Network and CPU Utilization shows the test scenario results including network utilization and server and consumer CPU utilization values.
Network and CPU Utilization
The results in Network and CPU Utilization give some idea of what is going on during the test scenario:
The test is server-bound preventing greater throughput (TPS). When payload sizes are less than 4000 bytes, server CPU utilization is high but not saturated. At 4000 bytes and greater, the CPU is saturated
Stateless tests require resources to handle the concurrent requests load. Take away a resource – CPU bandwidth or free memory – to operate on larger payloads and response times increase lowering overall TPS
The scale of the problem indicates there is a significant problem with the server. The payload size from 1000 to 5000 increases by a factor of 5, but TPS values decrease by a factor of 14, from 10.376 to 0.73In a stateless test, the TPS value should be proportional to the input
Since this is a stateless test, each level.
However, this is not the case for stateful services such as database and workflow applications. stateful services use data caches, and server queues and typically have session managers overhead. These items impact service CPU and memory utilization levels independent of the consumer request load.
For a scalability and performance test running in a defined software and hardware environment, a calibration resultstest helps determine the appropriate service concurrent request levels and usemessage thempayload tosizes. refineThe results show a Scalability Index for the finalservice.
Last updated: OctoberMarch 5,18, 20262024
If you have any feedback on this topic, please contact us at documentation@appvance.ai