To run a load test, LoadView utilizes Load Injector (LI) servers from Amazon Web Services (AWS) and Azure Cloud Services. Each load test uses a certain number of LIs depending on the number of virtual users we need to generate for a test. To set the number of virtual users simulated per LI, we use the Payload setting. It is important to understand that the payload value can affect the correctness of the test results and test cost. For example, a large number of virtual users simulated per LI leads to LI server overload and invalid test results. Alternatively, a small payload value will require more LIs to generate the necessary user load for the load test and increase the test cost.
To calculate an optimal payload value for the number of virtual users simulated per LI, we perform calibration of a load test device. The device complexity and required CPU resources of LI servers are considered during calibration. Calibrated Payload ensures LI server CPU load no more than the optimal 60-80 percent, and provides the maximum load distribution among the LI Servers.
Calibrated payload varies based on a task type:
- HTTP(S) Test: from 500 to 1,000 users.
- Web Page/Streaming Media/Web Application Test: from 8 to 25 users.
- Postman Collection Test: from 20 to 100 users.
To calibrate your load test device, on the Test Scenario page click Calibrate in the Load Injector Payload section.
LoadView runs a free mini-test on a dedicated load injector server to execute the calibration. The calibration duration and the number of virtual users per LI server is adjusted automatically during the test to achieve an average LI server CPU load of 70 percent. LoadView shows the calibration load curve and the corresponding CPU usage chart in the Calibration window in a real-time mode. If the target 70 percent CPU usage was not achieved during the calibration test run, the average payload will be calculated based on the received mini-test results.
The initial and maximum number of virtual users to run the calibration depends on the test type:
- HTTP(S) Test: from 50 to 100 virtual users.
- Web Page/Streaming Media/Web Application Test: from 4 to 10 users.
- Postman Collection Test: from 10 to 50 users.
Once the number of virtual users that allows to achieve 70 percent CPU load is calculated, it is provided in the Calibration window. To apply the value to the scenario, click Apply Recommended Value in the Calibration window.
User Delay Impact on Calibration Results
During calibration, LoadView calculates how much load the simulation of one virtual user puts on a LI server. If there are no user delays set, a LI constantly runs a test session in a loop without delays between the test sessions, thereby increasing its CPU usage. For example, CPU usage can reach 100 percent while executing a rapid API call.
On the other hand, if a user delay is set in the User Behavior profile, a LI server will be idle during the delay time period. In this case, CPU utilization will be lower than for the same test execution without user delays. Thus, more virtual users can be generated by a LI without an increase in CPU load.
Considering this, the longer the user delay time is set for a test, the higher payload value you will receive as a result of a device calibration.