HP LoadRunner Best Practices
HP LoadRunner Best Practices
HP LoadRunner Best Practices
and execution
White paper
Table of contents
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
About the author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Implementing a successful performance testing program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Formulating a high-level test plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Tips for scripting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Defining the directory structure and naming conventions ...................................5
Recording scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Creating dynamic scripts through correlation and parameterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Creating transactions ...........................................................5
Parameterizing and converting itoa (C++) code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Date_Time manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Get date seven days beyond current date . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Polling for batch run status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Logging messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Scripting checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Tips for building scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Executing tests via HP LoadRunner Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Scheduling options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Tips for data handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Using virtual tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
IP spoofing for client with same Netmask IP class and network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Parse string based on value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Randomize using switch and case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Tips for test execution and scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
HP LoadRunner timeout errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Extended logging code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Abstract: This guide provides tips and tricks for Headquartered in Houston, Texas, Waste Management
HP LoadRunner software configuration, scripting, protects the quality of life in many communities,
and execution. It is a conglomerate of lessons learned providing waste solutions to approximately 25 million
by HP LoadRunner power user Opral Wisham, residential and 2 million commercial customers in
including unique code as well as code collected from North America.
other testers. This guide is intended to help testers just
Opral’s responsibilities at Waste Management include
learning to use HP LoadRunner, as well as to provide
managing the capacity and performance verification
new best practices for those who have used HP
(CPV) processes for all of the company’s new
LoadRunner for many years.
applications. In her role as CPV technical manager,
About the author: Opral Wisham is currently a senior Opral has gathered many best practices to coordinate
performance and capacity specialist at Waste the testing activities of the company’s cross-departmental
Management, Inc., the industry’s leading provider CPV team and expedite the training of new team
of comprehensive waste management services. members.
Table 1. This table illustrates the CPV request Preparation Development and execution Analysis
functional flow of a performance
testing execution process, with Description Meet with CPV lead to Secure hardware Create automated scripts Analyze data
suggested deliverables for each overview solution under test provided by CPV for
Configure hardware/software Debug automated scripts
phase. each test cycle
Conduct a planning session
Data explosion Run pre-determined load
to outline requirements and Identify unacceptable
for each test cycle
resolve issues Create IDs performance
Provide results after each
Reserve test slot with CPV Business scenarios and Ensure accurate
test cycle
test cases hardware capacity
Provide summary report
Owner Business owner Business owner CPV Business owner CPV
administrator(s)
Deliverables Architecture review Performance requirements Summary report Position statement
board approval document (final) outlining tuning and recommendations
changes, CPV issues,
Performance requirements Support staff available
statistical data
document (draft) Test environment configured
Support staff established and data validated
Obtain automation tool,
if not available
3
Formulating a high-level test plan 1. List all testing milestones and deliverables for every
The performance testing team should begin by phase of the project, including the pilot, code
defining a high-level test plan that describes the freeze and production phases.
timeline for all testing efforts, including what types of 2. Create a comprehensive production physical
tests will be performed (e.g., online and infrastructure architecture diagram that shows a very low level
stress and load tests, batch performance tests, etc.). The of detail, including how each component will
plan should also indicate how the performance testing be connected.
team will interact with the development, deployment
3. List all hardware and software requirements for
and/or support groups within the enterprise. When
the tests.
possible, it helps to use the actual names and titles
of resources when explaining the interaction. 4. Determine all database data volume requirements.
5. Define all performance objectives.
The testing team should describe what the tests are
intended to measure and/or report. It is important to 6. Identify all performance benchmarks, if available.
clarify all terminology that may be at risk of being 7. Identify the total number of expected users.
misunderstood (e.g., “Transaction’s Response Time”
8. Specify the minimum and maximum number of
may refer to the time it takes to hit the “enter” button
concurrent users.
and get a result from the system, or it could indicate
the time it takes to perform a function that requires 9. Select five business scenarios (manual scripts),
some human intervention). Some metrics for web plus actions within each scenario that should be
applications will have different definitions from measured (transactions).
traditional online and batch applications, like average 10. Determine the user load mix.
hits/second, web server throughput, etc. 11. Define acceptable average response times per
The following checklist can be used when defining the transaction.
steps of a comprehensive performance testing process: 12. Define the average hits/second or number of
transactions within one hour.
13. Define throughput, if applicable.
14. Identify any concurrent batch processes that
are running.
15. Identify all lead, technical and functional
contact personnel.
4
Tips for scripting Recording scripts
Whenever possible, the performance testing team
Defining the directory structure and should use HP LoadRunner’s automatic recording
options. This method provides the most efficient way
naming conventions to record scripts for load testing most web-based
It is important to establish formal procedures for applications.
creating directory structures and establishing naming
conventions. The following nomenclature can be used Creating dynamic scripts through
for creating a directory structure (folder names) for
new projects:
correlation and parameterization
The performance testing team can use correlation to
retrieve a value before it is needed in a preceding
ProjectName
statement, enabling the server to dynamically generate
¬ LR_Scripts
the unknown value. The team can use parameterization
¬ Scenarios
when the values are known and they are not available
¬ TestDoc
prior to executing the statement.
¬ TestPlanAndReport
¬ TestExecution Below is a parameterization example:
¬ Recommendations 1. Parameterize, Source, Currency, Business Unit and
¬ Reports Account, using a file. Select next row should be
¬ LR_Reports “same as Business Unit.” Business Unit Select next
¬ Cycle1 row should be “sequential” or “random,” depending
¬ Cycle2 on the duration or number of iterations.
¬ HTML_Reports
2. Parameterize date as “date/time,” using
¬ Cycle1
%m/%d/%Y
¬ Cycle2
3. Parameterize Amount as a random number from
1 to 10,000.
Project and script names should be descriptive and
intuitive for new users of the system. For example: 4. Parameterize Accounting Period as “date/time,”
using %m
5. Parameterize the URL using a file. This will reduce
ProjectName: Should identify the project or application under test. the script changes required when the URL does
LR_Scripts: Files should be the name of the saved HP LoadRunner change.
scripts.
FunctionName_Type Correlation and parameterization are keys to creating
Type: Is func or rpt or qry a dynamic, data-driven test.
Scenarios: Files should identify the project and number of Vusers
simulated.
Creating transactions
ProjectName_VuserCount
The performance test team should then define the
TestDoc: Directory should contain all test documentation.
transactions to measure the performance of the server.
TestPlanAndReport
Each transaction measures the time it takes for the
Test Plan
server to respond to specified Virtual User (Vuser)
Test Cases
requests.
WorkLoad, etc.
Scripter Notes To mark the start of a transaction, the testing team
Test Summary should click the “start transaction” button from the
TestExecution recording toolbar. Enter a transaction name in the
Execution Progress Log transaction name box. Click “OK” to accept the
Comparison Log transaction name. HP LoadRunner Virtual User
Recommendations Generator (VuGen) will automatically enter a
Recommendations lr_start_transaction statement into the script. Click the
“end transaction” button to end the transaction. VuGen
will automatically enter an lr_end_transaction
statement into the script.
5
Performance testing typically creates transactions for Date_Time manipulation
the Log-in and Log-out and every place in the script
The following statements can be inserted to manipulate
where a “submit” appears. More transactions can be
the date time stamp. In this example, it was necessary
included, but all submits must have transactions to
for the date in the script to be one day after the current
measure the response times.
date. The parameter {Tomorrow} is saved where the
Dynamic transactions names are names that change current date appeared in the script. This parameter
depending upon a parameterized value. If there are insertion results in tomorrow’s date being inserted
several reports that are run randomly, the team can where the current date would have appeared.
define the transaction name dynamically using the
code below:
Action1()
{
Char rname[10];
sprintf( rname, lr_save_datetime(“Tomorrow is %m/%d/%Y”, DATE_NOW +
“Run_Report_%s”, ONE_DAY, “Tomorrow “);
lr_output_message(rname);
lr_start_transaction(rname); itemdata.....
lr_end_transaction(rname,2); *
*
*
“Name=PV_REQ_SR_WRK_DUE_DT$0”, “Value={Tomorrow }”,
Parameterizing and converting itoa ENDITEM,
(C++) code
If the team needs to increment a counter and use that
counter as a parameter, it will need to convert the Get date seven days beyond current
integer to a string by using sprintf. After conversion, date
the string needs to become a parameter. Therefore the
This code will add seven days to the current date. This
string should be saved into a parameter using
is useful when a due date is required seven days from
lr_save_string.
the current date:
6
Polling for batch run status “Name=PMN_FILTER_WS_OPRID”, “Value=CPVID”,
ENDITEM,
This example was used in a script that required clicking “Name=PMN_FILTER_PRCSTYPE”, “Value=”, ENDITEM,
the “refresh” button until the run status changed to “Name=PMN_FILTER_FILTERVALUE”, “Value=1”,
ENDITEM,
“success.” This code provides an automatic refresh
“Name=PMN_FILTER_FILTERUNIT”, “Value=1”, ENDITEM,
until the batch job has been completed. The next step
“Name=PMN_FILTER_SERVERNAME”, “Value=PSUNX”,
requires the completion of the batch job.
ENDITEM,
“Name=PMN_FILTER_PRCSNAME”, “Value=”, ENDITEM,
“Name=PMN_DERIVED_PRCSINSTANCE”,
int x; // flag will be 0 or 9
“Value={Process_Instance}”, ENDITEM,
char *temp, *temp2; // values to hold strings
“Name=PMN_DERIVED_TO_PRCSINSTANCE”,
Action1() “Value={Process_Instance}”, ENDITEM,
{ “Name=PMN_FILTER_RUNSTATUS”, “Value=”,
temp2=”Success”; //compare string 2 ENDITEM, LAST);
“RB=\n”, }
“Search=body”,
LAST);
web_submit_data(“PROCESSMONITOR.PROCESS Logging messages
MONITOR.GBL”,
“Action=http://crpu028a:8050/psc/fs84cpv/EMPLOYEE/ERP/c/
Vuser messages are sent to the Vuser log and output
PROCESSMONITOR.PROCESSMONITOR.GBL”, window by using one of the message functions. The
“Method=POST”, lr_output_message is most commonly used by CPV. Log
“RecContentType=text/html”, messages can be used to identify many things about
Vusers during script execution, including:
“Referer=http://crpu028a:8050/psc/fs84cpv/EMPLOYEE/ERP/c/
PROCESSMONITOR.PROCESSMONITOR.GBL? 1. Determining if a Vuser reaches a certain point
Page=PMN_PRCSLIST&Action=U&”, during script execution.
“Snapshot=t17.inf”, 2. Confirming that valid values are passed for
“Mode=NORESOURCE”, parameters.
ITEMDATA,
3. Confirming that passed values appear where they
“Name=ICType”, “Value=Panel”, ENDITEM,
should in the database (by capturing the passed
“Name=ICElementNum”, “Value=0”, ENDITEM,
values and knowing what to search for).
“Name=ICStateNum”, “Value={ICStateNum6}”,
ENDITEM,
“Name=ICAction”, “Value=REFRESH_BTN”, ENDITEM,
Scripting checklist
“Name=ICXPos”, “Value=0”, ENDITEM, HP LoadRunner script testing will occur after creation
“Name=ICYPos”, “Value=0”, ENDITEM, of each automated script. Scripts can be added to
“Name=ICFocus”, “Value=”, ENDITEM, performance test bed when:
“Name=ICChanged”, “Value=-1”, ENDITEM,
1. No scripting errors occur.
“Name=ICFind”, “Value=”, ENDITEM,
2. Looping successfully; replaces parameterization
variables with various values.
3. Activity is noted on each system tier.
4. A minimum load of ten users is simulated.
5. Scripts are run on a subsequent day (dates
correlated correctly).
7
Tips for building 5. Simulate a new user for each iteration: Instructs
8
Figure 1. Setting in “Network
Neighborhood” (On the Load
Generator Machine).
Regsvr32 “C:\WINNT\system32\vtsctls.ocx”
Make sure that “C:\WINNT\system32\COMCTL32.DLL” exists.
Install the virtual table on the controller and the machine that will be
Also, verify that all of the new IP addresses are on the list
used as the Virtual Table Server. Copy vtclient.dll and vtserver.dll to in “Network Neighborhood”/”Properties”/Protocol/
the lr/bin: folder, then locate the “include” folder (lr/include:) and Advanced.
insert vts2.h (all found in the product folder on the hard drive).
Verify that Wins address and DNS are completed with
the correct IP addresses.
IP spoofing for client with same Note:
Netmask IP class and network The IP SPOOFING ADD ROUTE BAT file needs to be
run on the target server in order for IP SPOOFING to
IP spoofing is used when a unique user IP address is
work.
required to identify unique data.
In Script:
CODE TO VERIFY IP SPOOF: lr_message (“ IP ADDRESS _ %s “,
lr_get_vuser_ip());
Adding IP Addresses:
GO TO IP WIZARD …..FOLLOW INSTRUCTIONS AND ADD IP
9
Parse string based on value Randomize using switch and case
Use the code below to parse a string and use the data Use the switch command to vary transactions. If you
in subsequent steps. The string “Wisham333445555” are testing multiple reports with significantly different
can be separated into multiple variables such as last web URLs but all other steps are the same, the user
name and ssn. can use the switch statement.
10
Tips for test execution
5=”T_EDIT_NUM;Web;ConnectTimeout;Connect timeout
(seconds);The maximum amount of time in seconds, that a Vuser
waits to connect to a server, before issuing an
error.;1;120;<0;10000>”
and scheduling 6=”T_EDIT_NUM;Web;ReceiveTimeout;Receive timeout
(seconds);The maximum amount of time in seconds, that a Vuser
waits to receive information from a server, before issuing an
HP LoadRunner timeout errors error.;1;120;<0;10000>”
Use the following steps to prevent HP LoadRunner 7=”T_EDIT_NUM;Web;NetBufSize;Network buffer size; Sets the
maximum number of the network buffer that may be used by the
timeout errors if you need less than 1,000 seconds Vuser. The default is 12288 bytes.;1;12288;<0;100000>”
for a function such as save data to complete: 8=”T_COMBOBOX;Web;CacheAlwaysCheckForNewerPages;Cach
e-Emulate always check for newer pages;;1;Yes;<Yes;No>”
1. From the Controller go to Run-time Settings –>
9=”T_EDIT_NUM;Web;PageDownloadTimeout;Page download
Browser Emulations tab –> Advanced timeout(seconds);;1;120;<0;32000>”
[web]
web_reg_save_param(“EMPLID”,
ReceiveTimeout=10000
“LB/IC=’PSEDITBOX_DISPONLY’ >”,
“RB/IC=<”,
2. Modify LoadRunner\dat\WebAdvancedOpt.txt. “Ord=4”,
Change: “Search=body”,
LAST);
// TURN FULL TRACE OF
[LIST]
lr_set_debug_message(LR_MSG_CLASS_EXTENDED_LOG |
1=”T_COMBOBOX;Web;EnableIPCache;DNS caching;Save a LR_MSG_CLASS_FULL_TRACE |
host’s IP address to a cache to conserve time in subsequent calls to
LR_MSG_CLASS_RESULT_DATA | LR_MSG_CLASS_PARAMETERS,
the host.;1;Yes;<Yes;No>”
LR_MSG_OFF);
2=”T_COMBOBOX;Web;HTTPVer;HTTP version;Indicates the version
of HTTP used by your application. Select version 1.0 or
1.1.;1;1.1;<1.0;1.1>”
3=”T_COMBOBOX;Web;KeepAlive;Keep-Alive HTTP
connections;Allow persistent HTTP connections to enable multiple
requests to be sent over the same TCP connection.;1;Yes;<Yes;No>”
4=”T_EDIT_NUM;Web;MaxConnections;Concurent connections
limit;Sets the maximum number of simultaneous keep-alive
connections that a Vuser can maintain during script
execution;1;4;<0;10>”
11
For more information on HP LoadRunner or other HP products or
services, visit: www.hp.com/go/software
© Copyright 2007 Hewlett-Packard Development Company, L.P. The information contained herein is subject to
change without notice. The only warranties for HP products and services are set forth in the express warranty
statements accompanying such products and services. Nothing herein should be construed as constituting an
additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
4AA1-2107ENW, May 2007