Jmeter HTTP Performance Testing Process

By

Sarat Addanki, Senior Vice President, Testing Solutions

Contact:
Suresh Sakamuri Head of Performance Testing &
Monitoring
[email protected]
+91-9966204823
Performance Testing Requirements Questionnaire
4800 Great America Pkwy,
Document
Suite 425,
Santa Clara,
(Proprietary andCA, 95054,
Confidential)
(408) 687-4595, Fax: (408)653-2030.
Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 2 of 31
Introduction: -

Apache JMeter is a 100% pure Java desktop application designed to load test client/server software (such as
a web application). JMeter can be used to simulate a heavy load on a server, network or object to test its
strength or to analyze overall performance under different load types. Additionally, Jmeter can helps you
regression test your application by create test scripts with assertions (content check points) to validate that
your application is returning the results you expect

Pre-requisites: -

Install latest version of JAVA/JRE 1.5 or later followed by set the JAVA bin and JAVA_HOME path as shown
below

 My ComputersRight ClickPropertiesAdvance TabEnvironment Variables

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 3 of 31
Getting started: -

1) Download the zipped binary(Apache JMeter 2.7) from http://jakarta.apache.org/jmeter/

2) Extract the ZIP files to any specified location in the local host

3) On Windows, launch JMeter by running jmeter.bat

4) After running jmeter.bat, Jmeter console will open up by default Test Plan, Workbench nodes shown
up as seen in the below snapshot

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 4 of 31
Note: -

TestPlan Node  Permanent workspace for created scripts (Refer Terminology section for Test Plan)

Workbench  Temperary work space for creating scripts (Refer Terminology section for Workbench)

Jmeter supports the blow list of protocols for performance testing of any web/client server applications

 HTTP/HTTPS, FTP, JDBC, JMS, LDAP, SOAP, RMI

This whitepaper will explains the HTTP/HTTPS performance testing process for “Google Web Search”
Application with effective utilization of Jmeter features/add-ons facility

What is HTTP/HTTPS Protocol: -

HTTP Protocol stands for Hyper Text Transfer Protocol. It is the protocol used to convey information of
World Wide Web (WWW). HTTP protocol is a stateless and connectionless protocol. HTTP/HTTPS protocol is
based on a request/response paradigm. In this protocol the communication generally takes place over a
TCP/IP protocol

Jmeters Terminology: -

Before we go further it would probably be helpful to discuss some basic Jmeter terminology. You may also
want to refer to the Jmeter GUI for more information

 Thread: For all intents and purposes, a thread can be thought of as a user (virtual user in LR)

 Thread Group: Think of a thread group as the scenario. A Business Scenario that has a user log in to
Web Tours application, book a flight, and logout might be recorded in a test script called one thread
group. During a test there might be one or several hundred users executing a thread group. The
thread group might contain conditional logic that loops through parts of the web application

 Test plan: A test plan is one or more thread groups. In the above, we discussed a thread group that
performing some actions as part of “Booking a Flight” scenario. Like this if we have multiple
Business Scenarios can also be called as Test Plan

 Sampler: A sampler is something that sends a request to a server. There are many types of samplers
for all the protocol. One can Crete custom request for specific protocol (web_custom_request)

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 5 of 31
  Listener: A listener listens to the responses generated by samplers. Listeners can be used to
compile the results of the test run and to check that a sampler generates the expected response

 Assertion: An Assertion is used to specify what a response should or should not contain. There are
several different types of assertions depending on what kind of response you are expecting and
what sort of check you'd like to perform (web_reg_find in LR)

 Logic Controller: Logic controllers provide a mechanism for controlling the flow of the thread group.
Adding a logic controller to your thread group is like adding if-then or do-while logic to a program.
Your test script can make run-time decisions about how to execute. Suppose you want to open
every object in a list but you do not know how many objects will be in the list ahead of time. A
"Loop Controller" could be used to execute an action (opening an object) a certain number of times
without requiring you to hard-code the requests into the script

 Workbench: The Workbench is like a working area. When recording scripts, JMeter places the
automatically-generated samplers under the workbench. They will not get executed as part of a
thread group until you move them from the workbench to a thread group (Virtual User Generator)

 Transaction Controller: The Transaction Controller generates an additional sample which measures
the overall time taken to perform the nested test elements. Note that this time by default includes
all processing within the controller scope, not just the samples

JMeter Test Plan Process:

There are several steps for achieving end-to-end web performance testing using HTTP Sampler

Test Plan Creation: -

1) Add a Thread Group to Test Plan

(Right click on Test Plan and select Add Threads (Users)  Thread Group)

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 6 of 31
2) Add HTTP Proxy server to the Workbench

(Right click on WorkBench and select Add Non-Test Elements HTTP Proxy Server)

3) HTTP Proxy Server node is added under WorkBench node

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 7 of 31
4) Open the internet browser and configure the proxy setting as shown in below snapshot. Make sure that

the port number should be same as the port number given in the Global Settings in Jmeter.

ToolsInternet OptionsConnectionsLAN Settings

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 8 of 31
5) Select “Put each group in new transaction controller” as Grouping option. Therefore every single click

can be recorded as one business transaction (lr_start_transaction in LR)

6) Select the HTTP Proxy Server node under WorkBench

Select HttpClient4 as Type in HTTP Sampler Settings (Recording Options in LR)

Select TestPlan > Thread Group as Target Controller in Test plan content (lr_start_transaction in LR)

Click on Start button

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 9 of 31
7) Now start recording by opening the browser and enter the URL http://www.google.com which will

access the Google Home page

8) Enter any search string in the text box and click on Google Search

9) Click on first link

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 10 of 31
10) Select the HTTP Proxy Server node under WorkBench and click on Stop button

Till the above step, Recording process is completed.

11) All the above business scenario can be recorded in jmeter as shown in the below snapshot, where all

the Transaction Controller names can be renamed in the following way:

/  01_Google_Homepage

/search  02_Google_Search and

/url  03_Google_ClickfirstLink

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 11 of 31
Correlation:
Correlation is the most important aspect of scripting. It generally includes fetching dynamic data from
preceding requests/calls and posting it to the subsequent requests

1) Correlate the URL (http://jmeter.apache.org/) – URL(dynamic content) will be different for every
iteration based on Search string

2) Copy the URL(Dynamic Content)and go to Response tab (Text mode) in “View Result Tree” listener of
search operation. In order to see the Response Data in “View Results Tree” listerner, we need to
execute script by at least one time followed by adding “View Results Tree” listener

Right click on Thread Group Listerners View Results Tree

Note:- In LR, there is a “Tree View” mode to see the server response where as in Jmeter, Thread Group
(script) should be executed at least once to see the server response

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 12 of 31
3) Dynamic Content from Response tab is <a href="/url?q=http://jmeter.apache.org/&

Capture the Dynamic value that is to be correlated along with its left and right boundaries. In this
example

Dynamic Value: - http://jmeter.apache.org/

Left bound: - <a href="/url\?q=

Right bound: - &

Replace the dynamic value with a regular expression i.e., (.*?) by adding Regular Expression Extractor as
explained in the next step

RegularExpression: <a href="/url\?q=(.*?)&

The regular expression is used to parse the response data. This must contain at least one set of parentheses
"()" to capture a portion of the string

4) Go to http request under “02_Google_Search” transaction controller->

Page 13 of 31
 Add -> Post Processor -> Regular Expression Extractor

5) Select the Regular Expression Extractor and enter the details such as Reference name, Regular
Expression, Template, Match No. as shown in the below snapshot.

Reference Name: It is the name of the JMeter variable which stores the dynamic value.

Match No. : Indicates which match to use. The regular expression may match multiple times.
 Use a value of zero to indicate JMeter should choose a match at random.
 A positive number N means to select the nth match.
 Negative numbers are used in conjunction with the ForEach controller
Default Value: If the regular expression does not match, then the reference variable will be set to the
default value. This is particularly useful for debugging tests
Template: attribute should be default as “$1$”
A small example to understand depth of Correlation : (Impact of Template Attribute)
Some times we want to capture multiple strings which are changing dynamically in the same LINE OF CODE
like;

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 14 of 31
<name=jojo&amp;age=25&amp;height=5.7&end> (RED colored indicates the dynamica values which
need to be captured in the same Reference Name)
RegularExpression Declaration is : <name=(.*?)&amp;age=(.*?)&amp;height=(.*?)&end>

Dynamic values are stored in the variable names as shown below

Parameterization:

Parameterize the search string “Jmeter”

1) Add CSV Data set Config element to the thread group.

AddConfig ElementCSV Data Set Config

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 15 of 31
 2) Open a notepad (inside bin) and enter your Search Strings. Save the file as search.csv. The extension

csv makes it a comma separated Value (CSV) file.

3) Select the CSV Data set Config element and fill in the fields as follows

Filename: search.csv

Variable Name: Search String

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 16 of 31
 4) Go to 02_GoogleSearch transaction and change the value of variable q and oq to ${keyword}. For

each iteration the value of q and oq will be replaced as fetched by variable Search String

Assertions: (web_reg_find in LR)

Add an assertion for the HomePage page.

1) Right-click the sampler named " / " then click AddAssertionResponse Assertion

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 17 of 31
 2) Add " Advanced search " as a check point in the Pattern to Text field

2. Now enter the 1. Click on add button

3) To know when an assertion fails add an Assertion Listener (Error Handling)

( Right-click on Thread Group AddListener Assertion Results)

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 18 of 31
 4) Run your script. If any of the assertions fail, a message will show up in the Assertion Results listener
as shown in below snapshot (lr_output_message & lr_error_message in LR)

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 19 of 31
Cookie Manager:

Many Web applications use cookies. JMeter provides cookie capabilities through a Cookie Manager. Adding
this element to a thread group allows us to send cookies to the application being tested, just as Web
browsers do.

Add HTTP Cookie Manager(Right click the Thread Group and select Add Config ElementHTTP Cookie Manager)
and set Cookie policy of the cookie manager to be “compatibility”. Remember also to set the "Clear cookies
each iteration" setting to checked

Timers: (lr_think_time in LR)

By default jmeter thread sends request without pausing between each reques.It is recomended to set delay
by adding timers to the thread group. Timers are used to create delays between requests made by each
simulated user. Without a timer, your requests hit the server back to back, as fast as possible. Timers allow
for a more realistic load on the server. There are different types of timers that allow you to produce either
constant or random delays.If you want to have each thread pause for the same amount of time between
requests, use Constant Timer

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 20 of 31
Give the constant delay as shown in below snapshot

User Defined Variables:

User Defined Variable element is used to initialize set of variables.

Add User Defined Variables element under Thread Group

(RightClick on Thread Group AddConfig ElementUser Defined Variables)

In our example we can define the Server Name as User Defined Variable.

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 21 of 31
Go to User Defined Variables element and fill the fields as shown in the below snapshot

Replace www.google.co.in with ${ServerName} in HTTP Request as show in the below snapshot

After above enhancements, execute the script for multiple iteration to validate the funcitonlity of the
application

1) Add View Results Tree Listener element to the thread group

Add->Listener->View Results Tree

2) Run the script(ctr+R) and observe the result in result tree

Results Output:- Every iteration it will fetch different search string followed by accessing Their respected
search string First Link only

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 22 of 31
Thread Group:

A Thread Group defines a pool of users that will execute a particular test case against your server. In the
Thread Group GUI, you can control the number of threads (number of users), the ramp ramp-up period
(how long it takes to start all the threads), The loop count (how many times the thread group will be
executed), and optionally, a start and stop time for the test.

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 23 of 31
SetUp Thread Group:

A special type of thread group that can be utilized to perform Pre-Test Actions. The behavior of these
threads is exactly like a normal Thread Group element. The difference is that these types of threads execute
before the test proceeds to the executing of regular Thread Groups.

Example: - Consider two thread groups of Web tours scenario i.e. Flight creation and Flight deletion.
Therefore, we need to execute the test plan with these two thread groups. The first thread group is Flight
creation, where the user reserves the flights. The second thread group is Flight deletion, where the user
cancels all the reserved flights. In this case, the thread group Flight deletion must have always a specific
number of reserved flights to be cancelled. So to schedule a test plan with these two thread groups, we
have to make the Flight creation thread group as “Setup Thread Group”, where a specific number of flights
will be reserved by the user and the thread group Flight deletion can be made as actual thread group.
Therefore, when the test plan is scheduled Flight creation thread group will execute followed by Flight
Deletion thread group will be executed where the Created Flights will be DELETED.

Teardown Thread Group:

A special type of Thread Group that can be utilized to perform Post-Test Actions. The behavior of these
threads is exactly like a normal Thread Group element. The difference is that these types of threads execute
after the test has finished executing its regular Thread Groups. In other words we can call it as vice versa of
setup thread group.

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 24 of 31
Remote Load Testing: -

Step 1: Configure the nodes

 Same version of Java installed on all slave machines for best recommendation
 Configure the CSV param files in the appropriate directory on Jmeter Master Machine

 Configure the number of Load Generators machine IP’s in the properties.txt to simulate the Threads
activity during the test. These properties will be picked up when the server is started and may be used
in the test plan to affect its behavior (e.g. connecting to a different remote server)

Step 2: Start the servers

To run JMeter in remote node, start the JMeter server component on all machines you wish to run on by
running the JMETER_HOME/bin/jmeter-server.bat (windows) script.

Step 3: Add the server IP to your client's Properties File

Edit the properties file on the controlling JMeter machine. In /bin/jmeter.properties, find the property
named, "remote hosts", and add the value of running Jmeter slave server's IP address. Multiple such salve
servers can be added, comma-delimited. If we define the JMeter property server.exitaftertest=true, then
the server will exit after it runs a single test.

Step 4: Start the JMeter Client from a GUI client

Now you are ready to start the controlling JMeter client. For MS-Windows, start the client with the script
"bin/jmeter.bat". You will notice that the Run menu contains two new sub-menus: "Remote Start" and
"Remote Stop" (see below figure). These menus contain the client that you set in the properties file. Use
the remote start and stop instead of the normal J Meter start and stop menu items

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 25 of 31
 Figure 1 - Run Menu

Add On to Jmeter:

Jmeter has several add-ons which will be providing the options as compare with Licensed performance test
tools.

UniversalServerAgent : - Windows and Unix servers monitoring

Stepping Thread Group, Utilmate thread group : - Types of Thread groups for simulating different load test
ramp up patterns during test

Active thread over time : - Type of Listener to see the number of users status during test

Discussed above add-ons installtion procedure followed by their Features;

Copying JMeterPlugins.jar file inside the JMETER_INSTALL_DIR/lib/ext. We can observe two thread groups
(ie.,Stepping Thread Group,Ultimate Thread Group) are added to Test Plan.

Stepping Thread Group:

JMeter have only one out-of-the-box option for threads (users) scheduling: simple ramp-up. But many
users, especially with HP LoadRunner experience miss more flexible thread scheduling algorythm. Stepping
Thread Group adds to JMeter thread scheduling similar to LoadRunner's. It allows to set up the Ramp up,
Ramp down, Duration explicitly where as in Normal Thread Group allows ONLY to configure Duration along
with the starting of Total Threads

Snapshot as follows

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 26 of 31
Ultimate Thread Group: -

It allows to seeing the expected behavior of the test which was designed for different Ramp up, Duration,
Ramp down for single thread group

Ex: - If we have single Thread group (one script) whereas different scheduler needs it (50% of threads
simulate different ramp up, duration, ramp down from total number of threads)

Active Threads Over Time: -

It allows seeing the Threads (Virtual Users) running status as per design the load pattern schedulers in
Normal Thread or Stepping up thread group or Set up or Tear Down thread groups (Virtual Users graph in
LR)

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 27 of 31
Listeners:

These are used to view and store the results of our test plan.Running a load test without capturing
performance metrics is kind of pointless. JMeter offers a variety of listeners to capture metrics.

Right click Thread Group then click AddListenerView Results Tree

Similarly we can add the listners such as View Resuits Tree,Summary Reports, Aggregate Reports etc.

View Results Tree(Tree view in LR)

View Results in Table

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 28 of 31
Summary Report

Some additional listeners are added to the Jmeter GUI by copying JMeterPlugins.jar file from JMeterPlugins-
VERSION.zip to JMETER_INSTALLED_DIRECTORY/lib/ext. Some of the listeners are listed below

jp@gc - PerfMon Metrics Collector: By using this listener we can monitor the CPU utilization, Memory,
Swap, Disks I/O, Networks I/O and so on. For monitoring CPU utilization, Memory, Disk I/O and so on, we
have to start the startAgent.bat file in JMeterPlugins-0.5.1\ServerAgent

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 29 of 31
jp@gc - Active Threads over Time: Active Threads over Time is a listener showing how many active threads
are there in each thread group during test run. The plugin produces graph like shown below:

jp@gc - Auto Stop Listener: Auto Stop Listener is used when we want to stop the test on some runtime
criteria. Currently 3 criteria available: average response time, average latency and error rate. These criteria
are used in OR logic, the component will ask JMeter to stop test if one of the criteria has been met.

i) Auto Stop on Response Time/Latency: There is combo-box switching which result parameter to
use in conditions: response time or latency. Test will be stopped only if specified response
time/latency exceeded for sequentially N seconds. To disable auto-stop on time criteria, just set
time value to zero

ii) Auto Stop on Error Rate: Error rate specified in percent. Rate can be float number. Test will be
stopped only if specified error rate exceeded for sequentially N seconds. To disable auto-stop
on rate criteria, just set error rate to zero

jp@gc - Bytes Throughput Over Time: This graph will display the amount of bytes sent and received by
JMeter during the load test

jp@gc - Hits per Second: This graph will display the hits generated by the test plan to the server per second

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 30 of 31
jp@gc - Transactions per Second: This graph shows the number of transactions per second for each
sampler

Jmeter HTTP Performance Testing Process Confidential & Proprietary of ARSIN

Page 31 of 31