9
Advanced Goal-Oriented
Scenario
In the previous two lessons you
learned how to manually create and run a
load test. In this lesson, you will
define a goal that you want your test to
achieve.
Before deploying an application, you
want to run an acceptance test to
make sure that the system will stand
up to the expected real-life workload.
You have a rate at which you expect
the server should perform, defined, for
example, by the number of hits or
transactions per second. This rate might
be determined by a business analyst
defining the requirements for the
application, or it might be taken from
previous versions of the application
in production, or other sources. You
set a goal for the number of hits per
second, transactions per second, or
the transaction response time that you
want to generate, and LoadRunner
automatically generates the required goal
for you, using the goal-oriented
scenario. While the application is under
constant load, you can monitor the
transaction response time to see the
level of service provided to the
customer.
In this lesson, you will create a
goal-oriented scenario to generate three hits
per second on your Web server, and to
maintain this level of load for five
minutes using a minimum of five Vusers
and a maximum of ten. In this
lesson you will cover the following
topics:
➤Which goal type should I use?
➤How do I create a goal-oriented scenario?
➤The Controller Window at a Glance
(Goal-Oriented Scenario)
➤How do I define the goal?
➤How do I determine load behavior?
➤Which online graphs should I monitor?
➤How do I run a goal-oriented scenario?
➤Did I achieve my goal?
Which goal type
should I use?
LoadRunner provides you with five
different types of goals in a goaloriented
scenario: the number of concurrent
Vusers, the number of hits per
second, the number of transactions per
second, the number of pages per
minute, or the transaction response
time that you want your scenario to
reach.
➤If you know the total number of Vusers that
can run your various business
processes, you can use a Virtual Users
goal type.
➤If you know the strength of your server, you
can use a Hits per Second, Pages
per Minute, or Transactions per Second
goal type.
➤If you know the desired response time for
completing a transaction, you can
use a Transaction Response Time goal
type. For example, if you do not want
a customer to wait more than five
seconds to log in to your e-commerce site,
specify a maximum acceptable
transaction response time of five seconds,
and see how many actual Vusers can be served.
How do I create a
goal-oriented scenario?
To emulate a real-life system with a
mix of user profiles, you can assign
several scripts to the scenario and
assign a percentage of the load between
them. You should set the percentage
according to the expected load.
In this tutorial, you will use just
one Vuser script to model a single group of
users performing identical actions.
1 Create a new
scenario.
Choose File > New, or click New, to open the New
Scenario dialog box.
2 Select the scenario
type.
Select the Goal-Oriented Scenario
option.
3 Choose a script.
Select basic_scriptfrom the Available
Scripts list, and click the Add button.
The script is displayed in the Scripts
in Scenario pane.
Click OK. The LoadRunner Controller
design view opens displaying the
basic_scriptin the Script Name column.
The Controller Window
at a Glance (Goal-Oriented
Scenario)
The Controller window (goal-oriented)
Design view contains three primary
sections:
➤Scenario Goal
➤Scenario Scripts
➤Service Level Agreement
Scenario Scripts: In the upper pane,
you determine the Vuser scripts, their
paths, the percentage of the total
target assigned to each script, and the load
generator machines. You configure the scenario settings
from here.
Scenario Goal: In the lower pane, you
can see your testing goal, the number
of users that you want to use to reach
that goal, the scenario duration, and
load behavior. You define the goal
settings from the Edit Scenario Goal
dialog box.
Service Level Agreement: When you
design a load test scenario, you can
define goals or service level
agreements (SLAs) for the performance metrics.
When you run the scenario, LoadRunner
gathers and stores performancerelated
data. When you analyze the run,
Analysis compares this data against
the SLAs and determines SLA statuses
for the defined measurements.
How do I define the
goal?
Now that you have selected a script to
run, you need to define the goal that
you want to achieve.
In this section, you will create a goal profile, and
define the scenario goal.
How do I determine
load behavior?
Now that you have defined the test
goal, you need to specify how and when
you want the Controller to reach your
target.
Users do not log on and off the system
at precisely the same time. To
emulate real users, LoadRunner
provides the capability in the Load Behavior
tab for users to gradually log on and
off the system. You also want the server
to remain under the load for a period
of time. LoadRunner lets you specify
in the Scenario Settings tab the time
that the server remains under load.
To define load test behavior:
1 In the Edit Scenario
Goal dialog box, select the Load Behavior tab, and
select Automatic.
This instructs the Controller to run
the required number of Vusers
simultaneously.
4 Close the Edit
Scenario Goal dialog box.
Click OK to close the Edit Scenario
Goal dialog box.
The scenario target information you
entered appears in the Scenario Goal
window.
Which online graphs
should I monitor?
After you have defined the test goal
and load behavior, you are ready to
configure the LoadRunner monitors. In
this test, you should monitor the
Hits per Second graph to follow the
load that is generated on the server. You
also want to monitor the Transaction
Response Time graph to see what
response times your customers will
have when the server is under load. In
addition, you can monitor the effect
of load on the Throughput and
Windows Resources graphs.
The Hits per Second, Transaction
Response Time, and Throughput monitors
have been configured for you. To
configure the Windows Resources
monitor, follow the procedures in
Lesson 7, “Creating a Load Testing
Scenario.”
How do I run a
goal-oriented scenario?
Now that you have configured the
scenario and goal settings, you are ready
to start the test and monitor the
application under load. In this section you
will run your goal-oriented scenario
and examine the test behavior.
1 Open the Controller
window’s Run tab.
Select the Run tab at the bottom of
the screen.
Since the scenario has not run yet,
all the counters remain at zero and the
graphs are blank. When you start the
scenario in the next step, the graphs
and counters will begin to display
information.
2 Specify a name for
the Results directory.
Choose Results >Results Settings to
open the Set Results Directory dialog
box, and enter a unique name for the
result set (for example:
travel_agent_3hps).
3 Start the scenario.
Click the Start Scenario button or
choose Scenario > Start.
The Controller begins the scenario.
You will see that 5 Vusers are ramped
up and start running, as LoadRunner
attempts to generate the required goal
of three hits per second.
During the test, the Controller
automatically starts and stops Vusers to
maintain the given goal.
You can check the Windows Resources
graph legend for a list of
measurements.
Did I achieve my
goal?
The objective of this lesson was to
make sure that the system would provide
an acceptable level of service to the
customer under expected real-life
workload. To emulate such conditions,
you set a load target of three hits per
second for the duration of the
scenario, when running a minimum of five
and maximum of ten Vusers. If three
hits are made by Vusers on the server
during each second of the scenario run
when running between five and ten
Vusers, then your goal parameters have
been achieved. If your target of three
hits per second is not reached,
LoadRunner displays a message that the
target you defined cannot be achieved.
Note: Since the license limits you to
running a maximum of ten Vusers,
your goal might not be reached.
After you have run the test, you
should save the scenario settings for future
use. To save the scenario, click File
> Save or click the Save button and enter
a scenario name in the File name box.
Where To Go From Here
Now that you have designed and run a
goal-oriented scenario, you can
proceed to Lesson 10, “Analyzing Your Scenario.”
10
Analyzing Your
Scenario
In the previous lessons you learned
how to design, control, and execute a
scenario run. Once you have loaded
your server, you want to analyze the
run, and pinpoint the problems that
need to be eliminated to improve
system performance.
The graphs and reports produced during
your analysis session present
important information about the
performance of your scenario. Using these
graphs and reports, you can easily
pinpoint and identify the bottlenecks in
your application, and determine what
changes need to be made to your
system to improve its performance.
In this lesson you will cover the
following topics:
➤How does an analysis session work?
➤How do I start my analysis session?
➤The Analysis Window at a Glance
➤Transaction Summary - Did I reach my goals?
➤Did my server perform well?
➤How can I pinpoint the source of the problem?
➤What other information can I gather about my
scenario run?
➤How can I publish my findings?
➤Conclusion
How does an analysis
session work?
The aim of the analysis session is to
find the failures in your system’s
performance and then pinpoint the
source of these failures.
1 Were the test
expectations met? What was the transaction response time on
the user’s end under load? What was
the average transaction response time
of the transactions?
2 What parts of the
system could have contributed to the decline in
performance? What was the response
time of the network and servers?
3 Can you find a
possible cause by correlating the transaction times and
backend monitor matrix?
In the following sections, you will
learn how to open LoadRunner Analysis,
and build and view graphs and reports
that will help you find performance
problems and pinpoint the sources of these problems.
How do I start my
analysis session?
1 Open HP LoadRunner.
Choose Start > Programs
>LoadRunner>LoadRunner. The HP LoadRunner
Launcher window opens.
2 Open LoadRunner
Analysis.
In the Load Testing tab, click Analyze
Load Tests. The LoadRunner Analysis
opens.
3 Open the analysis
session file.
For the purpose of this section of the
tutorial, and for more interesting
results, we ran a test scenario
similar to those you ran in the previous
lessons. This time, however, the test
incorporated 70 Vusers rather than 10
Vusers. You will now open the analysis
session created from the results of
this scenario.
In the Analysis window, choose File
> Open. The Open Existing Analysis
Session File dialog box opens.
From the <LoadRunner
Installation>\Tutorial folder, select analysis_session
and click Open.
The Analysis Window
at a Glance
Analysis contains three primary
windows:
➤Session Explorer
➤Graph Viewing Area
➤Graph Legend
Session Explorer: In the upper left
pane, Analysis shows the reports and
graphs that are open for viewing. From
here you can display new reports or
graphs that do not appear when
Analysis opens, or delete ones that you no
longer want to view.
Graph Viewing Area: In the upper right
pane, Analysis displays the graphs.
By default, the Analysis Summary
report is displayed in this area when you
open a session.
Graph Legend: In the lower right pane,
you can view data from the selected
graph.
Note: There are additional windows
that can be accessed from the toolbar
that provide additional information.
For example, a Properties window.
These windows can be dragged and
dropped anywhere on the screen.
The Summary Report
Open the Summary report from the
Session Explorer. The report includes
the following sections
Statistics Summary
In the Statistics Summary of the
report, you can see that a maximum of 70
Vusers ran in this test. Other
statistics such as the total/average throughput,
and the total/average hits are also logged here for your
information
The Transaction Summary lists a
summary of the behavior of each
transaction.
Look at the response times of each
transaction. The 90 Percent column
displays the response time of 90% of
the executions of a particular
transaction. You can see that 90% of
the check_itinerary transactions that
were performed during the test run had
a response time of 65.744 seconds.
This is double the average response
time of this transaction, 32.826, which
means that the majority of occurrences
of this transaction had a very high
response time.
We also see that this transaction failed 28 times.
Did my server perform
well?
In the previous section you saw
instability in your server’s performance.
Now you will analyze the effect of 70
running Vusers on the system’s
performance.
1 Study the behavior of
the Vusers.
Click Running Vusersin the graph tree.
The Running Vusers graph opens in the
graph viewing area. You can see that
there was a gradual start of running
Vusers at the beginning of the scenario
run. Then, for a period of 3 minutes,
70 Vusers ran simultaneously, after
which a gradual ramp down began.
Saving a template
Up to now you have filtered a graph
and correlated two graphs. The next
time you analyze a scenario, you might
want to view the same graphs, with
the same filter and merge conditions
applied. You can save your merge and
filter settings into a template, and
apply them in another analysis session.
To save your template:
1 From the Tools menu,
choose Templates > Save as Template...
2 Enter an appropriate
name for your template.
3 Clear the
Automatically apply this template to a new session option.
4 ClickOK.
Next time you open a new analysis
session and want to use a saved
template:
1 From the Tools menu,
choose Templates > Apply/Edit Template.
2 Select your template from
the list, and click Apply Template.
How can I pinpoint
the source of the problem?
Until now, you have seen that an
increase in load on the server had a
negative impact on the average
response time of the check_itinerary
transaction.
You can drill down further into the
check_itinerary transaction to see what
system resources could have negatively
influenced its performance.
Unique to LoadRunner Analysis, the
Auto-correlate tool can merge all the
graphs that contain data that could
have had an effect on the response time
of the check_itinerary transaction,
and pinpoint what was happening at the
moment the problem occurred.
1 From the graph tree, select
the Average Transaction Response Time graph.
Look at the check_itinerary
transaction, particularly at the slice of elapsed
time between 1 and 4 minutes. The
average response time started to increase
almost immediately, until it peaked at nearly 3 minutes.
5 Analyze the
auto-correlated graph.
Look at the legend below the graph.
In
In the Measurement column you can see
that the Private Bytes and Pool
Nonpaged Bytes, both of which are
memory-related measurements, have a
Correlation Match of over 70% with the
check_itinerary transaction. This
means that the behavior of these
elements was closely related to the
behavior of the check_itinerary
transaction during the specified time
interval.
We can deduce that at the instant when
the check_itinerary transaction’s
response time peaked, there was a
shortage of system memory resources.
What other
information can I gather about my scenario
run?
In addition to the graphs that appear
in the graph tree at the start of an
analysis session, you can display
different graphs to get other information
about your scenario run.
1 Display a new graph.
Click the Add New Graph button on the
toolbar or, alternatively, click New
Graph in the graph tree.
How can I publish my
findings?
You can publish the findings from your
analysis session in an HTML or
Microsoft Word report. The report is
created using a designer template, and
includes explanations and legends of
the presented graphs and data.
HTML Reports
The HTML report can be opened and
viewed in any browser.
To create an HTML report:
1 From the Reports
menu, choose HTML Report...
2 Select a file name
for your report, and the path where you want to save it.
Click Save.
Analysis creates the report and
displays it in your Web browser. Note how
the layout of the HTML report is very
similar to the layout of your analysis
session. You can click on the links in
the left pane to see the various graphs.
A description of each graph is given
at the bottom of the page.
Microsoft Word
Reports
You can present your analysis session
in a Microsoft Word report. The Word
report is more comprehensive than the
HTML report, since you have the
option to include general information
about the scenario, measurement
descriptions, and so on. You can also
format the report to include your
company’s name and logo, and the
author’s details.
Like any Word file, the report is
editable, so you can add further comments
and findings after you have built the report.
The data is gathered and the report is
created in a Word file, which opens in
Microsoft Word.
In addition to the graphs that you
generated during your analysis session,
the report includes an objective and a
conclusion, and other sections and
graphs that you chose to include while
building the report.
Conclusion
In this lesson you learned the basics
of analyzing a scenario run and
publishing your results in a report.
You have learned that performance
problems can be pinpointed by studying
various graphs that show bottlenecks
in the server, possibly due to too
heavy a load, and that you can
pinpoint the sources of these bottlenecks by
configuring graphs to display correlated data.
