Java Memory Leaks-Performance testing

My following posts explains about How to find Java memory leaks during performance testing , for now this post will explain about what is Java memory leak and how Java memory management will be ?

1.          About Java Memory Leak & why they happen?

Have you ever seen this situation?

  If the answer is yes, you may have a case of memory leak induced in your application, but fortunately we've got a cure for what ails you.

Above Error message Thrown when the Java Virtual Machine cannot allocate an object because it is out of memory and no more memory could be made available by the garbage collector.

An OOM or OOME (OutOfMemoryError) simply means that the JVM ran out of memory. When this occurs, you basically have 2 choices:

• Allow the JVM to use more memory using the -Xmx VM argument. For instance, to allow the JVM to use 1 GB (1024 MB) of memory:

Java -Xmx1024m...

• Improve/Fix the application so that it uses less memory

In many cases, like in the case of a memory leak, the second option is the only sound choice. A memory leak happens when the application keeps more and more references to objects and never releases them. The garbage collector will therefore never collect those objects and less and less free memory will be available until we reach the point where not enough free memory is available for the application to function normally. At this point, the JVM will throw an OOM.

How JVM Allocates Memory:

Let’s have a look at the Sun Hotspot JVM implementation as an example:

Java Heap space is divided into three regions or generation for sake of garbage collection - New Generation, Old or tenured Generation or Perm (permanent) Space. JVM allocates every generation its own memory pool. The JVM has at least one memory pool and it may create or remove memory pools during execution.

1. Eden Space (heap - young generation):
1. Pool from which memory is INITIALLY allocated for most objects.
2. Survivor Space (heap - young generation):
1. Pool containing objects that have survived GC of Eden space.
3. Tenured Generation (heap - old generation):
1. Pool containing objects that have existed for some time in the survivor space.
4. Permanent Generation (non-heap - stack):
1. Holds all the reflective data of the virtual machine itself, stores class level details, loading and unloading classes (e.g. JSPs), methods, String pool.
5. Code Cache (non-heap):
1. Hotspot JVM also includes a "code cache" containing memory used for compilation and storage of native code.

My next post will have how to find memory leak using various JDK Tools....

performance testing requirement gathering questionnaire

As a performance tester or engineer , you should interact with various stakeholders to gather performance testing requirements ( also called as NFR’s –Non Functional requirements ).

I am going to provide an inside about my experience during this activity what kind of questions you should ask for various stakeholders.

Performance testing stakeholders (mainly below list of people you may contact for NFR’s ) :

1)      Developers

2)      Project Managers

3)      Business architects

4)      System architects

5)      DBA’s

6)      Capacity Manager

7)      Production monitoring teams if any?

Setting up a testing environment, which should be replica of the production environment is most important and fundamental aspect in performance testing and results will be based on that environment. Accurate performance results can never be achieved unless the production environment is not replicated.  Like it should scalable to PROD ( 50%, or 80% or 100% ). Adjust the load numbers based on the scaling factor .

Ask what kind of application it is ( web , Client/server model, back end jobs, middle ware calls , etc …)?

How that application being used on LIVE (end users, admins , stores managers , etc…)?

Find out the key Business scenarios  (based on the traffic on the site) ?

Find out the max number of concurrent users it can support ?

Find out the SLA’s for those  Business scenarios ? Users ,Response times, throughput levels ( TPS, etc) , Servers utilization .

Type of Performance tests needs to be required ( Load , Soak , Stress tests , etc…  it depends on what is your performance goal) , for example if you are looking for to check if any memory leaks exists on your application than go for soak test , if you want to find out break even point (How many concurrent users your application supports ) than go for Stress tests, etc….

As per my experience you can ask n number of questions during requirements gathering but mainly focus on key areas which are below

Performance SLA’s

Application architecture

Load Model

Servers (App/DB/Web and Middle wares) access

Servers resource utilization threshold values ( <70% generally )

Tools & Environments ( which tool needs to use and performance test environment details ).

Monitoring tools available ( Site scope, Wily , Dyna trace , etc..)

 A typical Application architecture looks like this for 3-tier :

Performance Stress Testing Approach

Approach for Stress Testing

Stress testing will be conducted basically to find out the Break even point of your application , which is the point where your system starts degrading the performance. try to compare the number of users and response time graphs to find out the maximum number of users your application can support without deviating performance SLA's.

The following steps are involved in stress-testing a Web application:

1. Step1 - Identify stress test objectives.  Identify the objectives of stress testing in terms of the desired outcomes of the testing activity.
2. Step 2 - Identify key scenario(s).  Identify the application scenario or cases that need to be stress-tested to identify potential problems.
3. Step 3 - Identify the workload.  Identify the workload that you want to apply to the scenarios identified during the “Identify objectives” step. This is based on the workload and peak load capacity inputs.

         Generally work load would be double of your normal baseline numbers , however start gradually increasing the load to check the break point ( Multilevel Ram up scenario on HP Load runner ).

1. Step 4 - Identify metrics.  Identify the metrics that you want to collect about the application’s performance. Base these metrics on the potential problems identified for the scenarios you identified during the “Identify objectives” step.
2. Step 5 - Create test cases.  Create the test cases in which you define steps for running a single test, as well as your expected results.
3. Step 6 - Simulate load.  Use test tools to simulate the required load for each test case and capture the metric data results.
4. Step 7 - Analyze results.  Analyze the metric data captured during the test.

Please find correlated graph for number of users and average response times for the test. this would help you to find out how many number of users your application can support without crossing your SLA's.

SQL's Performance testing using Jmeter

Testing SQL's performance using Jmeter is quite simple , below steps would help for the same.

I have used Oracle 11g for testing purpose.

1) Create a JDBC request Sampler on Jmeter thead group

JDBC Request -> Provide Unique variable name ( this variable name will be refereed by JDBC connection config ) and Select a SQL Type (update or insert , etc...).

2) Create a JDBC Connection Configuration as Config element.

Make sure you you provide correct Driver class and Database URL , below are for Oracle 11g.

DataBase URL-> jdbc:oracle:thin:@Hostname:1521:InstanceName
JDBC Driver Class -> oracle.jdbc.driver.OracleDriver


3) Add Response Assertion for validate the correct response of the SQL output ( check point on Load runner (web_reg_find)).

4) Add Listeners based on your needs like summary reports , Response time graphs , etc...

Have a look at below screen for JDBC Configurations.

5) Run the thread group after setting the number of users ( threads).                                                        

If you have access to OEM ( Oracle enterprise manager console) try to monitor the SQL's after you run the jmeter thread group so that you can analyze how the SQL is performing.

Hope this helps :)