CODEWITHSUNDEEP
loopscsvjmeter
David
David

Reputation: 135

JMeter: How to reuse/reopen a CSV in several Thread-Group-Loops?

In JMeter I have a CSV list of users, and each of these users is supposed to upload an amount of files. The files are listed in a second CSV. Each user must upload all files. Since the server can't handle all threads at once I set the Thread group to use X users and loop Y times, so that in the end all users upload all files.

 Test plan
 - CSV Data Set Config (contains users) --> Recycle = false, Stop thread on EOF = false
 - Thread Group (X users, Y loops)
 - - While Controller ( ${__javaScript("${uploadFile}"!="<EOF>")} )
 - - - CSV Data Set Config (contains files) --> Recycle = false, Stop thread on EOF = false
 - - - HTTP Request (upload file)

For the first loop this all works OK: X threads are started. In each of the threads the files-CSV is opened and each users uploads all files. Then the second Thread-Group-loop starts, the next users are taken from the users-CSV, but when those threads come to the while-loop containing the files-CSV, they don't upload anything. It seems like they re-use the files-CSV from the first loop, and consider the CSV to be EOF or something like that.

As an example, the results with 3 threads/users and 2 loops looks like this:

**first loop**
user 1
  upload file 1
  upload file 2
  upload file 3
user 2
  upload file 1
  upload file 2
  upload file 3
user 3
  upload file 1
  upload file 2
  upload file 3
**second loop**
user 4
user 5
user 6

The users from the second loop should also go through the files-CSV and upload the files. Any idea what I am doing wrong? Thanks in advance!!

Upvotes: 4

Views: 3302

Answers (5)

kishor sharma
kishor sharma

Reputation: 339

looks like we ran into the same problem. I've found the answer. The answer is long so I cannot add it here. But you can find the answer here

Upvotes: 0

Zakhooi
Zakhooi

Reputation: 36

In CSV Data Set Config you can change the default setting 'Sharing mode' from 'All threads' to 'Current thread' or 'Current thread group' to reuse the CSV data. In this way the CSV data can be reused and EOF applies per thread or thread group.

See the settings below:

enter image description here

Upvotes: 1

Ali Mizan
Ali Mizan

Reputation: 1922

I couldn't find a simple solution to this. I ended up using beanshell scripts, which let you use code very similar to java to do some custom stuff. I made an example JMeter project to demonstrate how to do this (yes it's ridiculously complicated, considering all I want to do is repeat the CSV read):

  1. Files:

my file structure:

JMeterExample
|
⊢--JMeterTests.jmx             // the JMeter file
⊢--example.csv                 // the CSV file

contents of my CSV:

guest-id-1,"123 fake street",
guest-id-2,"456 fake street",
guest-id-3,"789 fake street",

so in this thread group, I'm going to just have 1 user, and I'll loop 2 times. I intend to send 1 request per CSV line. So there should be 6 requests sent total.

  1. Thread Group

enter image description here

  1. User Defined Variables

This is kind of optional, but the filepath is subject to change, and I don't like changing my scripts just for a change in configuration. So I store the CSV filename in a "User Defined Variables" node.

If you are storing the CSV file in the same directory as your JMeter test, you can just specify the filename only.

If you are saving the CSV in a folder other than the directory containing your JMeter file, you will need to supply an absolute path, and then slightly modify the beanshell script below: you'll need to comment out the line that loads the file relatively, and comment in the line that loads from an absolute path.

enter image description here

  1. BeanShell Sampler to parse and store CSV lines

Add a Beanshell Sampler which will basically take in a path, and parse & store each line as a variable. The first line will be stored as a variable called csv_line_0, the 2nd line will be csv_line_1 and so on. I know it's not a clean solution but... I can't find any clean simple way of doing this clean simple task. I copied and pasted my code below.

enter image description here

import org.apache.jmeter.services.FileServer;
import java.text.*;
import java.io.*;
import java.util.*;

String temp = null;

ArrayList lines = new ArrayList();

BufferedReader bufRdr;

ArrayList strList = new ArrayList();     

// get the file
try {
    // you can use this line below if your csvFilePath is an absolute path
    // File file = new File(${csvFilePath});

    // you can use this line below if your csvFilepath is a relative path, relative to where you saved this JMeter file
    File file = new File(org.apache.jmeter.services.FileServer.getFileServer().getBaseDir() + "/" + ${csvFilePath});

    if (!file.exists()) {
        throw new Exception ("ERROR: file " + filename + " not found");
    }

    bufRdr = new BufferedReader(new InputStreamReader(new FileInputStream(file), "UTF8"));
} catch(Exception e){
    log.error("failed to load file");
    log.error(e.getMessage());
    return;
}

// For each CSV line, save it to a variable
int counter = 0;
while(true){
    try{
        temp = bufRdr.readLine();
        if(temp == null || temp.equals("<EOF>")){
            break;
         }
         lines.add(temp);
         vars.put("csv_line_" + String.valueOf(counter), temp);
        counter++;

        

    } catch(Exception e){
        log.error("failed to get next line");
        log.error(e.getMessage());
        break;
    }
}

// store the number of CSV lines there are for the loop counter
vars.put("linesCount", String.valueOf(lines.size()));
  1. Loop Controller

Add a Loop Controller that loops once for each CSV line. ${linesCount} is a count of the number of CSV lines and is calculated from the above beanShell script.

enter image description here

  1. Beanshell script to extract data from current CSV Line

This script will run once per CSV line. It will go and grab the current line, and parse out whatever data is on it. You'll have to modify this script to get the data you want. In my example, I only had 2 columns, where column 1 is a "guestId", and column 2 is an "address".

__jm__loopController__idx is a variable JMeter defines for you, and is the index of the loop controller. The variable name is __jm__{loop controller name}__idx.

enter image description here

String index = vars.get("__jm__loopController__idx");
String line = vars.get("csv_line_" + index);
String [] tokens = line.split(",");
vars.put("guestId", tokens[0]);
vars.put("address", tokens[1]);
  1. Http request sampler

Here's the HTTP request that's using the data extracted.

enter image description here

  1. result

When running this, as desired, I end up sending 6 http requests over to the endpoint I defined.

enter image description here

Upvotes: 1

RowlandB
RowlandB

Reputation: 573

So this is how I set this up:

Double_CSV_Test_Plan

I set the Users CSV to Stop on End of File (and not Recycle on EOF).

I set the Files CSV to Recycle on End of File (and not Stop on EOF).

Note that I only have 1 user per thread. Yes, you will run more Thread Loops, but each Loop will be shorter.

Upvotes: 0

Manish Sapariya
Manish Sapariya

Reputation: 3605

I think you need to enable Recycle on EOF. I would also check logs of jmeter for some kind of hints about, why jmeter is not sending any more requests.

From the jmeter documentation, 

When the end of file (EOF) is reached, and the recycle option is true, 
reading starts again with the first line of the file.

If the recycle option is false, and stopThread is false, then all the 
variables are set to <EOF> when the end of file is reached. This value 
can be changed by setting the JMeter property csvdataset.eofstring .

Upvotes: 0

Related Questions