CODEWITHSUNDEEP
javacassandra
TrongBang
TrongBang

Reputation: 967

Cassandra Throwing WriteTimeout Exception without Thread.sleep

I have a batch job writing around 300,000 lines into cassandra. I divide them into smaller batches whose size is 50 lines each.

Pseudo code is below.

@Override
public void executeQuery(List<BatchStatement> batches) {
    List<ResultSetFuture> futures = List.of();
    for (BatchStatement batch: batches) {
        futures.add(session.executeAsync(batch));
    }

    for(ResultSetFuture rsf: futures) {
        rsf.getUninterruptibly();
        /* I have to add the following code to avoid WriteTimeoutException
        try {
            Thread.sleep(100);
        } catch (InterruptedException e) {
            logger.error("Thread.sleep", e);
        }
        */

    }
}

I don't know why without Thread.sleep, it always gives WriteTimeout Exception. How to avoid this?

Upvotes: 1

Views: 231

Answers (1)

Alex Ott
Alex Ott

Reputation: 87154

By using the batch statement on data (that most probably belongs to different partitions) you're really overload your system because coordinating node need to send requests to other nodes and wait for answer. You need to use batches only for specific use cases, and not the same way as you used them in relational databases - to speedup execution. This documentation describes the bad use of batches.

Sending individual asynchronous requests for every line will improve situation, but you need to take care that you don't send too many requests at the same time (by using semaphore), and by increasing the number of in-flight requests per connection via pooling options.

Upvotes: 2

Related Questions