CODEWITHSUNDEEP
javainfluxdb
Ammad
Ammad

Reputation: 4225

Influx DB write performance is too slow

I am using java to write into influxDB.

Assuming that influxDB instance is connected with database. Below is my code.

influxDB.enableBatch(500, 100, TimeUnit.MICROSECONDS);
   while (true) {
            try {
        Point point = Point.measurement("cpu").addField("idle", (Math.random()*1000)).build();
        influxDB.write(dbName, "default", point);
                } catch (RuntimeException e) {
            System.out.println(e.getMessage());
    }
    }

By using this logic i am able to write only 300 records per second which is way less than what we are expecting. 2000 writes per second will be sufficient. Wondering what parameter i should be optimizing?

Upvotes: 1

Views: 4706

Answers (1)

IIIIIIIIIIIIIIIIIIIIII
IIIIIIIIIIIIIIIIIIIIII

Reputation: 4088

influxDB.enableBatch(500, 100, TimeUnit.MICROSECONDS);

Means that you flush every 500 points or at least every 100 ms. Since you are saying you write 300 points per second I assume you are simply not generating more points in one second to write to influxdb.

I think your part which "slows" the creation of your points is the Math.random(). So try to use a fixed value and check if you get more points in one second now.

A good source for performance tests like you are trying to do is on the influxdb-java github. Taken from PerformanceTests.java there is this test which is pretty equal to the test you are doing: 

@Test(threadPoolSize = 10, enabled = false)
public void writeSinglePointPerformance() throws InterruptedException {
    String dbName = "write_" + System.currentTimeMillis();
    this.influxDB.createDatabase(dbName);
    this.influxDB.enableBatch(2000, 100, TimeUnit.MILLISECONDS);
    String rp = TestUtils.defaultRetentionPolicy(this.influxDB.version());
    Stopwatch watch = Stopwatch.createStarted();
    for (int j = 0; j < SINGLE_POINT_COUNT; j++) {
        Point point = Point.measurement("cpu")
                .addField("idle", (double) j)
                .addField("user", 2.0 * j)
                .addField("system", 3.0 * j).build();
        this.influxDB.write(dbName, rp, point);
    }
    this.influxDB.disableBatch();
    System.out.println("Single Point Write for " + SINGLE_POINT_COUNT + " writes of  Points took:" + watch);
    this.influxDB.deleteDatabase(dbName);
}

Upvotes: 2

Related Questions