CODEWITHSUNDEEP
questdb
jfarjona
jfarjona

Reputation: 145

Is QuestDB (embedded) TableWriter thread-safe?

I am using questdb (embedded) to store a bunch of time series. I would like to run my storage method inside a parallel stream, but I don't know if TableWriter is thread-safe.

Here is the code:

        SqlExecutionContextImpl ctx = new SqlExecutionContextImpl(engine, 1);

        try (TableWriter writer = engine.getWriter(ctx.getCairoSecurityContext(), name, "writing")) {
            tickerData.stream().parallel().forEach(
                r -> {
                        Instant         i   = r.getDateTime("DateTime")
                                               .atZone(EST)
                                               .toInstant();
                        long            ts  =  TimestampFormatUtils.parseTimestamp(i.toString());
                        TableWriter.Row row = writer.newRow(ts);

                        row.putDouble(0, r.getDouble("x1"));
                        row.putDouble(1, r.getDouble("x2"));
                        row.putDouble(2, r.getDouble("y1"));
                        row.putDouble(3, r.getDouble("y2"));
                        row.putDouble(4, r.getDouble("z"));
                        row.append();
                        writer.commit();
                    } catch (NumericException ex) {
                        log.error("Cannot parse the date {}", r.getDateTime("DateTime"));
                    } catch (Exception ex) {
                        log.error("Cannot write to table {}!", name, ex);
                    }
                });
        }

This throws all sort of errors, is there a way to make the storage process parallel?

Thanks,

Juan

Upvotes: 1

Views: 133

Answers (1)

Alex des Pelagos
Alex des Pelagos

Reputation: 1465

The short answer is TableWriter is not thread safe. You will be responsible to not use it in parallel threads.

A bit longer answer is that even in stand alone QuestDB parallel writing is restricted. It is only possible from multiple ILP connections at the moment.

Upvotes: 0

Related Questions