Reputation: 111
DSE VERSION 6.7 and DSE graph loader version 6.7
We have two vertex name "x" and "y" and there is connecting edges named "z" between for x to y. That edge has property also. So wee needs to load that also. And properties data types are Txt(string) except vertex partitions key wihich is UUID.
As mentioned in documentation from Datastax to load edges with properties we need gzip the CSV . https://docs.datastax.com/en/dse/6.7/dse-dev/datastax_enterprise/graph/dgl/dglMapScript.html Now we are loading data using graph loader to load edges between and x and y .
here is our groovy script
// CONFIGURATION
// Configures the data loader to create the schema
//config create_schema: false,preparation: true, load_new: true, load_vertex_threads: 0,dryrun: true,schema_output: '/tmp/loader_output.txt'
config create_schema :false,load_new :true
inputfiledir = '/root/'
waInput = File.csv(inputfiledir + "test.csv.gz").gzip().delimiter('|')
//load edge
load(waInput).asEdges {
label "z"
outV "x", {
label "x"
key "xId"
}
inV "y", {
label "y"
key "yId"
}
}
here is our csv file format
xId|yId|newCase|totalActiveCases|totalRecoveries|totalDeaths|allCasesTotal|totalTestConducted
8c49304d-71e9-4e9d-93b3-47ebfd0ce073|e31f0a23-64c0-44ea-add8-c67eb52c0187|0|0|0|0|0|0
d451b2b2-a4b5-4ed6-bb4e-128945795e57|e31f0a23-64c0-44ea-add8-c67eb52c0187|0|0|0|0|0|0
xId and yId is UUID and except them, all are Text(string) datatype in database
now we run the that groovy script with dse graph loader we get this error
2020-04-21 05:25:14 INFO DataLoaderImpl:213 - Scheduling [test.csv] for reading
2020-04-21 05:25:14 DEBUG Reporter:69 - Input queue [test.csv] throughput is [111.74281809353118] items/s
2020-04-21 05:25:14 DEBUG Reporter:120 - query times 9: p50 4515.0µs, p80 4515.0µs, p90 4515.0µs, p95 4515.0µs, p99 4515.0µs, p99.9 4515.0µs, p99.99 4515.0µs
2020-04-21 05:25:14 ERROR DataLoaderImpl:720 - Failed when finalizing loader, some items that were in the queue may not have been written
com.datastax.dsegraphloader.exception.LoadingException: com.datastax.driver.core.exceptions.InvalidQueryException: Null not allowed
at com.datastax.dsegraphloader.impl.loader.driver.DseGraphDriverImpl.executeGraphQuery(DseGraphDriverImpl.java:149)
at com.datastax.dsegraphloader.impl.loader.driver.DseGraphDriverImpl.getOrCreateVertices(DseGraphDriverImpl.java:366) at com.datastax.dsegraphloader.impl.loader.driver.SafeGraphDriver.lambda$tryGetOrCreateVerticies$28(SafeGraphDriver.java:137)
at com.datastax.dsegraphloader.impl.loader.driver.SafeGraphDriver.tryDriverCall(SafeGraphDriver.java:72)
at com.datastax.dsegraphloader.impl.loader.driver.SafeGraphDriver.tryGetOrCreateVerticies(SafeGraphDriver.java:125)
at com.datastax.dsegraphloader.impl.loader.vertex.VertexLoader.completeLoad(VertexLoader.java:100)
at com.datastax.dsegraphloader.impl.loader.DataLoaderImpl$LoaderCallable.call(DataLoaderImpl.java:645)
at com.datastax.dsegraphloader.impl.loader.DataLoaderImpl$DelegatingLoaderCallable.run(DataLoaderImpl.java:530)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: Null not allowed
Any solutions for that ?
Upvotes: 3
Views: 81