CODEWITHSUNDEEP
typesense
Tahola
Tahola

Reputation: 1015

Typesense : check duplicate before import

I am new to Typesense and its a bit confusing, is there a way to check for a duplicate before importing a new document ? I thought that adding a primary_key would avoid that but apparently no, what is the best practice then ?

This is my collection :

$schema = [
            'name' => 'videos',
            'fields' => [
                ['name' => 'title', 'type' => 'string'],
                ['name' => 'tags', 'type' => 'string', 'facet' => true],
                ['name' => 'date', 'type' => 'int32'],
                ['name' => 'video_id', 'type' => 'int32'],        
                ['name' => 'secondes', 'type' => 'int32'],
                ['name' => 'views', 'type' => 'int32'],
            ],
            'default_sorting_field' => 'date',
            'primary_key' => 'video_id',
        ];

Upvotes: 0

Views: 1035

Answers (2)

Rohan Kadam
Rohan Kadam

Reputation: 19

Use actions like "upsert" and "emplace" to prevent Typesense from inserting duplicate documents.

Difference between Upsert and Emplace

Upsert -

  1. When an ID is found in Typesense, Upsert updates the document with that ID; otherwise, it creates a new document with that ID.
  2. Even if the ID is found, Upsert won't update the document partially; instead, it needs data that matches the schema before it updates.

Emplace -

  1. Creates a new document or updates an existing document if a document with the same id already exists. You can send either the whole document or a partial document for update.

Read more about upsert and emplace - https://typesense.org/docs/0.25.1/api/documents.html#index-multiple-documents

Hence, rather than using upsert in your situation, I would advise using emplace.

Upvotes: 2

Tahola
Tahola

Reputation: 1015

Answering my own questions (it can help others), so I found out that there is an options called upsert :

Upsert creates a new document or updates an existing document if a document with the same id already exists

To use it you just need to replace create by upsert when you create a document :

// before
$client->collections['videos']->documents->create($document);


// after
$client->collections['videos']->documents->upsert($document);

Be sure to have a unique id and (that this one is a string) when you create your document, in my case I removed the primary_key param when I created my collection, instead I just set an unique id when I create my document ex: $newdocument[id] = strval($id);

Upvotes: 3

Related Questions