Jesus Diaz Rivero
Jesus Diaz Rivero

Reputation: 325

Unable to find results in vector search when using redis as a vector database

I'm trying to follow the redis documentation example to calculate the distance between two embeddings arrays but I'm using Typescript. I've managed to create an index and load data, but no matter how I run the query on redis I get 0 results.

Using JSON keys

My current setup looks like this (simplified):

const INDEX_NAME = 'idx:jobs';

const client = createClient({
  socket: {
    host: 'cache',
    port: 6379, // Number.parseInt(REDIS_PORT, 10),
    connectTimeout: 10000,
    // tls: true,
  },
});

const schema: RediSearchSchema = {
  '$.title': { type: SchemaFieldTypes.TEXT },
  '$.document': { type: SchemaFieldTypes.TEXT },
  '$.embeddings': { type: SchemaFieldTypes.VECTOR, DIM: 1536, DISTANCE_METRIC: 'COSINE', ALGORITHM: VectorAlgorithms.FLAT, TYPE: 'FLOAT32' }
}

client.on('error', (error) => {
  console.error('Redis error', { error });
  client.disconnect();
});

app.listen(port, async () => {
  await client.connect();
  await client.ft.create(INDEX_NAME, schema, { PREFIX: 'jobs:', ON: 'JSON' });
  const id1 = randomUUID();
  await client.json.SET(`jobs:${id1}`, '.', { title: 'dummy1', document: 'test stuff', embeddings: embedding });
  const id2 = randomUUID();
  await client.json.SET(`jobs:${id2}`, '.', { title: 'dummy', document: 'test stuff', embeddings: embedding });
  const id3 = randomUUID();
  await client.json.SET(`jobs:${id3}`, '.', { title: 'dummy', document: 'test stuff', embeddings: embedding });
  console.log(`Server is running at http://localhost:${port}`, client.isReady);
});

I am calculating the distance using the example's KNN method:

const embeddingArray = []; // This is an array with floats of length 1536
const embeddings = new Float32Array(embeddingArray);

const result = await client.ft.search(
    INDEX_NAME,
    '(*)=>[KNN 3 @embeddings $query_vector AS vector_score]', {
      DIALECT: 2,
      RETURN: ['vector_score', 'title', 'document'],
      PARAMS: {
        'query_vector': Buffer.from(embeddings.buffer),
      },
    }
);

Using Hash keys

I have also tried following the example in the `redis-stack` UI which uses hash keys, basically changing the `json.set` to:
await client.hSet(`jobs:${id1}`, { title: 'dummy1', document: 'test stuff', embeddings: Buffer.from((new Float32Array(embedding)).buffer) });

Changing the index to ON: 'HASH', and querying the index the exact same way. I still get 0 results.

Any pointers or help would be highly appreciated. I believe I'm doing something wrong when converting the data to binary, but I can't find the issue.

Upvotes: 0

Views: 105

Answers (0)

Related Questions