CODEWITHSUNDEEP
postgresqldenoopenai-apisupabase-database
Rodrigo Rubio
Rodrigo Rubio

Reputation: 1760

deno postgres.js embeddings query in deno under supabase functions

Trying to run an embeddings query on data stored from openAI. Currently using Supabase functions, using the following library. Any ideas on how to fix this?

OpenAI Embeddings response is successful -

const [{ embedding }] = embeddingResponse.data.data;
const query_embedding = embedding;

import postgres from 'https://deno.land/x/postgresjs/mod.js'


const res = await sql`
select
id,
content,
  1 - (documents.embedding <=> ${query_embedding}) as similarity
from documents
where 1 - (documents.embedding <=> ${query_embedding}) > ${similarity_threshold}
order by documents.embedding <=> ${query_embedding}
limit ${match_count}

`;

On execution, the following error returns. Please note, pg_vector extension has already been enabled an i'm able to save embeddings successfully -

Error... PostgresError: malformed vector literal: "0.0013529072,-0.01248068,-0.021260735,-0.009094394,-0.0059045693,0.04160703,-0.028719138,-0.018231653,-0.005847417,-0.0039685275,0.02343253,0.020260567,-0.0124663925,-0.009144402,-0.013738035,0.028161902,0.02916207,0.00086130627,0.010994715,-0.015588349,0.00782275,-0.0022450222,0.003548814,-0.024804192,-0.009637343,0.008608597,0.015702654,-0.033634257,-0.01856028,-0.027047427,0.032262594,0.006036734,-0.012052037,-0.0009858808,-0.019074652,-0.006551107,0.00666184,0.0058259843,0.021789396,0.020774938,-0.0035988223,0.006126035,0.013023629,0.031205272,-0.011916299,0.014002366,-0.020103397,-0.05046567,-0.0022110878,0.019017499,0.035948932,0.0016172376,0.016074145,0.005100862,0.0026808102,0.017088601,0.0016190236,-0.0064903824,-0.005561654,-0.000671542,0.027804699,0.0044936165,-0.016774263,0.023303937,0.0015582991,0.01573123,0.016902857,-0.008672894,0.0055973744,-0.012752155,0.025447156,0.015245433,-0.010523207,-0.022275193,0.023089616,-0.006783289,-0.042778656,0.00006451983,-0.012730722,0.010273164,-0.004147129,-0.011473367,0.021475058,0.009851664,-0.010101707,-0.0142952725,0.016202739,0.010051698,-0.010808969,-0.00961591,0.022803852,0.026518766,0.006436802,0.009687351,0.0027969012,-0.004782951,-0.026447326,0.018574568,0.002771897,-0.0052651754,-0.0076298607,-0.007937055,-0.006904738,-0.0023039607,-0.022289481,-0.018303093,-0.018688872,0.008122801,0.022546668,-0.021089278,-0.013216519,0.042435743,-0.00572954,-0.02237521,-0.020560617,-0.015102552,-0.00805136,0.0033041297,-0.0024039776,-0.026604496,0.012845027,0.0044936165,0.013016486,-0.033348493,0.008322835,0.015702654,-0.046379264,-0.011787706,-0.018874617,0.0034845173,0.076241456,0.017817296,0.02176082,0.0035005915,-0.0016243816,0.050151333,-0.012930756,-0.0032130429,-0.0128164515,-0.023546835,0.004125697,0.010901842,0.0049186884,0.013273671,-0.01856028,0.015602636,0.040635437,0.0076227165,-0.020274855,-0.014459586,0.009865953,-0.0024986365,0.040121064,-0.019631889,0.0039042311,-0.019674754,-0.046436418,0.0013707674

Schema definition is as following -

create table documents (
  id bigserial primary key,
  content text,
  embedding vector (1536)
);

#supabase #deno #openai #postgresjs

Upvotes: 0

Views: 345

Answers (2)

Rodrigo Rubio
Rodrigo Rubio

Reputation: 1760

Managed to get it working. See working version below, ended up using "JSON.stringify".

  const res = await sql`
    select
    id,
    content,
      1 - (documents.embedding <=> ${JSON.stringify(query_embedding)}) as similarity
    from documents
    where 1 - (documents.embedding <=> ${JSON.stringify(query_embedding)}) > ${similarity_threshold}
    order by documents.embedding <=> ${JSON.stringify(query_embedding)}
    limit ${match_count}
  `;

Upvotes: 2

dshukertjr
dshukertjr

Reputation: 18670

You need [] around your vector values.

import postgres from 'https://deno.land/x/postgresjs/mod.js'


const res = await sql`
select
id,
content,
  1 - (documents.embedding <=> [${query_embedding}]) as similarity
from documents
where 1 - (documents.embedding <=> [${query_embedding}]) > ${similarity_threshold}
order by documents.embedding <=> [${query_embedding}]
limit ${match_count}
`;

Upvotes: 0

Related Questions