CODEWITHSUNDEEP
mongodbflaskopenai-api
Jacob Lapkin
Jacob Lapkin

Reputation: 9

Openai api Turbo-3.5 saving and sending chat history

I'm building a simple chat app using the turbo-3.5 model. I was curious about best practice for storing chat history to send in each subsequent request to openai API turbo-3.5 model.

Should I save each input and output in mongo dib for each request and then retrieve the conversation for each request?

Should I load the initial conversation from mongo d and then use server session for the rest of the current session?

Should I load from mongodb and then use redis?

Should I use mongodb to load and then save the chat history in client session?

What is best practice for saving this chat history and then only sending at most the maximum number of tokens to open ai API? Let's assume I'm using a flask API and a react web app.

I have only tried just loading each new input and output into mongo db and then each request querying the new chat history and sending with the new request.

Upvotes: 0

Views: 1614

Answers (1)

h2stein
h2stein

Reputation: 616

This is actually not a question about OpenAI or ChatGPT, but a question about basic software architecture. Since you are developing a chat app, your performance requirements are relatively low, especially since the OpenAI API is not fast anyway. Therefore, you don't need Redis or any other high-performance database. You can use any database, including MongoDB, to store chat history.

The easiest way is to store the chat interaction (request + response) in a document and simply keep the relevant history in this document. As long as you are not targeting a huge audience, this will work. In MongoDB the document size limit is 16 MB, so you have plenty of space. If you need high performance, you should separate the history (or context) from the current increment. But a single MongoDB server should easily handle several thousand users, so this is probably not your problem right now.

Storing sensitive data in the client is acceptable if you don't store it permanently. But I would not recommend it, because it easily gets out of control and you have to validate the data from a user tampering with the data. Storing sensitive data in a server session is not such a good idea, as this is not scalable and raises security concerns.

Having multiple databases in one service should be considered only if you have high performance requirements.

Upvotes: 1

Related Questions