CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaaws-api-gatewaylarge-language-modelaws-networking
Gregory
Gregory

Reputation: 195

Build a secure LLM RAG system on AWS

I'm developing an AWS architecture for a Retrieval Augmented Generation (RAG) system:

AWS Architecture for RAG LLM

  1. Data (pdf, docx, txt, png, etc.) ingested and pre-processed are stored in an Amazon S3 bucket.
  2. A Lambda function is triggered to orchestrate the embedding process.
  3. The Lambda function uses an embedding model from Amazon Bedrock to generate embeddings of the document.
  4. The embeddings are stored in a Mongo Database that acts as a vector database.
  5. The user asks a question through the UI/UX frontend hosted in an AWS Amplify instance, and the question is forwarded to the Amazon API Gateway.
  6. The API request is managed by the API Gateway Endpoint and triggers the Lambda function for the application orchestration.
  7. The Lambda function retrieves the correct embedding vector correlated to the user question.
  8. The Lambda function also retrieves the chat history from the DynamoDB database.
  9. The context retrieved from the MongoDB Database and the chat history are augmented in the prompt for the request to Amazon Bedrock LLM model, which gives back an answer.
  10. The answer from the LLM is verified by another Lambda function triggered from the Lambda Orchestrator Function.
  11. The answer is forwarded to the UI/UX frontend.

SECURITY

Now, in terms of security, I don't want the system to be exposed to the public Internet (except for the front-end web app, that of course has to be public) therefore I'm putting the RDS MongDB database and all the Lambda functions inside the VPC (I know, technically the Lambda is always inside a VPC owned by the Lambda service, but in this case the Lambda functions are drawn inside my account VPC just to specify that they are configured to access resources in my account VPC). As you can see the connection with the S3 bucket and DynamoDB database is through a VPC Gateway Endpoint. While the connection with Amazon Bedrock service is through VPC Interface Endpoint (AWS PrivateLink).

QUESTION

Now my question is related to the connection between the front-end app and the API Gateway and the Lambda Orchestrator function. In particular, can a public API Gateway endpoint interact with the in-VPC Lambda Orchestrator function? Or do I need to set up an API Gateway private endpoint in the same VPC of the in-VPC Lambda function to allow communication? If I change the API Gateway to be private endpoint, how can I establish communication between the public front-end and the private API Gateway?

Upvotes: 1

Views: 631

Answers (0)

Related Questions