(QA construct): Update Q/A logic about generative-ai-cdk-constructs HOT 2 CLOSED

Reiterating the problem

The construct currently employs a long context window approach for document processing when a document name is specified. This means it likely uses the entirety of the document's content or a significant part of it as context for generating responses.

Potential Solution

A potential solution involves adding a new parameter to the GraphQL schema that allows users to select the method (long context window or RAG). This parameter could be something like responseGenerationMethod with possible values longContext and RAG.

Plan

1. Schema Update:
   a. Modify the GraphQL schema to include the new parameter.
2. Handling the Parameter in Lambda Functions:
   a. Update the AWS Lambda function logic to handle this new parameter. This will involve:

• Implementing a conditional logic to choose between the long context window approach and the RAG approach based on the parameter's value.
• For the RAG approach, you might need to implement or integrate a mechanism to fetch relevant document snippets based on the query, which are then used as context for the response generation.

5. Testing:
   a. Ensure thorough testing for both paths (long context and RAG) to confirm that the system behaves as expected in each case.
6. Update the documentation
   a. to include information about the new parameter, and provide examples of how to use it.

from generative-ai-cdk-constructs.

Perfect, thanks @MichaelWalker-git ! Just one comment: "For the RAG approach, you might need to implement or integrate a mechanism to fetch relevant document snippets based on the query, which are then used as context for the response generation." -> this is already there, need only the conditional logic :)

from generative-ai-cdk-constructs.