Giter VIP home page Giter VIP logo

Comments (2)

tskippervold avatar tskippervold commented on May 24, 2024 1

You can just adjust your prompts to the LLM of your choosing.

Here is an example from Anthropic:
https://docs.anthropic.com/claude/page/cite-your-sources

My point is, llamaIndex and all these libraries very often don't do anything special, but just have prompt templates they use agains the LLM you choose.

You can inspect the source code yourself here: https://github.com/run-llama/llama_index/blob/cfb2d7a58a8b8fe070ece26322b7df39e8d2b804/llama-index-core/llama_index/core/query_engine/citation_query_engine.py#L32

from raptor.

parthsarthi03 avatar parthsarthi03 commented on May 24, 2024

Adding on to @tskippervold answer, you can adjust define your own custom prompt by defining your own QAModel.

For example, to use your GPT-3 Turbo as a QA model with a prompt asking for citations, you can do the following:

from raptor import RetrievalAugmentation, RetrievalAugmentationConfig, BaseQAModel
from openai import OpenAI
from tenacity import retry, stop_after_attempt, wait_random_exponential

class GPT3TurboCitationQAModel(BaseQAModel):

    def __init__(self, model="gpt-3.5-turbo"):
        self.model = model
        self.client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

    @retry(wait=wait_random_exponential(min=1, max=20), stop=stop_after_attempt(6))
    def _attempt_answer_question(
        self, context, question, max_tokens=150, stop_sequence=None
    ):
        response = self.client.chat.completions.create(
            model=self.model,
            messages=[
                {"role": "system", "content": "You are an expert research assistant."},
                {
                    "role": "user",
                    "content": f"Given the following context:\n\n{context}\n\nAnswer the question: {question}\n\nPlease provide the most relevant and concise answer, and cite the specific parts of the context that support your answer.",
                },
            ],
            temperature=0,
        )
        return response.choices[0].message.content.strip()

    @retry(wait=wait_random_exponential(min=1, max=20), stop=stop_after_attempt(6))
    def answer_question(self, context, question, max_tokens=150, stop_sequence=None):
        try:
            return self._attempt_answer_question(
                context, question, max_tokens=max_tokens, stop_sequence=stop_sequence
            )
        except Exception as e:
            print(e)
            return e

RAC = RetrievalAugmentationConfig(qa_model=GPT3TurboCitationQAModel())
RA = RetrievalAugmentation(config=RAC)
RA.add_documents(text)

You can try experimenting with different prompts, or even try stronger models. If you have a sample dataset of questions and answers with citations, you could also use DSPy (https://github.com/stanfordnlp/dspy) to automatically optimize the prompt for your specific use case.

from raptor.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.