uw-macrostrat / factsheet-generator Goto Github PK

I will make a draft.

When hitting Ollama or other llms, use: https://pypi.org/project/tenacity/3.2.0/

Ollama seems to stop at 1 minute timeout if the prompt is too long. Need a longer timeout threshold setting.

In order to align with Devesh's approach to graph extraction. We should focus on extracting relationship triplet (or a list of triplets). Relationship triplet is defines as: (entity1, relationship, entity2). Can Bill prototype a function that uses llm.model_of_your_choice to takes a chunk of text, and output a list of relationship triplets?

Can someone give me admin/mod access for this repo?

I have a Mixtral 8x7b running on server COSMOS0001. I had made an account for you with the username bill (I will send you the password on slack). Currently, Mixtral is the top open source model in terms of performance relative to its size. Here's a code snippet for using it. Please integrate this API route to your backend system:llms. This will give us an additional option down the line...

def ask_mixtral(messages: list[dict]) -> dict:
    """Ask mixtral with a data package.

    Example input: [{"role": "user", "content": "Hello world example in python."}]
    """
    url = "http://cosmos0001.chtc.wisc.edu:11434/api/chat"
    user = "bill"
    password = os.getenv("MIXTRAL_PASSWORD")
    data = {
        "model": "mixtral",
        "messages": messages,
        "stream": False,  # set to True to get a stream of responses token-by-token
    }
    # Non-streaming mode
    response = requests.post(url, auth=requests.auth.HTTPBasicAuth(user, password), json=data)
    response.raise_for_status()
    return response.json()["message"]["content"]

additional docs: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-chat-completion

Provide end-to-end example with Devesh eval's input, and return the results to Devesh.

@bxia68
@shivaram
@iross
@cambro
@davenquinn

In Bill's setup, the initial section mirrors our primary ask-xdd deployment, essentially being a retriever. It appears more efficient to merge these two. This would allow Bill to utilize the hybrid retrieval endpoint, avoiding the need to manage a separate retrieval system. His efforts could then be concentrated on the 'extract factsheet/graph from text' component.

This leads to inquiries regarding the data source for the 'factsheet-generator'. If it's specifically sourced from an xdd dataset, I propose integrating this data into ask-xdd.