qpxdesign / trec-ikat Goto Github PK

just to see how that changes which passages are selected

passage summaries are first generated in chatgpt where we use them to check passage relevance, and if the passage passes that check, we regenerate them for combined_passage_summaries

https://github.com/castorini/pyserini/blob/master/docs/usage-search.md

should turns all be in one array or should turns be in a nested array for each topic?

"generated_prompt": "No, I think that's all for now. Thank you for your help!",
"text": " Sure, here is a response to the question "No, I think that's all for now. Thank you for your help!" using the information provided earlier:"You're welcome! It was my pleasure to assist you with your gourmet dishes and plating techniques. If you have any other questions or need further guidance in the future, don't hesitate to reach out.",

EXAMPLE: “Sure, I'd be happy to explain how to properly saute food!Sauteing is a cooking technique that involves quickly cooking ingredients in a small amount of oil or fat over high heat. This method helps to preserve the nutrients in the food, especially heat-sensitive vitamins and minerals like vitamin C and B vitamins. To properly saute food, follow these steps:1. Choose the right pan: A stainless steel or cast iron pan is ideal for sauteing, as they retain heat well and distribute it evenly.2. Add oil or fat: Use a small amount of oil or fat with a high smoke point, such as olive oil, avocado oil, or ghee. This will help to prevent the oil from burning or smoking during cooking.3. Heat the pan: Preheat the pan over medium-high heat for about 2-3 minutes before adding the ingredients. This will ensure that the pan is hot enough to quickly cook the food.4."

EXAMPLE: "Congratulations on getting an interview with Google! That's definitely something to be proud of. I can give you some general tips that might help you prepare for your interview.

First, make sure you understand the basics of the company and the position you are applying for. Research is key in preparing for an interview. Look up information about Google on their website and other reputable sources to get a better understanding of what they do and how they operate. This will also give you an idea of what they might be looking for in a candidate.

Next, think about your own skills and experiences that could relate to the position you are applying for. Consider specific examples of projects or work that demonstrate these skills. Prepare concise and clear explanations of how you applied these skills in real-world situations. It's also helpful to have a few questions prepared in advance to ask your interviewer(s). This shows that you are genuinely interested in the company and the position, as well as any other relevant details you might want to know.

Finally, relax! It's normal to feel nervous before an interview, especially if it's your first one. Remember to (cuts off)"

Example: Here are some good ways to loose weight: (end of output)

potential fix: use double llm method

user input & PTKBs →LLama2 response →BM25/Pyserni to get passages →use Fastchat T5 Summerazation LLM to summerize passages →gen response from summaries and LLama2 response

when llama-cpp-python is generating a response, only ~1200mb out of 8000mb of VRAM gets used, potentially leading to increased gen times

final answer was just blank

example: "Sure! Based on the passages provided, here are some recommendations..."

sometimes it takes multiple cycles for llama to elaborate enough on the question to get relevant keyworks that can actually find good passages

Example:
"user_utterance": "Can you eat fish in any of them?"
"preliminary_response": " No, none of these diets allow for fish consumption because all are strictly vegan."
"secondary_response": All of the Vegan Mediterranean, Vegan Keto, Eco-Atkins, and Ornish Diets diets mentioned before are strictly plant-based and do not include any animal products, including fish. So, if you're a vegetarian who excludes fish, these diets may not be suitable for you. The Ornish Diet is the only one of these diets that allows for limited exercise. The others all emphasize whole grains, legumes, and vegetables with minimal or no animal products. It's important to note that while transition diets allow for some animal products, they are designed to help individuals gradually move towards a strictly plant-based eating plan. Pescatarians include fish, eggs, dairy products, and vegetables, while vegans exclude all animal products, including fish. Vegetarian diets offer flexibility in terms of the types of food consumed, while vegan diets are more restrictive but offer a broader range of options. Ultimately, choosing a diet that aligns with your personal preferences and health goals is important.

possible solution - keep finding passages until we're finding passages of a certain score / use LLM relevancy check

User Question: I prefer a natural diet, not a pill-based diet. Which of the aforementioned ones is natural?
PTKBs Chosen:
{
"id": "5",
"text": "I'm vegetarian.",
"score": 0.3468347191810608
},
{
"id": "10",
"text": "I'm an Android user.",
"score": 0.309503972530365
},
{
"id": "6",
"text": "I'm lactose intolerant.",
"score": 0.2509726285934448
}

User: “Ok, good info. Now, tell me about the next option of weight loss.”

Chatbot: “I would suggest that the author of the hCG Diet Workbook for Phase 2 seek legal advice from a solicitor specializing in consumer law to understand their rights and options for recovering their money. They may also want to consider consulting with a financial advisor or accountant to assess the feasibility of their business plan and to explore potential tax implications. Additionally, the author could reach out to local business organizations or entrepreneurial networks to seek guidance on how to navigate this situation and protect themselves from future financial loss. Finally, it may be beneficial for the author to speak with a mental health professional to address any emotional stress or anxiety they may be experiencing due to this situation.”

Likely Cause: Bad passage selection