Comments (2)
shahules786
commented on June 15, 2024
Hi @ColabDog , context recall is the measure of how much of the required context is actually captured. So it is not possible to calculate it without some ground truth. Since in our case, we wanted to estimate this from the ground truth answer and not the ground truth context we use the method of identifying how the ground truth is captured in the retrieved context.
from ragas.
jjmachan
commented on June 15, 2024
closing this since the question has been answered
from ragas.
Related Issues (20)
- embedding nodes: 0%| Segmentation fault (core dumped)
- TestsetGenerator -> RuntimeError: ... got Future <..> attached to a differen t loop HOT 3
- Adapted output keys set(output.keys())={'深度', '相关性', '清晰度', '结构'} do not match with the original output keys: output_keys[i]={'structure', 'clarity', 'depth', 'relevance'}
- Answer Correctness giving wrong results for batches and single records HOT 1
- Testset generation ValueError: invalid literal for int() with base 10: HOT 1
- Is it possible to add an argument to the evaluate() function to configure the group name? HOT 1
- Un-deprecate multiple ground truth answers? HOT 1
- Grammar and punctuation improvements in critique prompts HOT 3
- [R-255] remove `async=True` from ragas Evaluate
- Tried Generation Test Set from Together APIs and Hugging Face Embeddings HOT 2
- [R-256] make better example dataset for getting started
- [R-259] Which is the best LLM for evaluation? HOT 1
- Contradiction in evaluate is_async parameter docstring and code HOT 3
- Can this part of the code be applied to Chinese scenarios HOT 1
- is_async missing in context_relevancy in ragas 0.1.8 HOT 1
- division by zero when computing output score
- [R-262] ImportError: llama_index must be installed to use this function. Please, install it with `pip install llama_index`. HOT 1
- How to save reason of each evaluation metrics? HOT 1
- Using context utilization without ground truth throws error message
- Evaluation Error with Gemini/VertexAI: InvalidArgument due to Empty Text Content
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ragas.