Comments (4)
gabriead
commented on June 15, 2024
I can offer a debugging session with the authors/contributors if that's helpful.
from ragas.
shahules786
commented on June 15, 2024
Hi @gabriead , thanks for raising this issue. We are continuously working to improve the quality of metrics. We do have some updates coming soon but would certainly benefit from your input. If you wish to do this over a call I can personally write to you over an email. 😊
from ragas.
jjmachan
commented on June 15, 2024
Hey @gabriead as shahul said we are continuously working to improve the quality of the metrics in ragas and really appreciate your offer to help us debug with a combined debugging session - would be super helpful for us. I've sent a mail to the mail-id in your github but if that is not the correct one do let us know your mail id or write to us at [email protected]
looking forward to meeting you Adrian and having a nice debugging session :)
from ragas.
jjmachan
commented on June 15, 2024
@gabriead we have made a lot of improvements to ragas now for reproducibility so this should have improved. Please feel free to reopen if this is still an issue.
@shahules786 shall we close this?
from ragas.
Related Issues (20)
- [R-222] linear bug report HOT 1
- [R-225] test issue HOT 1
- [R-227] Add an MLFlow Integration
- Question: ragas built-in visualization of results (e.g. HTML exports)? HOT 2
- [R-244] add ragas as part of CI/CD for users with github action
- Faithfulness vs. context recall HOT 2
- LLM generates trimmed JSON for context recall and faithfulness metrics HOT 2
- [R-247] Integrations: wandb and wandb tracer HOT 1
- [R-248] setup Devin in Ragas
- I want to know how Testset Generator to create dataset works. Can anyone give like flowchart or anything. I wanna know the high level understanding HOT 5
- value error: probabilities are less than 0
- documentation on cosine similarity range is wrong HOT 1
- [R-249] metrics reproducibility: better statement generation
- Init attribute is mutable and shared (not intended?)
- Get Started section for "Generate a Synthetic Test Set" is broken HOT 2
- Question of computing Context Relevancy
- [R-250] metrics reproducibility: improve answer_correctness
- ExceptionInRunner: The runner thread which was running the jobs raised an exeception. Read the traceback above to debug it. You can also pass `raise_exceptions=False` incase you want to show only a warning message instead HOT 2
- RAGAS compatibility with mistral models
- httpcore.ProxyError: 407 Proxy Authentication Required
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ragas.