Comments (6)
@KerolosAtef @avinash31d , Thank you for your interest in our work. Please find the details about the Vicuna-based quantitative evaluation benchmark here: https://github.com/mbzuai-oryx/Video-LLaVA/tree/main/quantitative_evaluation.
from video-llava.
@KerolosAtef We attribute this to the randomness introduced by the temperature parameter in both the tested model and the LLM used for evaluation. This will be addressed in our future work.
from video-llava.
+1
from video-llava.
thank you very much, but also the Vicuna model doesn't output the same results for each run.
I have tried to reproduce some of the results of video chat GPT and this the results:
ActivityNet : Acc :36.13 instead of 40.8
TGIF: Acc: 63.07 instead of 66.5
from video-llava.
okay good,
I want to make sure of something, for the Zeroshot datasets (MSVD, MSR-VTT,Activity_net,TGIF) Are you used the testing data or the validation data?
from video-llava.
We follow the same approach as Video-ChatGPT, i.e. using test splits.
from video-llava.
Related Issues (16)
- Using ASR caption instead of heavy audio encoder can be more efficient
- Flash Attention
- Demo on Gradio
- CLI Demo can be me made much simpler by adding more instructions in the README.md section
- RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
- Is 8 cards 4090gpu (24g) enough to train your model?
- Time codes
- License
- Weight link not available
- Comparison between running the model with grounding and without Grounding.
- When will the code available? HOT 1
- Training Details HOT 1
- Segmentation Error
- Error while loading tokenizer HOT 1
- requirments conflicts for whisper-at and torch 2.1.0 during installation HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from video-llava.