paullerner / pyannote-db-plumcot Goto Github PK
View Code? Open in Web Editor NEWThis project forked from benjisympa/pyannote.db.plumcot
Not yet ready for prime time.
License: Other
This project forked from benjisympa/pyannote.db.plumcot
Not yet ready for prime time.
License: Other
add loader in Plumcot/loader/loader.py
so that one can access to episodes summaries, synopsis and keywords through the API.
See #52
Before releasing the corpus, we should provide example on how to load transcripts (and their forced-aligned version when available).
This probably means that we need to implement a dedicated API for that in the pyannote.database plugin.
>>> from pyannote.database import get_protocol
>>> protocol = get_protocol('TheBigBangTheory.????.Transcription')
>>> for file in protocol.train():
... transcription = file['transcription']
... for line in transcription:
... line['speaker']
... line['text']
... # other fields of interest? time?
Looks like deepspeech-gpu
0.7.3 can no longer be installed (at least on my macOS).
Where is this dependency used? Can we make it optional?
pip install ./pyannote-db-plumcot
ERROR: Could not find a version that satisfies the requirement deepspeech-gpu==0.7.3 (from pyannote.db.plumcot==0+untagged.455.g0b364bc) (from versions: none)
ERROR: No matching distribution found for deepspeech-gpu==0.7.3 (from pyannote.db.plumcot==0+untagged.455.g0b364bc)
TODO rely on __init__
path everywhere
Example with TheWalkingDead.Season01.Episode02
Here "mom" is annotated as "theodore_douglas"
Here "Lori" is not annotated in "labelDoccano" but is (automatically) in "referenceTo"
Also there "he" is annotated as "UNKNOWN" in "referenceTo" (and absent in "labelDoccano") whereas in this case it's easy to see that "he" refers to "shane".
The first two lines come from nowhere, they're not in the original transcript
That explains the mix-up described above.
I'm aware that the json format is not user-friendly, but there you can see that "mom" is annotated as "lori_grimes", but "lori" and "amy" are not annotated
This is also intended as a documentation for the current script (more or less translated from @sharleynelefevre).
See also #13 for thoughts about a better annotation.
entities.py [--uri=<uri>]
-> defaults to process all series. You currently have to hard-write the serie uri and season.Plumcot/data
like in the other scripts, this can be done using the path of the installed Plumcot
package in a few lines of code:import pyannote.database
import Plumcot as PC
from pathlib import Path
DATA_PATH=Path(PC.__file__).parent / "data"
You currently have to hard-write input path and output path. Note that the output path is not consistent with the actual structure and should be Plumcot/data/<serie>/csv_semi-auto-annotation
.csv
, or, better, follow standard formats (e.g. .conll
)transcript with speaker names (e.g. Plumcot/data/TheBigBangTheory/transcripts/TheBigBangTheory.Season01.Episode01.txt
)
semi-automatically annotated transcript in a .conll
and a .csv
formats.
uri
.conll
format is intended for Doccano, I'm not sure what good is .csv
.conll
file (previous output)
.conll
file without tabs at the end of the line so it's suitable for Doccano
Note this is not in the entities script. You need to:
See CONTRIBUTING.md for annotation instructions.
.conll
file (previous output)
.json1
files, manually annotated (i.e. corrected)
.json1
file (previous output)
.csv
file. Note that this has nothing to do with the previous .csv
format: this one only has three fields : "idSent", "idChar" and "label".
.csv
file (previous output).csv
file (output of semi_auto_loc_annotation).Yet another .csv
file which merges the two inputs. This is the format described in CONTRIBUTING.md and which is used in Plumcot.loader
.
Note this is insanely slow, roughly 15 minutes to merge 2 files in one.
.csv
file (previous output)
Evaluation of the automatic annotation
How to reproduce?
from pyannote.database import get_protocol
protocol = get_protocol('TheBigBangTheory.SpeakerDiarization.0')
test_file = next(protocol.test())
test_file['entity']
# AlignmentError: [...] are different texts.
Is this a known issue?
Is this related to your suggested changes in #13?
These would prevent a lot of hacks currently present in https://github.com/PaulLerner/pyannote-db-plumcot/blob/video/Plumcot/loader/loader.py and https://github.com/PaulLerner/pyannote-db-plumcot/blob/video/scripts/ner.py :
whitespace
for each token which represents the whitespace following the token, might be ''
)There might be a need to make BreakingBad and GameOfThrones annotations uniform:
Note: it might be easier to do that using the original serial speaker dataset and not the rttm version, see https://github.com/PaulLerner/Prune#convert
l
instead of I
(e.g. l am a TV character.
). This led to poor POS-tagging and is fixed in Plumcot.loader.CsvLoader
.Since 123e37c, a 1-1 mapping is done between the actor and the character so that an actor can only play one character. This was done to circumvent the fact that actors could be credited several times in movie series (e.g. TheLordOfTheRings) and then it would broke the episodes.py
script that generates the credits.txt
file from characters.txt
and episodes.txt
. Only now it's tricky to update the annotations prior to 123e37c (as some names have changed), see #19
Another solution would be to refactor the episodes.py
script and the credits.txt
file (I'm not sure why we have this huge unreadable matrix in the first place) so that one actor can play several characters and a character can be played by several actors. However this implies fixing all usage of the credits.txt
file...
We could then:
close #19
revert:
fix TheLordOfTheRings, either:
same for BuffyTheVampireSlayer but it's a lot easier, the only name that has changed is janice
→ janice_penshaw
(which is inexistent for all I know)
update README and CONTRIBUTING
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.