opennewslabs / autoedit_2 Goto Github PK

A ? is included in the video sequence portion of the export menu. It currently directs users to section (4) Reconnect in video editing software of choice of the manual. However, because of its location in the export modal, it seems that it would make sense and give users more context to direct to the previous section in the user manual, (3) Exporting a video sequence (EDL).

1. Open autoEdit 2 app
2. Open any transcript
3. Click Export to open menu
4. Click ? in video sequence section
5. Observe users taken to (4) of user manual (http://www.autoedit.io/user_manual/usage.html#reconnect-in-video-editing-software-of-choice)
6. Note being directed to (3) might make more sense

Users have the ability to select/deselect portions of the transcript by highlighting text. Currently, however, selecting text and clicking outside of the highlight doesn’t always persist. It appears that users must click completely outside of the transcript for the selection to remain, which isn’t immediately apparent.

1. Open autoEdit 2 app
2. Add file to have access to transcript
3. Open transcript show page
4. In highlight mode, begin selecting text
5. Observe text is highlighted
6. Click to remove highlight
7. Note selection does not persist

EDL file verification 2 (Chronological order)
Below are the details and files needed for EDL testing via video editing software.

Video: norman_door.mp4 -- one of the test files you sent QA

Selected text: Norm_long_TTEDLC_13_12_2016_15-19-05.txt

EDL file: Norm_long_chronologicalEDLCO_13_12_2016_15-18-41.edl.zip

A Clear highlights option allows users to remove all selected text within a transcript. If a user clears all highlights via this button then navigates away from the transcript and returns, the changes do not persist.

1. Open autoEdit 2 app
2. Add file to have access to transcript
3. Open transcript show page
4. Highlight sections of text
5. Use Clear highlights option
6. Observe highlights are cleared
7. Navigate away from transcript and return
8. Note highlighted sections are back

EDL - Chronological order
Below are the details and files needed for EDL testing via video editing software.

Video: First_Town_Hall_For_Transcription.mp4 -- one of the test files you sent QA

Selected text: TC plain text EDL chron Town_Hall Video_EDL_13_12_2016_14-44-36.txt

EDL file: EDL (Chron) Town_Hall Video_chronological_13_12_2016_14-31-31.edl.zip

I was just curious why IBM watson was used as the API for this tool? (I'm not considering Gentle which is described as not being so performant.)

Watson works great but I guess it could add quite some value to integrate other APIs as well. For example, Google's speech API supports 101 languages and offers 1 hour of free transcription per month. There are probably quite a few other APIs available as well, and more are bound to come.

As far as open-source is concerned, I also wondered about alternatives to Gentle? For example Tensorflow open-sourced by Google (AFAIK).

When creating new credentials with IBM Watson there seem to be an authentication problem.

Even when following the user manual and Bluemix instructions

However when using previously created (older credentials) it seems to work fine.

I found this PR in the IBM Watson Node SDK watson-developer-cloud which may or may not be related to the problem.

EDL - Chronological order
Below are the details and files needed for EDL testing via video editing software.

Video: Pietro_test.mov -- one of the test files you sent QA

Selected text: Pietro_Gentle_EDL_14_12_2016_10-43-51.txt

EDL file: Pietro_Gentle_chronological_14_12_2016_10-43-18.edl.zip

After @probonopd help in making a Linux version, see issue #36, for 1.0.8 release, and @sedubois, @jsdephillips, @sqr interest in a Windows version, thought I'd open a separate issue to see what it would take to make a Windows version.

Reporting here things that might need to be done to do Windows version

• setup ffmpeg and ffprobe dependency binaries for Windows version. ffmpeg-static, ffmprobe-static.
• setup electron build process for ffmpeg and ffprobe to package right dependency depending on the operating system, osx, linux and windows.
• Adjust path across the app so that / in osx and linux can be auto transformed to \ in Windows, by path module.
• Compiling Windows version, test out if can do on Mac or if need Virtual machine or actual Windows PC. To see what else breaks, error message in console, and iterate accordingly.

Upon opening the app, users are brought to a landing screen that lists transcripts if they are available. In the nav are links view transcripts, add new files or view help options. Also included is “autoEdit 2” text. This text currently displays as a link when moused over but does not direct users anywhere.

1. Open autoEdit 2 app
2. View options in nav
3. Hover over “autoEdit 2” text in nav
4. Observe text displays as link
5. Click text
6. Note link does not direct anywhere

As raised by one of the users when exporting an EDL file (paraphrasing)

for videos longer than 15 minutes it starts drifting out a sync of 30 to 20 seconds. In recognising the in and out points of the video in the EDL. Losing Accuracy.

To reproduce this issue
Get a video file NTSC, 30fps

1. Add video to autoEdit
2. Get a transcription
3. Scroll towards the end of the transcription and select one of the last lines
4. export EDL and reconnect in premiere (or other editing software of choice)
5. When reconnecting sequence chose NTSC, 30fps in the sequence.
6. See if the video segment correspond to the text selection in the app.

If bug occur, expecting it to drift of and not be same selection.

Further test. with same EDL file.

1. import in premiere, but this time when chosing sequence settings chose PAL 25fps
2. See if the video segment correspond to the text selection in the app.

From previous tests expecting the text selection in the app to match video segments in the EDL sequence using when choosing PAL and 25fps

Considerations

The timecode module used in the EDL composer has a default setting on 25fps.

Possible fix

Allow to pass in fps to timecode as a variable and give the user the option to export the EDL as a NTSC sequence?

Need to look more into how timecodes and PAL/NTSC work.

On the transcript show page users can access info about the specific file. Currently, longer strings of text (e.g. file path names) can extend outside the designated borders.

1. Open autoEdit 2 app
2. Add file to have access to transcript
3. Open transcript show page
4. Observe info button under video
5. Click info to reveal dropdown
6. Note text can extend past borders

First, great software and it's already improving my productivity.

Simply, I'd like finer grained control over playback speed. At least a -.75 option, as -.5 is laboriously slow. Is this already configurable?

When entering edit mode a notification displays at the top of the page with instructions. It reads “Click on a word to edit. Use cmd + a to select a word. Tab or click to move to the next word.” However, using cmd + a in edit mode doesn’t appear to do anything. Clicking a word automatically allows users to edit. It may be beneficial to remove cmd + a from the instructions.

1. Open autoEdit 2 app
2. Add file to have access to transcript
3. Open transcript show page
4. Enable edit mode
5. Observe instructions
6. Attempt to use cmd + a to select a word
7. Note clicking word allows it to be edited and cmd + a does nothing

I've been asked a few times about this, that I thought I'd make a note on the current workaround.

If a transcription is stuck with the spinning wheel after 5 minutes or so, it is safe to assume something went wrong, that did not raise any errors and the incomplete transcription is not going to be processed further, but there is also no option to delete it (button is disabled).

One way to delete all the incomplete transcriptions is to

1. open the console of the autoEdit app by pressing on alt cmd i (all at the same time).

2. in there copy and paste the following code

for (i = 0; i <  appTranscription.transcriptionsList.models.length; i++) { 
    if(!appTranscription.transcriptionsList.models[i].attributes.status){
appTranscription.transcriptionsList.models[i].destroy();
    }
}

3. hit enter

4. if it hasn't delete all the incomplete once, repeat.

From a development point orview, one way to improve on this, the backbone view needs to be
changed to add some logic to make the transcription deletable if it is still marked as processing
after 5 minutes or so. (better do 6 or 7 to stay on the safe side tho).

Users have the ability to export text selections in a variety of options, including in Chronological order and Selection order. Currently, all exports list selections in chronological order despite choosing Selection order.

Note: This was tested via Plain text and JSON options

1. Open autoEdit 2 app
2. Add file to have access to transcript
3. Open transcript show page
4. Click Export to view options
5. Export via plain text using selection order
6. Open exported file
7. Observe selections listed in chronological order

The autoEdit app currently allows unsupported file types to be uploaded without displaying an error message. This results in a continuous loading wheel and the file cannot be deleted.

1. Open autoedit app from applications or dock
2. Click New
3. Click Choose File
4. Upload unsupported file (CSV or doc)
5. View app attempting to load file displaying no error message
6. View continuous loading wheel

Is your feature request related to a problem? Please describe.
I would like to use AWS's Transcribe as one of the STT APIs.

Describe the solution you'd like
Please implement AWS Transcribe as one of the STT APIs

Describe alternatives you've considered
There are other services which do this, including Google's Cloud Speech to Text.

Additional context
https://aws.amazon.com/transcribe/resources/

After exporting SRT subtitles a first time with Transcriptions > Export > Captions, the button doesn't do anything anymore when trying to export a second time. The same behavior does not occur e.g when exporting EDLs. I don't know any workaround. Using macOS 1.0.10.

1. Open autoEdit 2 app
2. Add file to have access to transcript (e.g. norman_door.mp4 using Gentle)
3. Open transcript show page
4. Click Edit
5. Select a word to edit (e.g. changed fox to vox)
6. Click Export button

autoEdit2 app

Expected: exported file should display edited transcription
Actual: exported file still has original transcription
Plain text selection: Norman_Gentle_EDL_14_12_2016_13-55-05.txt

Hi Peitro, amazing project!

Small tip: If other users encounter a spinning icon that doesn't stop in autoEdit 2's transcriptions window, it could be related the to incorrect IBM username / password. IBM Bluemix's website lists the password FIRST before the username.

I uninstalled as you recommended, reinstalled, input the IBM username / password and my video processed shortly.

As the title says, I tried Watson for the first time and set it up in the Germany region, but then the watson subdomain needs to be suffixed with -fra (Frankfurt, I guess), which autoEdit2 doesn't take into account and just throws an unauthorized error.

Would be great to either be flexible on the region or at least give a tip that the UK region is required.

Describe the bug

Unable to edit credentials in Adobe CEP Panel

To Reproduce
Steps to reproduce the behavior:

1. Go to Premiere, click on window, extensions, autoEdit2 to load the panel
2. Click on the cog wheel ⚙️ to open the settings view in the autoEdit panel
3. click on any of the STT services, Speechmatics, IBM , etc..
4. when clicking on the input field, unable to add text/edit/delete text.

Expected behavior

Expected to be able to edit text in settings panel input

Desktop (please complete the following information):

• OS: Tried on Mac, but was told same happens in Windows.
• Version of your operating system: Mac OS Sierra 10.12.6
• Version of autoEdit: 1.0.12 autoEdit, 1.0.1 adobe Panel.

Additional context
Add any other context about the problem here.

You can add title text and description in create new transcription view. so comparing the two views might be a good way to figure out what is the issue.

For now. autoEdit desktop and autoEdit Panel, share the same database, and user folder in the system library, so as a workaround, downloading the desktop app, and adding credentials there, should reflect in the Adobe Panel. - you will need to close and re-open the panel.

Per the test plan, there is a known issue where the preview does not self-update when the video is done processing and users must go back to the list and then open the transcript for it to display. Currently, however, it appears that the preview does not display for larger video files even when navigating away and returning. It remains “Video is being processed” indefinitely.

File sizes where this occurred thus far:
193MB
392MB

1. Open autoEdit 2 app
2. Upload large video file
3. Allow processing to complete
4. Open transcript
5. Observe video preview is still processing
6. Navigate away and return
7. Note preview processes indefinitely

The file in the below example was uploaded on Friday and still shows as processing:

test issue to try things out with waffle.io

Sometimes, the Metadata isundefined, and is therefore not possible to open the transcription as it raises error when not finding the attributes.

Cannot reproduce systematically. so perhaps race conditions with metadata read module, ffprobe binary, or some misuse of variable scope in interactive transcription generator module?

Needs more investigating.

Users are able to export highlighted sections of transcript for editing by clicking on the Export button. However, when I exported this video's Plain text transcription, the whole transcription was exported instead of the lines I highlighted. Note: I had highlighted text in revchron (bottom to top)

Video: norman_door.mp4
Lines selected in autoEdit2:

Exported selected text file: Norman_Gentle_14_12_2016_11-20-22.txt

EDL file: Norman_Gentle_chronological_14_12_2016_11-08-22.edl.zip

As originally raised by @Riptwo on twitter.

does autoEdit2 use the X-Watson-Learning-Opt-Out true header for Watson transcriptions?

Shotr answer:

No.

Long answer:

It uses the watson-developer-cloud node module. But I haven't specified that option (I think maybe it wasn't around when I first did it)

I see now it has been added as an option in the SDK node module

At the moment as mentioned there, by default all request are logged

Possible fix:

To implement, it be a matter of adding

var myInstance = new watson.WhateverServiceV1({
  /* username, password, version, etc... */
  headers: {
    "X-Watson-Learning-Opt-Out": true
  }
});

Where the Watson STT instance is initialised changing it as follows to add the headers informations.

  speech_to_text = watson.speech_to_text({
	    username: keys.username,
	    password: keys.password,
	    version: 'v1',
 	    headers: {
                 "X-Watson-Learning-Opt-Out": true
              }
	  });

Things to think about for integration (ie going beyond hard coding a solution):

In the UI/UX It would also be good to add this as an option when creating the transcription, or in the settings where the credentials are added to give more fine grained control to the user on whether to opt in or out, rather then hard coding it.

Help and feedback wanted.

When on the transcript show page, a notification displays above the transcript letting the user know which mode is being used -- highlight or edit. Currently, there is a typo when in highlight mode.

It reads “You are in hilight mode.” In this instance hilight should be changed to highlight.

1. Open autoEdit 2 app
2. Add file to have access to transcript
3. Open transcript show page
4. Ensure highlight mode is active
5. Observe notification at top of page
6. Note typo of “hilight”

Vimeo recommends WebVTT as subtitle format, so it would be nice if autoEdit2 could export in this format instead or additionally to SRT.

Hello,

I am using Auto Edit 2 for a documentary and my intern started the transcription process. I have noticed that the transcripts are using 'running time' instead of the 'actual time-code' embedded in the clips. When I look at the clip info the time-code for the clip is correct. Is there a way to have the actual time-code in the transcripts? This will be an issue when trying to link clips created in the EDL with paper edit in your program as the time-codes do not match.

Next, we made a test EDL with paper edit but when I load it into the AVID Media Composer there are no clips in the timeline. I exported a text version of the same EDL and there are events in the EDL.

Also, is there a way to differentiate different speakers? We have several interviews that are 2 people. But all the interviews have the producer asking the questions. In any case it would be helpful if we could add the name of the main person speaking when we start a new transcript.

Any suggestions would be welcome.

Than you.

Jacquelyn

The transcriptions dashboard is a list of all processed transcripts. Currently, there is no way to sort these files. This could pose problems if there are a multitude of uploaded transcripts, as the user would have to scroll through all files to locate the desired transcript. For convenience, it would be nice to be able to sort these, possibly via Title and/or Date.

1. Open autoEdit 2 app
2. Upload several files and allow them to process
3. Return to dashboard
4. Observe no way to sort uploaded transcripts

Users have the ability to export full transcripts via the Plain text transcription and Timecoded plain text transcription options. It may be beneficial to add “full” to the Plain text transcription description (as in the Timecoded option) for further clarity to users.

“You can export the text of the full transcription as plain text without timecodes.”

1. Open autoEdit 2 app
2. Add file to have access to transcript
3. Open transcript show page
4. Click Export to view options
5. Observe description for Plain text transcription
6. Note “full” missing from description

Sometimes, a transcription finishes to process in the transcription list UI and when you click, it does not take you to the transcription page, and in the console it shows an error saying the filename in metadata is not defined.

But then if you leave it enough time, after a bit you are able to click again and be taken to the transcription page.

In this scenario it seems to be because in db.js the callbacks from theinterative_transcription_generator returning a transcription, the metadata info, and a video preview file, might not happen the order one would expect. Eg reading the metadata might take longer etc.. And the transcription callback is the one setting the transcription status to true to mark it as done. Which as a result makes the transcription in the transcription list clickable.

One possible solution is to add a few extra status attributes for the individual callbacks to the transcription json model.

eg with default values

metadataStatus = false
videoStatus = false
transcriptionStatus = false

Then in "pseudocode"
for transcription calllback

if metadataStatus == true && videoStatus == true
transcriptionStatus = true

for video preview callback

if metadataStatus == true && transcriptionStatus == true
videoStatus = true

for metadata read callback

if transcriptionStatus == true && videoStatus == true
metadataStatus = true

Users have the ability to export a JSON version of the transcript in either EDL Chronological or EDL Selection Order. Currently, EDL Chronological is listed twice in this section. One option should list EDL Selection Order per its description.

1. Open autoEdit 2 app
2. Add file to have access to transcript
3. Open transcript show page
4. Click Export to view options
5. Observe JSON section
6. Note duplicate EDL Chronological options

A confirmation popup displays when attempting to clear highlights where the user can either cancel or proceed. There is currently a minor typo in this text where “highlights” should be changed to “highlighted” for better readability.

Current: “...clear all of the highlights sections from this transcript.”
Edited: “...clear all of the highlighted selections from this transcript.”

1. Open autoEdit 2 app
2. Add file to have access to transcript
3. Open transcript show page
4. Highlight sections of text and use Clear highlights option
5. Observe popup displays
6. Note typo “highlights” instead of “highlighted”

When viewing a transcript the associated video displays on the left side of the screen next to the text. The video player is currently very small, which makes it difficult to track through the video, adjust volume, etc., with accuracy. Additionally, the fullscreen button is cut off.

While there are other methods to navigate through the transcript and adjust volume (clicking text in transcript, using shortcuts), it may be beneficial to increase the player size so users can successfully and accurately control the video.

1. Open autoEdit 2 app
2. Add file to have access to transcript
3. Open transcript show page
4. Observe video on left side of screen
5. Note size of player, controls and difficulty in using

Is there any reason not to publish a Linux version?

As said in the title, sometimes you delete a transcription and then after a while, eg when you reopen autoEdit, it comes back in the list.

Best guess, a database issue, to do with caching?

Todo

Check if it's a known issue in these two components

• linvodb3
• medeadown

Following up from #44 it be good to find a way to do same as for Linux version #36 where there is auto deployment to create the windows version as well. (and Ideally for Mac too)

On the Transcript Show page a user can export the transcription by clicking the "Export" button.
In the Export menu modal, one of the options has a typo.

1. Open autoEdit 2 app
2. Add file to have access to transcript or open an existing transcript
3. On transcript show page, click Export button
4. On Export menu modal, scroll to Plain Text Selections

Expected: Plain Text EDL Chronological
Actual: Palin Text EDL Chronological

This would be a way a round to make sure that console shortcut cmd + alt +j
does not ovveride other app such as chrome shortucts.
Could use focus and event lister to add and remove global shortcut from app when is not in focus

http://docs.nwjs.io/en/latest/References/Window/#synopsis

//get the window 
var gui = require('nw.gui');
var win = gui.Window.get();
//listner
win.on('focus', function() {
    console.log('New window is focused');
  });

in lib/app/app.js line 60.

Another option is to use mouse trap js. Which is what is used for the keyboard shortcuts in transcription show page. Would need to figure out a way to set this so that it works on every window.

A confirmation popup displays when attempting to clear highlights where the user can either cancel or proceed. If the user cancels, the associated popup does not make sense. It reads, “relax, nothing was cancelled.”

It appears it should read “relax, nothing was cleared.” or something similar.

1. Open autoEdit 2 app
2. Add file to have access to transcript
3. Open transcript show page
4. Highlight sections of text and use Clear highlights option
5. Press cancel
6. Observe popup
7. Note text

There is a Search field on the Transcription Show page, currently the field does not return any results if a user adds text to the field.

1. Open autoEdit 2 app
2. Add file to have access to transcript or open an existing transcript
3. On transcript show page, enter text in the Search field

Expected: Entered text that matches text in the transcription should be highlighted
Actual: No text is highlighted, error message 'Uncaught ReferenceError: searchTextArray is not defined`

If a user enters incorrect or invalid IBM credentials there is no way to remove them and enter updated credentials - even after deleting and reinstalling the mac app.

Hi, I am a graphic designer, I want to help others in graphic design.

After I reviewed your project, you have no logo on this project. Therefore I want to contribute to this project by creating a new logo / icon. what do you think?

1. Open autoEdit 2 app
2. Add file to have access to transcript (e.g. JesselynRadackonVimeo.mp4)
3. Open transcript show page
4. Highlight sections of text (e.g. i found that)

Expected: only the selected text is highlighted
Actual: text above selected text is also highlighted, unselecting unintended highlighted text will remove highlight from both lines

Thanks for this tool! We're creating guided meditations for Imagine Clarity and it allows us the following workflow:

1. record ad libitum in English
2. transcribe with autoEdit then export transcription to plain text
3. edit text as needed somewhere else
4. record English audio professionally with the edited text as visual support
5. translate edited text to French
6. record French audio similarly

In our case, basically we don't need the paperedits feature, as each recording is independent, and each recording contains almost exactly what we need for the transcription, except for some words or sentences which are repeated when the speaker wants to rephrase his speech. So what we can use is the Transcriptions > Edit tab.

However, we are then forced to do step 3 (text editing) outside of it as there is limited text editing possible, but we lose the benefit of having a seamless sync between speech and text, i.e. we can't do a text search in the audio, etc.

With some more capabilities, the tool could offer a fantastic editing experience (and thus be true to its name 😉):

• reintroducing punctuation and sentence boundaries (as special characters "between" the words) and to add/remove line breaks. So that the text can be re-organized in a coherent sequence of paragraphs with proper, grammatically correct sentences.
• moving the text cursor around (between and within words) by using the keyboard arrow keys. Currently you can only navigate within each single word. Would be great to seamlessly move in and out of the words just with the keyboard.
• selecting several words with a drag of the mouse or a double/tripe-click, just like we would select text in a text block. Then being able to delete the selected text when hitting backspace. There could be two deletion modes, just deleting the detected text, or deleting the text and the corresponding audio from the source. This second deletion feature isn't needed for ad lib audio, as we re-record afterwards, but it's needed to edit the final recording.
• allowing to increase the threshold amount of silence used to decide when words are displayed on a new line. As we do guided meditations, the speech tends to be slower, with "contemplative" or "reflexive" pauses in the middle of sentences. It would be great to have a more "cursive" text display where the sentences flow naturally. For example it currently typically detects text like this:

today
we're going
to bla bla

• optionally (not critical) offering a butt to create a new word which hasn't been properly detected (right now I just add undetected words to the closest word, but then the word boundaries are lost).
• NB: in the Paperedits section, it says "you don't have any transcriptions" instead of "you don't have any paperedits".

In the long run I would say that such a tool which keeps audio and text in sync, with text search capabilities, offers great potential and it could be integrated in various systems such as CMSes. Maybe be this could be released as a plugin? Keep up the good work! 👍

Hi, I attended the NY Video meetup where I learned about your app. Kudos to your app. I also blogged about it and that meetup. Anyway, I tested your platform but I could not get my uploaded audio to work, i.e., transcribe. Btw, your Contact nav doesn't seem to be working.

Dennis

Users have the ability to edit specific words in the transcription text via edit mode. However, upon clicking on a word to edit, the video autoplays by default. This means that users can never pause the video to edit text and it continually plays in the background.

It seems that in edit mode, autoplay of video when single clicking text should be disabled or users should be able to control if the video is played or not when editing.

1. Open autoEdit 2 app
2. Add file to have access to transcript
3. Open transcript show page
4. Enable edit mode
5. Click on word in transcript
6. Observe video autoplays
7. Note video cannot be paused while editing words