Giter VIP home page Giter VIP logo

adobe-connect-to-mkv-to-kaltura's Introduction

Adobe Connect to Kaltura

Abstract

This code generates MKV files out of AC recordings and [optionally] ingests them onto Kaltura.

IMPORTANT NOTE: this process was tested successfully on Ubuntu 16.04 against an on-premise Adobe Connect instance. It should work equally well against Adobe's SaaS and may very well work with other modern Linux distros (though it was never tested with anything but Ubuntu 16.04 and so, its use is recommended). Also, more recent versions of Firefox seem to block the use of Flash entirely. If you plan to use the Flash-based version of Adobe Connect, you should download Firefox 60.0 from here https://ftp.mozilla.org/pub/firefox/releases/60.0/linux-x86_64/en-US/firefox-60.0.tar.bz2 rather than use what is currently available from the 16.04 official repos (71.0+build5-0ubuntu0.16.04.1 at the time of writing this).

General flow

  • Download the ZIP archive and concat the audio FLVs into one MP3 using FFmpeg [this is done in order to determine the session's length so that we know how long to record using FFmpeg's x11grab]
  • Using Selenium and Mozilla's Geckodriver, launch Firefox with Xvfb and navigate to the recording's URL so that it plays using the Adobe SWF
  • Create the needed pulseaudio sinks so that the audio track can be captured [see capture_audio.sh]
  • Use FFmpeg's x11grab option to capture the screen display and audio [via pulse]

Pre-requisites

  • cURL CLI
  • unzip
  • xvfb
  • pulseaudio [provided by the pulaseuadio and pulseaudio-utils packages in Debian/Ubuntu]
  • pidof [provided by the sysvinit-utils package in Debian/Ubuntu and by sysvinit-tools in RHEL/CentOS/FC]
  • Firefox with the Flash plugin loaded [tested with 60.0.1]
  • Ruby [2.3 and 2.5 were tested]
  • Ruby Gems: adobe_connect, selenium-webdriver, open3, kaltura-client, test-unit, shellwords, logger
  • FFmpeg [with x11grab and Pulse audio support, tested with FFmpeg 4.0.2]
  • The Mozilla Geckodriver [tested with v0.20.1]:

Configuration

When using Ubuntu or Debian, you can simply run ac_setup_debian_ubuntu.sh in order to complete the initial setup.

Flash plugin

NOTE: As of 2021, the Flash-based interface for Adobe Connect has been replaced with a new HTML-based interface. If you are planning to use this newer interface, you can safely ignore this section.

In order for this to work, the Flash plugin must be loaded by Firefox. Download libflashplayer.so from Adobe and symlink to /usr/lib/mozilla/plugins/flashplugin-alternative.so

For example, if you've placed the plugin under /usr/lib/adobe-flashplugin/libflashplayer.so, run:

# ln -s /usr/lib/adobe-flashplugin/libflashplayer.so /usr/lib/mozilla/plugins/flashplugin-alternative.so

ENV vars

Set the needed values in ac.rc and make sure it is sourced before running the wrapper as the various scripts rely on the ENV vars it exports. You can just add:

. /path/to/ac.rc

to ~/.bashrc, /etc/profile or place it under /etc/profile.d

In the event you'd like to use ffmpeg and ffprobe binaries from alt locations [i.e: not what's first in PATH], you can export:

FFMPEG_BIN=/path/to/ffmpeg
FFPROBE_BIN=/path/to/ffprobe

IMPORTANT NOTE: these vars need to be set GLOBALLY [for the user you intend to run this with, that is]. Setting them in your interactive shell and running ac_wrapper.sh will not work. Start a new interactive session and make sure they are set BEFORE running the wrapper.

The AC_LOGIN_REQUIRED ENV var

If your AC instance does not require the user to login in order to play the recording, you can set the value of AC_LOGIN_REQUIRED to false to skip that step. The code in ac_new.rb assumes that the user and passwd text field IDs are name and pwd respectively and that the submit button ID is login-button. If that's not the case in your AC I/F, you will need to change the code accordingly.

The code further assumes the login URL is:

@base_url + "/system/login?logintype=oldstyle&next=/admin"

You may need to adjust that as well.

The AC_HTML_VIEW ENV var

In 2020, Adobe Connect switched to a new HTML-based interface that replaces the older Flash-based one. As of 2021, the SaaS hosted version of Adobe Connect no longer includes the Flash-based interface. On-premise installations of Adobe Connect may still include the older interface, however, and this is preferable for recordings made before the switch, as there are some issues with playing older recordings in the newer interface. This ENV var controls whether the script attempts to use the older or newer interface. If set to false (the default), it will attempt to use the older Flash-based interface. If set to true, it will use the newer HTML-based interface. On newer versions of Adobe Connect, including the SaaS hosted version, it must be set to true for recordings to be properly captured.

Generating the recording list to process

generate_recording_list.rb can be used to generate a CSV containing the recordings metadata. It accepts a text file with all the SCO IDs, separated by newlines; i.e one SCO ID per line, makes the needed AC API calls and outputs the data in the following format:

SCO-ID,SCO-FOLDER-NAME,SCO-NAME,PATH-URL

To generate a list of recordings to process, run:

$ ./generate_recording_list.rb </path/to/sco/ids/file> > /path/to/asset/list/csv

Parallel processing

This code is capable of processing multiple recordings concurrently and the only real limitation is HW resources [namely: CPU, RAM].

In order to process several recordings simultaneously, a wrapper around xvfb-run is needed. xvfb-run-safe is expected to be found in a directory included in the PATH ENV var. If need be, you can change ac_wrapper.sh so that it looks for it elsewhere.

The number of concurrent jobs to run is determined in ac_wrapper.sh based on the value of the MAX_CONCUR_PROCS ENV var.

Running

Once ready, run:

$ ./ac_wrapper.sh </path/to/asset/list/csv>

Where </path/to/asset/list/csv> is the path to a CSV file in the format described above.

Output

The resulting MKVs will be placed under $OUTDIR/$RECORDING_ID.full.mkv. Where $RECORDING_ID is the relative path for the given recording; i.e: //sco//url-path - the last field in the input CSV file.

If the KALTURA_.* ENV vars are set, $OUTDIR/$RECORDING_ID.full.mkv will then be uploaded to Kaltura. Full logs are written to /tmp/ac_$RECORDING_ID.log. If there's a problem, start by looking there.

If the KALTURA_METADATA_SYSTEM_NAME and KALTURA_METADATA_XML ENV vars are set, the original creation date of the Adobe Connect recording will be saved as a custom metadata field for the entry. If a metadata profile with system name KALTURA_METADATA_SYSTEM_NAME does not exist, it will be created by the script.

If the SCOID_USER_MAPPING ENV var is set, the generate_recording_list.rb script will use this mapping file to map SCO IDs to new content owners. If the ENV var is not set, the script will default to using the data returned by the Adobe Connect API to set the content owners.

If the SCOID_DURATION_MAPPING ENV var is set, the generate_recording_list.rb script will use this mapping file to extract recording durations where possible. NOTE: to maintain compatibility with information exported from the Adobe Connect system, the durations in the mapping must be provided in minutes.

If the KALTURA_ENABLE_REPLACEMENT ENV var is set, the logic will attempt to replace the content of an existing Kaltura entry with the same reference ID value as the given SCOID, rather than create a new entry. If no entry with a matching reference ID exists, a new one will be created.

Contributing

Where to get help

Get in touch

You can learn more about Kaltura and start a free trial at: http://corp.kaltura.com Contact us via Twitter @Kaltura or email: [email protected]
We'd love to hear from you!

License and Copyright Information

All code in this project is released under the AGPLv3 license unless a different license for a particular library is specified in the applicable library path.

Copyright © Kaltura Inc. All rights reserved. Authors and contributors: See GitHub contributors list.

adobe-connect-to-mkv-to-kaltura's People

Contributors

bodrick avatar hilak avatar jfmontanaro avatar joshuamharder avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

adobe-connect-to-mkv-to-kaltura's Issues

session recording

hi
this scrip can record adobe connect meet?
can recorrded chats, attendees and teacher notebook? this problem that other recording cannot record chats and links in abobe connect meet has been solve?

Helpful program I wrote for this system

Hello!

I am using this Kaltura conversion program to help a company move hundreds of recordings to MP4 and reduce the need for the real-time conversion feature that Adobe Connect provides (which is a massive time waste). The Kaltura conversion program works generally well, but finding SCOs for specific categories can take a very long time. To speed this up, I wrote a program which helps me find piles of SCOs for each category in only a few clicks. I licensed it as AGPLv3, just like this Kaltura converter.

If there's any way to promote this wider among users of this conversion program, that would be great. I just hope it's useful for other people and not just myself.

https://github.com/gjsman/Adobe-Connect-SCO-Viewer

image

Video stops recording at wrong times

I'm having a problem where my videos incorrectly identify when to end. This is on Ubuntu 16.04.
I recorded 2 videos. One ended only 22 minutes into the recording, whereas the other went 18 minutes past the recording ending before suddenly stopping. Curiously, I redid the first video twice afterwards, and all three times it consistently ended at 22 minutes in.

Missing ENV var X_SERVER_DISPLAY_NUM

My log file:


nohup: ignoring input
Loaded suite ./ac_new
Started
2019-09-03 00:49:02 WARN Selenium [DEPRECATION] :firefox_profile capabilitiy is deprecated. Use Selenium::WebDriver::Firefox::Options#profile instead.
2019-09-03 00:49:02 WARN Selenium [DEPRECATION] :firefox_profile capabilitiy is deprecated. Use Selenium::WebDriver::Firefox::Options#profile instead.
2019-09-03 00:49:07 +0000: ERROR ./ac_new.rb(67): Missing ENV var X_SERVER_DISPLAY_NUM:( Make sure you export it.
.
Finished in 5.629896872 seconds.
-------------------------------------------------------------------------------
1 tests, 1 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
-------------------------------------------------------------------------------
0.18 tests/s, 0.18 assertions/s


I installed xvfb-run-safe using code from https://github.com/eddelbuettel/prrd/blob/master/inst/scripts/xvfb-run-safe.
I am on a new Ubuntu 18.04 installation.

Overlapping Audio

Hello,

I set up a server and (besides #15 ), I'm having another problem: I set it to record 4 meetings at once, and the audio for all 4 meetings appeared in all 4 recordings - a combined mess.

I have no idea how to fix this. I have the latest version of pulse audio, on Ubuntu 18.04, if that matters.

Frequent Failures

I am frequently having failures because of this:


E
===============================================================================
Error: test_vconn1(Vconn1):
  NameError: undefined local variable or method `log' for #<Vconn1:0x0000560d807d1980>
  Did you mean?  load
/root/kaltura-adobe-connect/adobe-connect-to-mkv-to-kaltura/ac_new.rb:109:in `test_vconn1'
     106:         return false
     107:       end
     108:     else
  => 109:       log.error('Failed to get duration from audio track or XML metadata :(')
     110:       return false
     111:     end
     112:
===============================================================================
E
===============================================================================

I don't know what else to say. Help.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.