wgbh-mla / bostonlocaltv Goto Github PK

These records are pretty sparse. Very similar to what comes out of MARS. So if/when a record has video... we'll want to put it front and center.

See staging http://lsboslocal02.wgbh.org/ on development branch.

It's looking like rake jetty:config_solr is working for us, so let's go ahead and get all of the jetty/ dir out of revision, instead of keeping the configurations around, which was causing some issues.

This was done in a sort of wonky way all around, and if we ever re-instante voting on videos, when we should probably fix some of that stuff.

But... that is not in the foreseeable future, so let's just clean some stuff up here.

The source id (from original Filemaker db's) is stored in the Artesia xml at xpath //WGBH_IDENTIFIER/@ITEM_IDENTIFIER. We need to be able to retrieve solr docs based on this.

Not caught by rspec test, so we need one in there that would have caught this.

Occurs after merge of branch fix/rspec-cleanup into master

My first thought would be to:

1. make sure apache server is set up to AllowOverrides for the DocumentRoot of the media directory.
2. make sure mod-rewrite is enabled
3. use a .htaccess file to write some rewrite rules. Check this out for instance http://www.htaccess-guide.com/deny-visitors-by-referrer/. Not sure if that would work, but something like it maybe?

Find out what one of the largest, if not the largest, video that will be on News is, and test streaming that.

Try it locally from a browser using a local HTML file that contains a <video> element.

Are there any particular field that we require other than:

first_name
last_name
email
password

Gemfile is borked in some weird way.

Blacklight gems are pointing to github, which is not good, since everything else is so far behind. We don't want to fast forward too much at once, and introduce a ton of bugs.

Here's what I've tried that does NOT work:

1. Pin to semantic versions specified Gemfile.lock.

gem 'blacklight', '3.2.0'
gem 'blacklight_advanced_search', '1.1.2'
gem 'blacklight_range_limit', '1.1.0'

Results: Trying to start Rails server (in development environment) results in error:

gems/blacklight_advanced_search-1.1.2/lib/blacklight_advanced_search.rb:46:in
`config_defaults': You have a nil object when you didn't expect it! (NoMethodError)
You might have expected an instance of Array.
The error occurred while evaluating nil.find_all

2. Pin to git revision specified in Gemfile.lock.

Results: The server starts, and some pages work (e.g. homepage). But trying to filter by collection results in:

NoMethodError in Catalog#index
Showing /Users/andrew_myers/.rvm/gems/ruby-1.9.3-p327/bundler/gems/blacklight-90be0ebab652/app/views/catalog/_results_pagination.html.erb where line #4 raised:
undefined method `total_pages' for #<struct current_page=1, num_pages=1523, limit_value=10>

And link the people name and subject to a search within the "show" page for an item

1. Grab a big list of media files from mlamedia01.wgbh.org
2. Write a script (bash, ruby, whatever) to concurrently stream videos.
3. Continually increase the number of concurrently streaming videos until it breaks. This will be a useful number in understanding the capabilities of the media server.

We need to figure out where to put the video when we ingest it. Here are a couple of options:

1. Same machine, different directory. Not a good long term solution, but if it's trivial, we could start here.
2. Same machine, streaming http server. This is similar to how splat serves content for openvault. I'm a naive as to how it works. It may have been set up this way b/c splat is shared, in which case, we won't have those requirements for News.
3. hosted CDN. I tend toward this type of solution, since I don't know much about high capacity, high availability, content delivery. I'd just as soon pay the pros to handle it for me. But, there's surely some legal issues here.
4. Remote machine that we manage. If hosted CDN is a non starter for legal reasons, then this would probably be the next thing to consider. But this then puts the onus on us to set it up to be as performant as what we would find in a hosted CDN.

something like:

# given list of partial solr docs
updates = [
  {
    'id' => '10001',
    'video_s' => 'path/to/video',
    'image_s' => 'path/to/image'
  },
  {
    'id' => '1002',
    'video_s' => 'path/to/video',
    'image_s' => 'path/to/image'
  }
]


updates.each do |update_doc|
  doc = Blacklight.solr.find({:q => "id:#{update_doc['id']}"})
  doc.merge! update_doc
  Blacklight.solr.add doc
end

Blacklight.solr.commit

or something like that.

I think we should have some type of introduction on the users/sign_up and users/sign_in pages in the sidebar, also we need to think of what content we want if any on the users/edit page

The ID syntax has change from things like BPL_ASDFASDFAS to 321_BPL_ASDFASDFAS, where the numeric prefix is the can number. K. Colbron had told the WHDH people to put it in there. Not sure why, but assume it's for a reason. Would rather trust that, than assume it's meaningless

After #32

this is two fold, since they are getting merged prior to being inserted into the Solr index.

1. Preserve separation of Contributor and Contributor Role in Solr Index
2. Fix all references to that Solr field so it displays correctly.

unless it servers some purpose I'm not considering.. it's sticks out and is redundant.

I think we would want something like this:

1. Videos and thumbs are named the same as the URL slug for the record. E.g. if the slug for a record is "bob-shaw-reports-in-dorchester-june-3-1989", then the video should be named "bob-shaw-reports-in-dorchester-june-3-1989_video.mp4", and the thumbnail would be "bob-shaw-reports-in-dorchester-june-3-1989_thumb.png". This would make is easy for a human to browse the media server and identify which videos and thumbs belong to which records.
2. We need a way to make sure the slug for the record and the names of the video files and thumbs match up.
3. We need a way to get all the videos up on the media server, and named the correct thing.

We should make a decision on what content we want to have over here

I liked the idea Drew had of making and "advanced" search section that could be hidden when not in use

The bit that says "Explore the Collections -> Vote!", the paragraph below it, and the "Browse Collection" button can all go away

Currently there are 28 failing rspec tests.

Set the keepers to 'pending'. Nuke the rest.

In most cases (i hope) the solr ID is the same as the source id of the originating database (typically a one-off Filemaker db from somewhere).

These IDs are being entered into Artesia, so that when the digital metadata is exported, we'll can look up the solr doc with those IDs.

However, I noticed that some of the IDs in the solr docs are small integers, and they don't look like they will match any original ID from Filemaker dbs.

So... I'd like to make sure we are able to look up all the solr docs we need to.

Currently, an "active" menu item (e.g. if you are on the page to which the menu item links) it is grayed.. which makes it look inactive. Which is kind of silly.

unnecessary labeling--

Now that we have images displayed in search results, let's make it not look like crap, eh?

See screen grab of Sadie's spread sheet here.

Need at least 20 hours of video to test capacity.

When accessing catalog/foo, where "foo" is a non-existent solr id,

in development, it raises an error and gives you a stacktrace

in production, it shows you a page that says you are being redirected, but it does not redirect you. Follow the link, and it takes you to the home page, with a flash msg that says the record could not be found.

Should probably show a more friendly 'record not found' page. And I'm not a fan of the fact that it takes you back to the home page.