Giter VIP home page Giter VIP logo

marcossueiro / nypr-archives-ingest-scripts Goto Github PK

View Code? Open in Web Editor NEW
2.0 2.0 2.0 10.3 MB

A series of xslt stylesheets that transform xml output from exiftool or BWF metaedit, merge data from several sources, and generate xml and html files for ingest into NYPR Archives' various systems

XSLT 36.22% CSS 0.32% HTML 63.46%
digasystem pbcore david bwfmetaedit archives audio wnyc nypr-archives importing-instantiations stylesheets

nypr-archives-ingest-scripts's Introduction

nypr-archives-ingest-scripts

Stylesheets for importing instantiations into the NYPR Archives

2021-01-25 Commit:

Simplified masterRouter, added more flexibility, added unstructured exiftool error Fixed instantiationID2Exif bugs - orders instantiation IDs, better filename generation (better segment suffix handling), better date handling Better parseDAVIDTitle error handling, new parseInstantiationID template better selectionOrExcerpt Bug fixes

Issues: Error log counts off Simplify masterRouter further

Oct 2020 update:

  1. "SeriesQC" takes a series name and performs quality control on cavafy entries
  2. LoC entries accept new 'https' entries
  3. New/improved connections among exiftool, cavafy, BWF MetaEdit and DAVID schemas
  4. Better handling of segment files within an asset
  5. Better CV control of physical items via cavafyFormats.xml
  6. Bug fixes

nypr-archives-ingest-scripts's People

Contributors

marcossueiro avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar

Forkers

nypublicradio

nypr-archives-ingest-scripts's Issues

Better generation field

From template "determineGeneration" in parseDAVIDTitle.xsl :

        <!-- Determine generation based on text flags such as WEB EDIT, etc 
        or from the instantiation suffix (e.g. 3b means part 2) -->
        <!-- We conisder the following Generations with two parts A:B
            A. An original master (Master:) or Derivative (Copy:)
            B. Complete (preservation or access, respectively) 
               or partial (segment)
        This means there are four possible generations:
           Master: preservation
           Copy: access
           Master: segment
           Copy: segment]
        -->

rules?

I'm wonder if there is an easy way to build in “rules.” That’s to say, a reference at the top of each column to state which values in each are a passing (vs. failing) value. Like: Abstract/1, assetDate/1+, etc. Right now genre might be genre/1 but I know that you are considering changing that to genre/1+

Choose outputs

Include a local file to choose outputs for FADGI, DAVID, etc.

Implement CodingHistory protocol for EQ, etc

See https://xsltfiddle.liberty-development.net/jxNakAJ/57

From Statement of Work document:

Data embedded in the CodingHistory field must follow EBU Technical Recommendation R98-1999 (https://tech.ebu.ch/docs/r/r098.pdf).

Additionally, NYPR further specifically requires that the free-text (T=) parameter of the first line follows the following format:

T= [Technical information] ; [Engineer’s comments]

i.e., the technical info and engineer’s comments will be separated by a space, a semicolon, and a space.

The Technical information needed depends on the carrier format:

Analog discs

Example for a microgroove RIAA disc with fused grooves:

T=[St] 0.7 ET [RF] 50 Hz [TO] 500 Hz [RO] 13.7 dB [Sp] 33.3 rpm [Ca] 0dBFS=+18dBu ; Some fused grooves

Include the following information in this order:

Stylus size and shape, preceded by [St] and a space, expressed in thousands of an inch, followed by shape indicator.  

E. g.

[St] 2.0 ET for 2 mil elliptical truncated stylus

[St] 3.5 FCR for 3.5 mil full conical radius stylus

Compensation EQ low frequency rolloff or “rumble filter”, preceded by [RF] and a space, expressed as its corner frequency. Enter 0 if no LF rolloff or rumble filter are applied (e.g. 2-point compensation EQ). 

E.g.

[RF] 50 Hz for a 50 Hz low-frequency rolloff

Compensation EQ turnover, preceded by [TO] and a space, expressed as its corner frequency. 

E.g.

[TO] 500 Hz for NAB EQ turnover

Compensation EQ high frequency rolloff, preceded by [RO] and a space, expressed as attenuation in dB at 10000 Hz. 

E.g.

[RO] 12 dB for AES EQ rolloff (-12 dB at 10kHz)

Analog tapes

Example for a standard, 15 ips NAB tape with no noise reduction and no issues:

T=[EQ] NAB [NR] None [Sp] 15 ips [Ca] 0dBFS=+18dBu

Compensation EQ, preceded by [EQ] and a space, expressed as its common name. 

E.g.

[EQ] NAB

Noise reduction, preceded by [NR] and a space, expressed as its common name. Enter ‘None’ if no noise reduction is applied. 

E.g.

[NR] Dolby SR

Digital tapes and discs

Preemphasis, preceded by [PE] and a space, expressed as its common name (EIAJ). Enter ‘None’ if no emphasis is applied. 

E.g.

[PE] EIAJ

All formats

Playback speed, preceded by [Sp] and a space, expressed in the standard US format such as 15 ips or 33.3 rpm 

Calibration, analog sources, preceded by [Ca] and a space, expressed as dBFS=dBu to indicate the calibration of the converter in its analog-to-digital interface mode. 

E.g.

[Ca] 0dBFS=+18dBu

Calibration,  digital sources, preceded by [Ca] and a space, expressed as Slave to indicate the calibration of the converter in its digital-to-digital interface mode. 

E.g.

[Ca] Slave

Widespread transfer issues, after a space, a semicolon and a space, named following a consistent controlled vocabulary such as the AV Artifact Atlas (https://www.avartifactatlas.com/tags.html#audio). 

Work at the asset level

This will greatly reduce processing time, but it is tricky. Try building up the log and then using "for-each-group"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.