Giter VIP home page Giter VIP logo

peaks.js's Introduction

Build Status npm

Peaks.js

A client-side JavaScript component to display and interact with audio waveforms in the browser

Peaks.js uses the HTML canvas element to display the waveform at different zoom levels, and has configuration options to allow you to customize the waveform views. Peaks.js allows users to interact with the waveform views, including zooming and scrolling, and creating point or segment markers that denote content to be clipped or for reference, e.g., distinguishing music from speech or identifying different music tracks.

Features

  • Zoomable and scrollable waveform view
  • Fixed width waveform view
  • Mouse, touch, scroll wheel, and keyboard interaction
  • Client-side waveform computation, using the Web Audio API, for convenience
  • Server-side waveform computation, for efficiency
  • Mono, stereo, or multi-channel waveform views
  • Create point or segment marker annotations
  • Customizable waveform views

You can read more about the project and see a demo here.

Contents

Installing Peaks.js

You can start using Peaks.js by either including the UMD bundle in a <script> tag in your web page, or by installing it using npm or yarn and including it in your module bundle with Webpack, Rollup, Parcel, etc.

Add a script tag

To add the Peaks.js UMD bundle to your web page, add a <script> tag:

<script src="https://unpkg.com/peaks.js/dist/peaks.js"></script>

The UMD bundle is available at unpkg and cdnjs.

Install with npm

We recommend that you use an ES module bundler.

Run the following commands to include Peaks.js in your module bundle:

npm install --save peaks.js
npm install --save konva
npm install --save waveform-data

Note that Peaks.js uses Konva and waveform-data as peer dependencies, so you must also install those modules.

Add Peaks.js to your web page

To include Peaks.js in your web page, you need to add container <div> elements that Peaks.js will use to render the waveform views, and a media element for your audio or video content. Here is an example HTML fragment:

<div id="zoomview-container"></div>
<div id="overview-container"></div>
<audio id="audio">
  <source src="sample.mp3" type="audio/mpeg">
  <source src="sample.ogg" type='audio/ogg codecs="vorbis"'>
</audio>

The container divs should be left empty, as shown above, as their content will be replaced by the waveform view canvas elements. They should also be styled to have the desired width and height:

#zoomview-container, #overview-container {
  width: 1000px;
  height: 100px;
}

Initialize Peaks.js

The next step is to initialize a Peaks instance with Peaks.init() and your configuration options.

The following examples show how to initialize Peaks.js with a minimal configuration. Refer to the Configuration section in the API documentation for details of all the available options.

Using a script tag

<script src="https://unpkg.com/peaks.js/dist/peaks.js"></script>
<script>
(function(Peaks) {
  const options = {
    zoomview: {
      container: document.getElementById('zoomview-container')
    },
    overview: {
      container: document.getElementById('overview-container')
    },
    mediaElement: document.getElementById('audio'),
    webAudio: {
      audioContext: new AudioContext()
    }
  };

  Peaks.init(options, function(err, peaks) {
    if (err) {
      console.error(`Failed to initialize Peaks instance: ${err.message}`);
      return;
    }

    // Do something when the waveform is displayed and ready
  });
})(peaks);
</script>

Using an ES2015 module import

import Peaks from 'peaks.js';

const options = {
  zoomview: {
    container: document.getElementById('zoomview-container')
  },
  overview: {
    container: document.getElementById('overview-container')
  },
  mediaElement: document.getElementById('audio'),
  webAudio: {
    audioContext: new AudioContext()
  }
};

Peaks.init(options, function(err, peaks) {
  if (err) {
    console.error('Failed to initialize Peaks instance: ' + err.message);
    return;
  }

  // Do something when the waveform is displayed and ready
});

Generating waveform data

Peaks.js creates its audio waveform visualization by processing the audio to produce waveform data. There are two ways that you can do this:

  • Pre-compute the waveform data from the audio, using audiowaveform, and provide the data to Peaks.js from your web server
  • Compute the waveform data in the browser using the Web Audio API

Using the Web Audio API can work well for short audio files, but involves downloading the entire audio file to the browser and is CPU intensive. Pre-computing the waveform data is preferable for longer audio files, because it saves your users' bandwidth and allows the waveform to be rendered faster.

Pre-computed waveform data

Peaks.js uses waveform data files produced by audiowaveform. These can be generated in either binary (.dat) or JSON format. Binary format is preferred because of the smaller file size (including after gzip or Brotli compression).

You should also use the -b 8 option when generating waveform data files, as Peaks.js does not currently support 16-bit waveform data files, and also to minimise file size.

To generate a binary waveform data file:

audiowaveform -i sample.mp3 -o sample.dat -b 8

To generate a JSON format waveform data file:

audiowaveform -i sample.mp3 -o sample.json -b 8

Refer to the audiowaveform documentation for full details of the available command line options, or use the manual page:

man audiowaveform

Once you have created a waveform data file, you can use this from Peaks.js by passing a dataUri option to Peaks.init():

import Peaks from 'peaks.js';

const options = {
  zoomview: {
    container: document.getElementById('zoomview-container')
  },
  overview: {
    container: document.getElementById('overview-container')
  },
  mediaElement: document.getElementById('audio'),
  dataUri: {
    arraybuffer: 'sample.dat' // or json: 'sample.json'
  }
};

Peaks.init(options, function(err, peaks) {
  // Do something when the waveform is displayed and ready, or handle errors
});

Web Audio based waveform data

Peaks.js can use the Web Audio API to generate waveforms, which means you do not have to pre-compute a waveform data file beforehand.

To use Web Audio, omit the dataUri option and instead pass a webAudio object that contains an AudioContext instance. Your browser must support the Web Audio API.

import Peaks from 'peaks.js';

const audioContext = new AudioContext();

const options = {
  zoomview: {
    container: document.getElementById('zoomview-container')
  },
  overview: {
    container: document.getElementById('overview-container')
  },
  mediaElement: document.getElementById('audio'),
  webAudio: {
    audioContext: audioContext,
    scale: 128,
    multiChannel: false
  }
};

Peaks.init(options, function(err, peaks) {
  // Do something when the waveform is displayed and ready, or handle errors
});

Alternatively, if you have an AudioBuffer containing decoded audio samples, e.g., from AudioContext.decodeAudioData then an AudioContext is not needed:

import Peaks from 'peaks.js';

const audioContext = new AudioContext();

// arrayBuffer contains the encoded audio (e.g., MP3 format)
audioContext.decodeAudioData(arrayBuffer)
  .then(function(audioBuffer) {
    const options = {
      zoomview: {
        container: document.getElementById('zoomview-container')
      },
      overview: {
        container: document.getElementById('overview-container')
      },
      mediaElement: document.getElementById('audio'),
      webAudio: {
        audioBuffer: audioBuffer
      }
    };

    Peaks.init(options, function(err, peaks) {
      // Do something when the waveform is displayed and ready, or handle errors
    });
  });

Next steps

We recommend that you take a look at the demos, which show how to use the various options and APIs that Peaks.js provides.

Refer to the API documentation for details of all the available configuration options and the API methods and events, and to Customizing Peaks.js for more information on advanced customization options.

If you're having difficulty, please refer to the Frequently Asked Questions page or raise an issue.

Demos

The demo folder contains some working examples of Peaks.js in use. To view these, enter the following commands:

git clone [email protected]:bbc/peaks.js.git
cd peaks.js
npm install
npm start

and then open your browser at http://localhost:8080.

There are also some example projects that show how to use Peaks.js with popular JavaScript frameworks:

API

Refer to the API documentation for details of all the available configuration options and the API methods and events, and to Customizing Peaks.js for more information on advanced customization options.

If you're updating your code to use a new release of Peaks.js, please refer to the Version Migration Guide for details of any breaking API changes.

Building Peaks.js

This section describes how to build Peaks.js locally, if you want to modify the code or contribute changes.

The first step is to clone the git repository and install the dependencies:

git clone [email protected]:bbc/peaks.js.git
cd peaks.js
npm install

Next, use the following command to build the library:

npm run build

This will produce various builds of Peaks.js:

  • dist/peaks.js and dist/peaks.min.js are stand-alone UMD modules that contain all dependencies. Note that custom markers do not work with these builds. See the FAQ for more details

  • dist/peaks.esm.js is an ES module bundle that does not include the Konva.js and waveform-data.js dependencies

  • dist/peaks.ext.js and dist/peaks.ext.min.js are UMD builds that do not include the Konva.js and waveform-data.js dependencies

The build also creates an associated source map for each JavaScript file.

Testing

Tests run in Karma using Mocha, Chai, and Sinon.

  • npm test should work for simple one time testing.
  • npm test -- --glob %pattern% to run selected tests only
  • npm run test-watch if you are developing and want to repeatedly run tests in a browser on your machine.
  • npm run test-watch -- --glob %pattern% is also available

Contributing

If you'd like to contribute to Peaks.js, please take a look at our contributor guidelines.

License

See COPYING.

This project includes sample audio from the BBC radio programme Desert Island Discs, used under the terms of the Creative Commons 3.0 Unported License.

Credits

This software was written by:

Thank you to all our contributors.

Copyright

Copyright 2024 British Broadcasting Corporation

peaks.js's People

Contributors

chrisn avatar thom4parisot avatar jdelstrother avatar is343 avatar ziggythehamster avatar chainlink avatar dependabot[bot] avatar johvet avatar rowild avatar dodds-cc avatar tscz avatar evanlouie avatar liscare avatar ffxsam avatar un-chein-andalou avatar jodonnell avatar bitwit avatar zachsa avatar spidy88 avatar markjongkind avatar gmarinov avatar thinkpunk avatar cky917 avatar ahombo avatar aularon avatar yosiat avatar tdurand avatar karoid avatar samstarling avatar rramphal avatar

Stargazers

Brian Muigai avatar Anael Orlinski avatar Code-wy avatar Marshall Jones avatar  avatar blurk avatar Gao Lei avatar Sage Baggott avatar Gumplefik avatar BA7LYA avatar  avatar  avatar Muhammad Mahmoud avatar David O'Trakoun avatar Yoones Khoshghadam avatar Piotr Kowalski avatar Grace Meredith avatar  avatar Akshay Malviya avatar Lothar avatar Kai Schwaiger avatar Hanlin avatar Tom Jensen avatar Slade avatar Jiล™รญ Krblich avatar Filipe Herculano avatar zkkzk avatar  avatar Jason S avatar RiverRay avatar fifteen42 avatar Ahmet Emin รœnal avatar OnlineTech avatar Abhishek Katiyar avatar Marc Camuzat avatar Echo Hui avatar Liu Yue avatar Kevin Kirsten Lucas avatar  avatar Andreas Roussos avatar ๅฎถ่พ‰ avatar Sadegh Karimi avatar Hudson Newey avatar Stefan Zweifel avatar Maythee Anegboonlap avatar Christian Schmidt avatar Sebastian Schlatow avatar Philip Nuzhnyi avatar Marek Sล‚owiล„ski avatar Ulises Rosas avatar Tim Lewis avatar Lark avatar Bochao avatar Fayez Nazzal avatar Francesco Castrovilli avatar nasso avatar Jean Leao avatar Hendrik Brombeer avatar NIXIDE avatar Brandon Carlisle avatar Feng avatar JK avatar  avatar shenyu avatar mcpanl avatar Indigo avatar Luke Askew avatar  avatar  avatar Yash Chudasama avatar Whoami avatar ๆ— ้‡ๅŠ›ๅนฟๅœบ avatar Robin avatar Cristรณferson Bueno avatar Marina Goritskaia avatar  avatar Alan Gunning avatar Ben avatar Jonny Furness avatar Hamad Binqali avatar Atef Ben Ali avatar Phรกt avatar Ryan Martin avatar oasis avatar Krzysztof Witczak avatar Caio Pereira avatar sun avatar  avatar jessie J taylor avatar Florian Huber avatar Leo avatar Dariusz Kobuszewski avatar Cheah Shao Qi avatar Jan avatar  avatar  avatar Daniel Turuศ™ avatar Rรฉmi Pace avatar Muhammad Ubaid Raza avatar Icebob avatar

Watchers

Sean O'Halpin avatar Yves Raimond avatar  avatar  avatar  avatar ND avatar davidwei_001 avatar Henry Cooke avatar Peter Schmidler avatar Cristiano Contin avatar send2vinnie avatar srobinson avatar Leigh Hunt avatar Christian Hochfilzer avatar william avatar Hans Wobbe avatar Gary hilgemann avatar Anthony Onumonu avatar ๆƒ…ๅฐ avatar Goutham Gandhi Nadendla avatar  avatar timelyportfolio avatar James Cloos avatar  avatar Tom Schulze avatar Michael Demarais avatar peiman F avatar Andrew Yang avatar Michael Anthony avatar Darkphร s Dรฉ Lรดne avatar  avatar ็”ฐๆ™บๆ–‡ avatar  avatar Marco Medrano avatar justiceli avatar  avatar What's in a name? avatar Qi avatar odayibasi avatar  avatar Lei He avatar CROTEL avatar Noel Koutlis avatar Q avatar Ben Clark avatar Marty Hauff avatar Kwangju Kim avatar  avatar  avatar Chrissy Pocock-Nugent avatar Vis avatar Chad Harrington avatar eriser avatar  avatar  avatar sandianyu avatar Matthew Wilde avatar mason avatar ywentao avatar ๅฐ้ป‘ avatar Misa Ogura avatar  avatar Marc Gรณrriz Blanch avatar EMCV avatar al-sabr avatar  avatar Muhammaad Awais avatar likewater avatar  avatar  avatar  avatar  avatar Brian Muigai avatar

peaks.js's Issues

Improve `addSegment` signature

Keep BC using argument.length, deprecate it and remove it in 0.2.0.

// old signature
peaks.segments.addSegment(peaksstartTime, endTime, segmentEditable, color, labelText);

// new signature
peaks.segments.addSegment({
  startTime: Number,
  endTime: Number,
  editable: Boolean,  // optional
  color: String, // optional
  label: String // optional
});
  • write tests
  • update signature
  • Update README.md

simple js?

Is there a reason why simple peaks.js/peaks.min.js is not compiled anymore and you have to use RequireJs and Bower, or make it yourself?

I noticed that there is: http://wzrd.in/standalone/peaks.js but it does not seem to work.

I am still stuck in 0.0.X version and wish to upgrade but I find it difficult figuring where to start. Would be great if I can get some tips how to do so.

Thank you :)

Visualizing all audio and zooming on the same canvas

Is there a way to show only one waveform and zoom on it instead of always having two ? I have some space constraints and it would be awesome to use only one waveform (one canvas) to zoom in/out and show the entire file.

All the examples always shows two, so i donโ€™t know if it is possible.

If it is not possible right now, would it be an welcomed addition ? :-)

Revamping audio/UI related events

tl;dr

The UI should drive events only, subscribed and handled by views.

Interface to Internals Events

  • waveform_seek
  • waveform_zoom_level_changed
  • waveform_size_changed (originally window_resized)

Events should contain:

  • time
  • currentFrame
  • origin

Internals to Interface Events

  • player_time_update
  • player_play
  • player_pause
  • zoomview_resized
  • player_ready
  • waveform_ready
  • peaks_ready (will be a separate issue later on)

Goals

  • UI will collect interactions and spread them across the views
  • avoiding side-effects caused by cascaded events
  • being able to drive the UI through events only
  • driving the interface just by acting on the media element

MP3 causing offset, OGG works fine

Based on issue #21
Hi @oncletom, issue #22 and my one was unrelated but it was an issue as well to me so thank you very much for fixing it. It is a lot better than my workaround solution. The issue I had in #21 remains but I have found out who is the culprit of this is. I basically do 2 files (mp3 and ogg) so it would support more browsers. I got mp3 as priority and ogg as a alternative. The problem is that if you use MP3s it causes an offset and if you use OGGs it is spot on. It seems MP3s fail to load in time or something. I didn't test it on WAVs because I would not use them anyway (files are too big). OGGs, however, do not work on IE and Safari. I can switch priorities aroud of course but IE and Safari users will have an offset.

Any solution to make it work well with MP3s too?

To test simply try any file (with clear beats if possible) with OGG and seek and then try the same with MP3. MP3s seem to be slow in loading though animation starts earlier.

Time calculated is twice the waveform length

Hello sirs, I was hesitant posting this issue as I can't determine whether this issue is related to mp3 encoding or something in the calculation of the waveform time length. But if you have a look at this member's upload: http://www.internetdj.com/song/109702 you'll see the waveform time is about 12 minutes, when in fact the song is actually about 6 minutes. You can seek successfully to about the 6 minute mark, but if you click anywhere beyond that, the song stops and you have to reload the page to listen again. It's only this particular member's tracks so I suspect it's something up with the generated waveform data. Any tips you could provide would be appreciated.

Segment API is not working

console.log(peaks.segments.getSegments());
returns undefined. (TypeError: api.waveform.segments is undefined)

Same is with: peaks.segments.addSegment(startTime, endTime, editable, labelText)

Version 0.0.6 bugs

I upgraded to the latest version and I encountered some issues with it. I'll try to explain as best as I can here.

  1. I built a fresh peaks.min.js and overwritten my current one and I got "options not defined" error. Only overview appeared but you couln't seek and it simply only played from start to finish. I checked demo_page.html again and noticed quite a few differences so I tried to fix it. I made the same way as in examples but still got "options not defined" error. I didn't try 0.0.5 version but when I used 0.0.4 I had all my code around document.ready function and it worked fine. Now it doesn't work when I have this function but works when I remove it. However, zoomview when I click I get: ReferenceError: frameIndex is not defined and it breaks the waveform and the animation breaks. It works fine until I click on zoomview, if I stick on overview, I have no issues. So what has changed or is it a new bug?

demo_page.html gets "options not defined" error too. So I cannot understand what is wrong if the demo page is not working.

  1. This is actually was since 0.0.1. I cannot download and run the examples immeadiatly, the pathing in demo_page.html and also in demo_page_dev.html. It might be just me but I don't understand why. For example: <script src="/build/js/peaks.min.js"></script> has / at the beginning and it cannot find peaks.min.js it should be <script src="build/js/peaks.min.js"></script> and there are about 5 more paths wrong and everytime I want to test a new version I need to keep changing. It might be because I simply run the file on the server and not make one using "grunt server-demo".

This is the new demo page after fixing paths: http://tinyurl.com/nw5uwqo

and this is my working sample with an exception of broken zoomview: http://tinyurl.com/qhegyp9

Can you have a look and suggest if I have anything wrong. I think is broken since the example is not working. :(

Jumpy playhead

#99 worked from CDN and surprisingly I managed to upgrade from 0.0.X to 0.4.3 within 30min. I have a tiny annoying issue though.

Unlike in 0.0.X version, playhead in the latest version is feels really jumpy. Did I forget something imortant? In the old version it is really smooth, I don't seem to find anything in my code logically that could cause this.

This is the video that shows it: https://dl.dropboxusercontent.com/u/680402/peaks.mp4

This is what my code basically looks like:
(function(Peaks){
var options = {
container: document.getElementById('waveform-visualiser-container'),
mediaElement: document.getElementById('audio'),
dataUri: {
arraybuffer: datFile
},
keyboard: false,
height: uriParam("height"),
zoomWaveformColor: zoomWaveformColor,
overviewWaveformColor: overviewWaveformColor,
playheadColor: 'rgba(255, 250, 255, 1)'
};
Peaks.init(options);
})(peaks.js);

I am using peaks.js file which I stored on my own server.

CLI builder

So as we could just do the following:

npm install

npm install peaks.js
./node_module/.bin/peaks.js build

bower

bower install peaks.js
cd bower_components/peaks.js && npm install && npm run build

git / zip install

git clone git://github.com/bbcrd/peaks.js.git
cd peaks.js && npm install && npm run build

Add Point Markers Individually

We'd like to be able to set point markers per point (rather than globally for the whole waveform)

Currently Implementing This feature

Error: Unable to determine a compatible dataUri format for this browser.

Im using the standalone peaks.js file doing:

var Peaks = (peaks.js)
var p = Peaks.init({
        container: document.querySelector('#peaks-container'),
        mediaElement: document.querySelector('.peaks')
      });

On:

 <div class="player-container">
    <div id="peaks-container"></div>
      <audio class="peaks" controls>
        <source src="./test.mp3" type="audio/mpeg" />
      </audio>
  </div>

But get:

Uncaught Error: Unable to determine a compatible dataUri format for this browser. on line: 18405 of peaks.js

it throws on if (!uri) there

I understood that dataUri was optional?

I'm just trying to create a waveform in the browser..

Markers API

tl;dr

This API replaces the Segments and Points API.

Why?

Segments and points contains basically the same data. They differ only in their rendering aspect.
Hence a very similar codebase.

Also, we want to deal with multiple/optional layers of data, to toggle them.

Goals

  • light object structure
  • data to rendering approach
  • a marker = timing informations + free structure (meta)data + rendering function
  • view system to narrow the selection per waveform view

Example

var p = Peaks.init({
  // ...
});

// add a marker
p.markers.add({
  timestamp: 30.5,
  duration: 20,
  data: { name: 'John Reith', url: 'http://dbpedia.org/page/John_Reith,_1st_Baron_Reith', labels: ['speaker'] }
});

// add another one which is non-editable, with a custom shape
p.markers.add({
  timestamp: 30,
  editable: false,
  data: { comment: '...', pid: 'b01mxx3h' }
  shape: (segment, view) => {
    const shape = new Konva.Shape({ ... });
    shape.sceneFunc(...);

   if (view.name === 'overview') {
     return false;
   }

    return shape;
  },
});

// add views ...
p.markers.views.speakers = (marker) => marker.data.labels.indexOf('speaker') !== -1);
p.markers.views.comments = (marker) => 'comment' in marker.data;

// fetched by view
p.markers.views.speakers.forEach(marker => { ... });

// all of them
p.markers.forEach(marker => { ... });

// change some data in markers and force update their rendering
p.on('marker.click', (event, view, markers) => {
  markers
    .filter(view.objectInView)
    .forEach(marker => { ... });

  markers.render();
});

Work

  • Definition of the features
  • Definition of the API

closes #64

Thoughts on zoom strategy - continuous zoom and long audio files

Having played around with peaks.js zooming for a couple of days, using data generated from long audio files (2 - 3 hours), I've made the following observations (ignoring the bug reported in #90).

  • Peaks.js works well for stepping between zoom levels
  • Resampling data when a new zoom level is selected is quite slow
  • Drawing waveforms at various zoom levels is very fast
  • The method for animating between zoom levels could be improved I think - currently there's a lot of resampling going on (slow), and (ignoring the bug) a lot of redrawing (fast).

I think it would be nice to support continuous zooming - rather than stepping up and down between pre-defined zoom levels, being able to zoom to any level between the minimum and maximum without a significant performance hit. For example, moving a zoom slider, or using the trackpad to zoom in or out. The existing behaviour would still be supported - jumping up or down to the next zoom level would see peaks animating the current zoom from one level to the other, just as if the user had altered the position of a slider from one level to the other.

Some thoughts on how this could be achieved (just a starting point...):

  • Pre-generate resampled data at fixed zoom levels (for example, [512, 1024, 2048, 4096]). Cache these resampled data sets so they don't need to be regenerated at a later point. Resampling is expensive, and doing it on-demand causes noticeable rendering delays.
  • When rendering the waveform at certain zoom level, peaks would retrieve the nearest (rounded down) data set from the cache. So in the example above, if the zoom level request by the user was 768, peaks would use the cached data for the 512 zoom level. If the zoom level was 1600, peaks would use the cached data for the 1024 zoom level.
  • Peaks would then render the waveform at the zoom level taken from the cache, but would render a waveform that is wider than the current zoom view (current zoom view width * (actualZoomLevel / cacheZoomLevel)). The waveform layer would then be scaled down (KineticJS pseudo code - setScaleX( cacheZoomLevel / actualZoomLevel) ) to fit the zoom view window. In the example above, assuming a window of 1000px wide and a requested zoom level of 768..
    • Peaks would use the pre-generated resampled data for the 512 zoom level
    • It would use this to render a waveform that is (1000 X (768/512)) = 1500px wide
    • The layer would then be scaled in the X axis to (512/768) = 0.66667, bring it down in size to 1000px to neatly fit the window.
  • It's a bit like Google Earth - there's a continuous zoom, but at fixed points in the zoom the images are swapped out for the next level of detail.
  • This approach would allow data to be pregenerated at fixed zoom points, but would allow the display of any zoom level between the minimum and maximum without regenerating any data. Drawing is cheap, calculation is expensive here.

Concerns / assumptions / things to check:

  • Memory usage if caching several sets of large data
  • Visual differences when drawing at 100% scale vs stretched / compress scale. How noticeable would it be when switching from one set of data for the next.

Just a starting point for discussion.

Zoom causes a browser freeze if done after waveformOverviewReady

Tested browsers:

  • Safari 7.1.3 (OS X Mavericks)
  • Safari 7.1.4 (OS X Mavericks)
  • Firefox 38 (OS X Mavericks)
  • Opera Developer 30.0.1820.0 (OS X Mavericks)
  • IE 11 (Windows 8.1)
  • IE 11 (Windows 10)

I haven't tried it with Project Spartan, but I have no reason to believe that it will work there.

If I have the following init code (where peaks is window.peaks because Browserify doesn't work as expected with ES6 modules and I don't want to figure out why):

    $mediaElement = $("audio")
    p = peaks.init
      container: $(".waveform-container").get(0)
      height: 200
      logger: console.error.bind(console)
      mediaElement: $mediaElement.get(0)
      segments: []
      zoomLevels: [256, 512, 2048, 8192]

      dataUri:
        arraybuffer: "https://SOME_BUCKET.s3.amazonaws.com/SOME_FILE.dat"

    p.zoom.setZoom(2)

Then zoom works as expected: I am now zoomed at the 2048 zoom level.

If I zoom after waveformOverviewReady fires, the browser locks up. If I don't zoom like I'm doing here, it still locks up. The demo that is currently in master under npm run start freezes as well. I was able to get a LITTLE bit of profiling information from IE (and only IE) and it seems like it's freezing in Kinetic.Animation._animate. All I see is a bunch of garbage collection for tens of seconds (and then I stopped the profiler). Maybe it's stuck in an infinite loop?

I think this is also why the build is failing to run properly under CI - perhaps it times out because the browser freezes.

What should I try to diagnose this? Is there something I can configure or comment out to make it zoom without animating?

Segment not re-drawing after marker reaches end of container

Hi,

On Mac, Chrome 37.0.2062.94. I've built your latest version from github on my local machine, and I noticed after starting the stream, the segments view doesn't re-draw to reflect the currently playing segment, it remains on the segment that loaded upon page load. This is different than the experience on this sample page:
http://waveform.prototyping.bbc.co.uk/

Is this a feature difference between these two versions?

Thanks for an awesome package and great instructions.

Michael

Gaps between segments

I tried some examples to fix the gaps between segments which looks like this:

[{"startTime":55.77,"endTime":58.1,"editable":false,"color":"rgba(215, 40, 40, 0.9)","labelText":"segment26"},
{"startTime":58.1,"endTime":60.42,"editable":false,"color":"rgba(215, 40, 40, 0.9)","labelText":"segment27"},
{"startTime":60.42,"endTime":62.74,"editable":false,"color":"rgba(215, 40, 40, 0.9)","labelText":"segment28"},
{"startTime":62.74,"endTime":65.06,"editable":false,"color":"rgba(215, 40, 40, 0.9)","labelText":"segment29"}]

So the first endTime is the same as second startTime here, background color is default to green, segments are red, but on the visualization on Peaks, there are still visible small gaps between them, may I ask does it design like this?

Screenshot:
screenshot from 2014-07-15 01 28 07

Rendering is clipped

I've been looking at integrating the waveform / peeks.js code into the "window on the newsroom" - "media tagger" application. I've got the waveform displayed however the waveform is being displayed as clipped and I can't see where I would adjust/normalise the scale.

Is this something that should be done during the waveform data generation or during the rendering via peaks.js.

Some pages showing the output so far (on Reith) :-

http://10.161.159.142:9000/#/dv/edit?videoId=1971555
http://10.161.159.142:9000/#/dv/edit?videoId=1972238
http://10.161.159.142:9000/#/dv/edit?videoId=1971448

Thanks

Mark

Providing a map version of `dataUri`

tl;dr

The dataUri is pretty weak at the moment because of the feature detection.
It also implies URLs to end with .dat or .json.

How it works

Simplified version

We can specify what is the format associated to that URL.

peaks.init({
  dataUri: 'http://server/whatever/you/want',
  defaultDataFormat: 'arraybuffer'
});

Or provide a map of URLs:

peaks.init({
  dataUri: {
    arraybuffer: '/path/to/file/my/binary/file',
    json: '/path/to/file/json'
  }
});

Fallback version

More complex but handles possible custom formats.

They iterate in the declared order and if configure returns a falsy value or throws an Error, it fallbacks to the next available format.

peaks.init({
  dataUri: [
    {
      format: 'arraybuffer',
      uri: '/path/to/file/my/binary/file',
      configure: function (xhr) {
        xhr.responseType = 'arraybuffer';
        return xhr;
      },
    {
      format: 'json',
      uri: '/path/to/file/json',
      configure: function (xhr) {
        try {  xhr.responseType = 'json'; }
        catch (e){
          console.log('xhr.responseType is not supported, the response handler will try a JSON parse before failing.');
        }

        return xhr;
      }
  ]
});

Using provided factories

Same as above but it hides the complexity.

peaks.init({
  dataUri: [
    peaks.data.arraybuffer('/path/to/file/arraybuffer.bin'),
    peaks.data.json('/path/to/file/json')
  ]
});

Works well with a single format. It assumes the defaultDataFormat from the factory.format property.

peaks.init({
  dataUri: peaks.data.arraybuffer('/path/to/file/arraybuffer.bin')
});

Safari issue: Playhead & Waveform creeping backward when you pause playback

When you try out the demo at http://waveform.prototyping.bbc.co.uk on Safari on OSX 10.9 the playhead behaves oddly creeping slowly backward from the time you hit pause, updating both waveform views as it goes.

It looks like there is an event loop in the form:

  1. Player captures a "waveform_seek" event and changes its playback position
  2. This causes a "timeupdate" which is captured in WaveformOverview
  3. WaveformOverview moves the playhead to the new time
  4. This emits a "waveform_seek" event based on playhead position for a slightly earlier time and the loop starts again at 1

Is this expected? It seems to behave properly in Chrome.

Beats don't match anymore if you try to seek

When playing from start to finish without touching (interacting) with waveform, the beats matches the waveform. When you try to seek in the overview it doesn't match anymore (has an offset of 2-3s). Same issue is with the example you have here: http://waveform.prototyping.bbc.co.uk/
This bug kinda defeats the purpose in having a waveform in the first place because it is inaccurate.

"Could not detect a WaveformData adapter from the input"

Hello,

I'm getting the following error in my Javascript console when using a .dat file as the dataUri for Peaks.js:

Could not detect a WaveformData adapter from the input.

I'm generating the file like this:

audiowaveform -i 20141218-180000-greg-james.mp3 -o 20141218-180000-greg-james.dat

Generating a .json file in the same way works fine, although the waveform appears to be constantly clipped (see screenshot below), even though the MP3 is not. Any ideas what I'm doing wrong? Thanks!

screen shot 2014-12-22 at 16 11 16

sticky backward seeking and undraggable segment handles

Hi folks,

I just started playing with peaks and find it's mostly working brilliantly! but a couple of things..

I don't know why but backward seeking is kinda sticky:
When clicking far ahead of the playhead in the overview view the zoom view jumps right to the correct spot -awesome, but when clicking far behind the play head in the overview view the zoom view doesn't move all the way to the correct spot -less-awesome. With a big sized audio file you have to do a lot of clicking on the overview to move the zoom view all the way back so this can be annoying.

also I wonder why dragging segments is disabled in the overview:
in waveform.segments.js ln 68:70

          if (segmentGroup === segmentOverviewGroup) {
            draggable = false;
          }

I guess dragging segments in the overview was disabled for a reason, but still I need to drag segment handles around in the overview so I went ahead and enabled it. Will it cause problems? I haven't seen any issues so far but thought I'd check.

Disturbing Image

I realize this out of the general scope of issues since it isn't technical in nature. Further, I'd like to begin by saying I loved the article at http://www.bbc.co.uk/rd/blog/2013/10/audio-waveforms which led me to this repo and am grateful for your commitment to open source. But as soon as I started skimming through the README doc to see if it fit the needs of my organization, I was taken aback by the image in the sample clip of the young woman. I clicked the image to see if it would take me to the original clip so I could assure myself it wasn't something related to violence against women or porn (my initial gut reaction to the image). When that didn't work I returned to the repo and examined it more closely, belatedly noticing the small identifying "Roxie falling off the table" at the bottom of the sample clip.

I'm sure my reaction (and initial assumption) was not your intention, so I wanted to reach out and let you know because I'm confident I'm not the only person responding in this manner.

Right click context menu

We'd like the ability to right click on points or segments to add or remove them. Any thoughts on how this might be accomplished?

Allow configuration of segment appearance

It would be useful to be able to specify (globally, rather than per-segment) whether segments should be rendered in the zoomview only, the overview only, or in both views.

I will add new attributes to the initialization options, and make changes in waveform.segments.js to support these options.

deamdify is missing in Browserify CDN.

Error: "browserify exited with code 1"

code: 1
stderr: Error: Cannot find module 'deamdify' from '/tmp/peaks.js114915-9818-lnskk1/node_modules/peaks.js'

I think because I moved them from optional to devDependencies recently.

Views API

Peaks should provide an easier way to add, remove and order views.

Using native views with default settings:

peaks.init({
  views: ['zoom', 'overview']
});

Using only one configured native views:

require(['peaks', 'peaks/views/zoom'], function(Peaks, ZoomView){
  var zoom = new ZoomView({
    height: 240,
    backgroundColor: '#fff',
    textColor: 'red'
  });

  var p = Peaks.init({
    views: [zoom]
  });
});

Combining a default native view and your custom metadata view:

require(['peaks', 'my/html/view'], function(Peaks, CustomView){
  var tableView = new CustomView({
    speakersDataUrl: '//localhost:3000/speakers/worldcup/france-vs-switzerland.json'
  });

  var p = Peaks.init({
    views: [tableView, 'overview']
  });
});

View

A view is an asynchronous renderer, performing actions based on peaks timed interactions.

var ZoomView = Object.create(BaseView.prototype, {
  id: 'zoom',
  init: function(config, peaks){
    peaks.on('segment.add', this.onAddSegment.bind(this);

    peaks.emit('load.views.' + id, this);
  },

  onAddSegment: function(segment){

  },

  timeUpdate: function(time){

  },

  render: function(){

  }
});

closes #2
closes #46
closes #59

Disabling the zoom view

Wow, I have to say that is a massive contribution! Thanks so much for the great writeup and project. I'd like to use peak.js in one of my project but don't really have a need for the zoom container. There doesn't seem to be an option to disable it though. I'm willing to get my hands dirty and contribute a pull request. Any suggestions / preference on how you'd like to see that implemented?

Incorrect behaviour when dragging zoom view

Found some incorrect behaviour in the current demo page:

  • Press button to create Segment at current time
  • Press button Zoom out
  • Drag zoom view
  • Segments and time markers drag correctly, but the waveform view in the background does not move. The master waveform is then out of sync with the segments

I will have a look at this problem while making other changes.

Method to destroy peaksInstance

So we are currently using peaks.js in the context of a single page application and have situations where multiple peaksInstances can be created before a page reloads.

What we are seeing is that after loading the second peaks instance, any browser resizing crashes the tab. Note, we can load many peaks instances on top without issue but the browser resizing is what causes the crash.

We suspect that we are creating memory leaks by not properly destroying a peaksInstance before replacing it and there are perhaps lingering callbacks that should be deregistered.

So this is sort of a feature request/request for guidance on how we might go about making sure peaks has completely cleared. There appears to be no peaks.destroy() method.

Much appreciated!

Document an end-to-end example

ref #106

Given we have an audio file, we want to visualise its waveform in a Web browser:

  • audio to data audiowaveform -b
  • browserify toolchain
  • configuring peaks

Error in require.js example Startup command line (kineticjs)

Hi Guys,

I have noticed a mistake in the require.js path (I think) on the main git page under the section "Start using require.js":

At the moment it says:

requirejs.config({
  paths: {
    peaks: 'bower_components/peaks.js/src/main',
    EventEmitter: 'bower_components/eventemitter2/lib/eventemitter2',
    Kinetic: 'bower_components/Kinetic/index',
    'waveform-data': 'bower_components/waveform-data/dist/waveform-data.min'
  }
});

However the correct default path to Kinetic (after building with bower) is actually:

bower_components/kjineticjs/kinetic

...or am I missing something?

Cheers

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.