persist metadata about hyperamp HOT 6 CLOSED

As we talked about, lets try to go the nosql leveldb way to better understand leveldb and to persist said metadata.

from hyperamp.

Some reading:

• https://www.youtube.com/watch?v=Ssct_O1LET8
• https://github.com/cyberwizardinstitute/workshops/blob/master/leveldb.markdown
• https://github.com/google/leveldb
• http://leveldb.org
• https://github.com/Level/levelup
• https://github.com/substack/leveldb-handbook

from hyperamp.

rethinking how we deal with reading, storing, retrieving metadata a bit.

what do you think of

• scanning the client-specified media library folder in the main process
• streaming those results to something like https://github.com/blahah/yunodb
• sending limited metadata to the client via IPC (after initial scan is complete)
  • send searches to the db, return results to renderer
  • take advantage of paged results for default and filtered lists
  • use a smarter datagrid implementation to deal with paging/large amounts of rows
    • example: http://zippyui.com/react-datagrid/#/examples/basic

that way we limit any possible slowdown on the renderer and offload heavier work to the main process, we start interacting with leveldb a bit more, and we have a persistent document store

from hyperamp.

Sorry I missed your feedback until now. Like the general idea a lot. I don't know anything about yunodb so I'm going to read about it. My thinking was that we would use something like https://github.com/sethvincent/level-model and stream processing to achieve paging and stuff, but I'm open to things that already do this. Part of me things we should really be using SQLite for indexing, but I'm hoping that using levels will eventually come in handy for replicating library info with hyperlog, or something like it (booo conflicts).

I imagine in order to have a robust grid we will also need something like data-grid.

I think the best approach is to just take a stab or two at the problem and see where we get.

from hyperamp.

I'm thinking we should preserve the streaming interfaces (automatic paging) with something like this:

https://github.com/pfrazee/pauls-electron-rpc

Its basically https://github.com/maxogden/websocket-stream + https://github.com/substack/multiplex-rpc but for IPC instead of websockets.

But yes, having a single level/db instance on the main process, and then have the player request streams of data / chunks of arrays of objects to render.

from hyperamp.

What a yack shave.

I just persisted on save using electron-config, despite writing 4 leveldb modules. I say this is closed by
#114 for now.

Open new issues when we start tackling larger music library persistence.

from hyperamp.