sciencefair-land / sciencefair Goto Github PK

This is the overall ticket for installation issues.

Replace manual paging with infinite scrolling of search results.

Possibly we can just rip apm from electron?

Basically we just want a wrapper around npm, with ~/.sciencefair/packages as the install location, and with a nice interface for finding, installing and managing the packages.

And we'll need to structure the API with extensibility in mind.

We need a simple about pane that shows the version, links to the website, and tells whether there's an update available.

Might be provided automatically by using electron-builder (see #44)

If we browserify the code before running it in electron we get the benefit of being able to greatly reduce the final distribution filesizes, as well as increase performance using things like https://github.com/shama/yo-yoify.

Someone has walked this path successfully with electron apps before: http://maxgfeller.com/blog/2016/08/30/electron-browserify/

In a different format, or eventually to use a converter?

Papers that are undownloaded can't be shown by with lens, so don't show lens icon

We need a place where people can manage settings, including packages (see #19).

@freeman-lab thoughts on what that might look like?

It's confusing to me to have 'collections' be so prominent - it'd be nice if it was a drawer and not open by default

Take a (sub)set of search results and export them in a variety of citation formats, ready to go into a reference manager.

Could also take the references of all the selected papers and export them.

upstream issue max-mapper/sqlite-search#6

Paste (cmd-v on mac) doesn't paste in the search bar or datasources modal

I know from other issues you're generally aware of these, but you asked for more documentation tracking them, so here goes.

Excited to see your progress. Youve got my star and watch..

At some point in the last few days the download speed stopped working.

@maxogden @freeman-lab can you elaborate on what you had in mind? I'd like to get this in place before going too much further - it will get harder to do the more we build without it.

This is an issue with dat-storage, and has been reported: dat-ecosystem-archive/dat-storage#2

When there's a fix there, we'll cut a new release of sciencefair to integrate it.

I'm big on keybindings I guess haha; seemed like unexpected behavior to me.

I should probs update https://github.com/shama/modal-element at some point, but ref code for this can be found here: https://github.com/yoshuawuyts/base-elements/blob/master/modal.js

Italics text in article titles not rendered in search results.

Example: "Homo Naledi" in http://doi.org/10.7554/eLife.10627

https://github.com/electron-userland/electron-builder

This should reduce the complexity of building for distribution, and comes with nice benefits like automatic upgrading.

Hi,
for starters, I would like to say, that I really like the idea behind this app.
I was wondering, however, how do you plan to approach the problem of browsing a journal's archive and downloading papers of interest?

For example, two acknowledged open-acces journals in chemistry, Molecules ans BeilsteinJOC, do not offer a public API (as far as I know). There is a possibility of concluding the URLs and thus making the requests, but it does not seem to be a "clean" approach.
There is another option (in my opinion a better one), where you use PubMed API to do the job. As far as I know, there is no JavaScript wrapper for that, so it has to be written.

Either way, I would like to participate somehow in the project by, for example, writing a JS module for PubMed API (I've already created a PubChem API wrapper). I see myself rather as a hobbyist JS programmer, since I'm still a PhD student (Chemistry), but I'll be glad to help.

Would be nice to have an "Open in Finder" option when reading a paper that just open the folder where the data is stored

Hyperdrive v8 will be much faster and the API is expected to stabilise - it will track node's fs module API from now on.

https://github.com/mafintosh/hyperdrive/tree/v8

Release is expected very soon - in the meantime we should start migrating as we'll definitely want this in v1

Science Fair will be demoed at the Decentralised Web Summit by @Karissa

This is the stuff we need to do to prep:

• sync the eLife corpus (@blahah) --> https://github.com/codeforscience/sciencefair-elife
• generate eLife bibJSON metadata collection by parsing the XML (@blahah)
• code for linked metadata and files hyperdrive live feeds (@mafintosh)
• strip back Science Fair to the core features (@blahah)
• electron OSX app packaging script (@freeman-lab)
• publish the eLife metadata and files as a linked feed (@blahah + @mafintosh)
• seed the feed from a bunch of places so it's fast (@mekarpeles ?)
• incorporate the feed as the default data source (@blahah)
• run through the planned demo and iron out bugs (@Karissa)
• BONUS: add this paper + PDF viewing [landing, DOI record, PDF]

Working in the dat branch...

The bunyan logging library is used by one of our dependencies, and it tries to import native bindings for error trace reading. For whatever reason the library that provides those readings, dtrace-provider, doesn't rebuild properly for electron, so the import fails. Three errors are printed - they're harmless but it would be good to get rid of them eventually.

Track progress on this in bunyan here: trentm/node-bunyan#216 and here: trentm/node-bunyan#487

Was searching for a keyword of which I suspect had no results. The page was left as "searching" without returning any results. I think it might be useful to provide some feedback so people know when there are no results. Perhaps sources scanned or similar, or time out after a while. Just so people don't get frustrated haha.

Possibly related to #66

Tried to run sciencefair on my local machine on Node 5.1.1.

Uncaught Error: Cannot find module '/private/tmp/sciencefair/node_modules/sqlite3/lib/binding/electron-v0.36-darwin-x64/node_sqlite3.node'

Anything obvious that, I'm missing?

What would be nice is if we had prerelease packages for OSX/Win just so we can demo it. Would that make sense at this stage?

Alpha and development builds are blocked by macOS Gatekeeper.

Builds should be auto-signed by electron-builder (see #44 & #49).

• buy mac developer account
• generate app certificates
• sign certificates and store securely for CI retrieval
• setup certificate urls and passwords as encrypted variables in CI settings
  • travis
  • appveyor
• test code signing of build artifacts
  • macOS
  • windows

Show a dashboard summarising search results, perhaps sitting in the space to the right of the search bar, with cards that can also be separate packages.

For example:

• histogram of date distribution
• top ten authors by number of authored papers in the result set
• top ten annotations of any given kind (see #16)
• most cited papers in the results

Currently when we start in dev mode we download an example db using dat, but while we're still rapidly iterating it might be easiest to just bundle the example db with the app, as it's pretty small.

[Beta6] After successfully downloading papers, double clocking them from Saved does not load Lens. Furthermore, while hover highlighting is still present, all UI elements lose click interaction - the app's UI is essentially locked up after attempting to read a paper.

Hi I came and checked it out but have no idea what Science Fair is meant to do, or who it's for. Maybe add a sentence or two about the project and what it is trying to achieve, so people can then evaluate and comment on the basis of the project's goals?

I am sure that would help others who come visit too. And congrats on finally getting a big project out the door.

Using hyperdrive

Hey, I guess the amazing Lens reader rely on the browser search functionalities?
How do we search from the electron app?

@freeman-lab's pdf-to-png allows creating a thumbnail from a PDF.

It will be slow to generate these on the fly for each client - it requires downloading a full PDF to create the thumbnail. So a nice setup task is to generate a dataset of thumbnail images from all the papers in our set.

We can start with EuropePMC - download all the PDFs and use pdf-to-png to generate thumbnails with DOIs as filenames. But it should be equally doable for all the open access sources, and even for non-OA papers a thumbnail should be legitimate.

It would be cool to put the images somewhere reusable and behind a nice API.

something like https://www.npmjs.com/package/summarizer modified to play nice with JATS XML or, worst case, PDF

It would be cool to associate individual figures with each paper, and provide thumbnails that can be followed to the full-size images.

I solved a good chunk of this problem for a related project, but the solution only works for articles in EuropePMC: open_access_figures.

The getfigures script there runs getpapers with a user-specified query to download the XML and figure images (where available) for all papers matching the query in EuropPMC. It then parses the XML to identify the figures and extract the captions, and it also generates URLs linking to the thumbnails on EuropePMC (which are predictable once you have the full image filenames).

So, I think the initial goal should be to generate the following:

• a dataset containing all the figures from EuropePMC
• another with the thumbnails for those figures
• metadata for each article - an array of figures with:
  • caption
  • thumbnail (url / checksum)
  • full figure image (url / checksum)

Here's how to achieve that:

• the captions can be generated using the EuropePMC fulltext dumps here:
• the full figures can be pulled using getpapers (or the EuropePMC api directly)
• the thumbnails can be scraped once the full images are pulled, the same way as in the getfigures script linked above.

Help and ideas very welcome. I'm happy to do this myself, but won't get to it for at least a few days, possibly longer.

using sciencefair-1.0.0-beta8.dmg on mac os 10.10.5. update: same on sciencefair-1.0.0-beta7.dmg

Windows version (on Windows 10 Creator's Update, x64) stalls at a grey screen with an animated square. Window title is 'initial'.

Performance, size and complexity benefits to doing so.

Read about choo v5 here: https://medium.com/@yoshuawuyts/choo-v5-bc775b007b5e#.zbt2pdmwk

And the changelog: https://github.com/yoshuawuyts/choo/blob/master/CHANGELOG.md

Just an issue to track ideas on how to distribute the search index generated from the initialization step

https://github.com/yoshuawuyts/nanocomponent/

we're on 0.36.8, and the most recent is 1.2.1

Very exciting project. I'm really looking forward to using it.

When I install 1.0.0-beta3 on Ubuntu 16.04 using the deb the install appears to run fine, but when I launch the application I just get an empty gray window:

If I install using npm and everything from command line, everything works, but installing from release .deb gets to 99% sync, and then subsequent restart just has the tiles flickering.

On Linux and OSX, beta9 seems to work pretty well. On my rather paltry windows testing machine however, the app appears to complete the initial loading and presents a search screen, but never returns any results.

When I open the devtools console (Ctrl Shift I or F12), I get an error from leveldb, that the process cannot get a handle on the LOCK file.

I initially thought that this was a problem with the way leveldb constructs the LOCK file path (i.e. with a / as opposed to an OS-specific path separator, but after trying to manipulate the path inside a running electron instance I don't think that's true.

I'm looking for anyone with a Windows machine (preferably better than my netbook) and some knowledge of levelDB to help debug. If any interested parties could download beta9, run it, and if anything seems off open the console and paste the error here I'd appreciate it.

If anyone can offer any insight into the issue (assuming it is reproducible), that would be even better.

I wanted to select all text to clear it but it wouldn't work. Perhaps b/c preventDefault is called? Idk haha. Anyway: figured feedback might be useful (:

Saved items counter goes up or down by 1 when an item is double clicked.

This happened while items were in a downloading state. Cannot verify whether fully downloaded items behave similarly due to #51 .

EuropePMC runs various text mining pipelines on their papers. We can get the datasets and make a sciencefair package for each one.

For example, adding eupmc-premined-genes would get the genes dataset. Now when you search for a gene name, you not only get the papers with that gene name in their title, but also those with that gene annotated in the text. The annotations could also be displayed inside lens.