Giter VIP home page Giter VIP logo

davros's Introduction

Davros File Storage

Davros lets you store all of your files in the cloud and sync them between your devices.

Screenshot of Davros

Try Live Demo

Prerequisites

Davros is built to run inside Sandstorm, an open source web application platform. You can either run Sandstorm yourself or let someone else host it for you.

Installation

Development

Davros is built using Node.js and Ember.js. To run locally, you'll need to install node.js. We also recommend that you use yarn to install dependencies since that will lock versions to the ones that are used during deployment.

To install yarn run npm install -g yarn Then:

  • yarn install
  • yarn dev --port=3009

At this point you'll have Davros running at http://localhost:3009. Substitute 3009 for another port if you want. Note that running Davros this way is not particularly safe; it relies completely on Sandstorm for user management and authentication.

In development, you can connect your desktop client to http://localhost:3009/ with any username and password.

Sandstorm Development

A few parts of Davros are dependent on running within Sandstorm, such as the authentication instructions on the Clients page. To run inside Sandstorm, first get vagrant-spk. Then, within the Davros directory, run

vagrant-spk vm up
vagrant-spk dev

While this works it requires that you re-run vagrant-spk dev manually anytime you make a change. To have this done automatically consider using ember-cli-vagrant-spk instead which will monitor your filesystem for changes and restart the server for you.

Linting

  • yarn lint:hbs
  • yarn lint:js
  • yarn lint:js -- --fix

Releasing

Releasing an app is a little bit convoluted. This assumes you are on a non-linux machine, running sandstorm via vagrant-spk. On linux, you might be able to get away with skipping step 1.

  1. vagrant-spk vm ssh then cd /opt/app && rm -rf node_modules/sharp && yarn -- this is because the sharp module has native components that need to be built on linux
  2. Edit .sandstorm/sandstorm-pkgdef.capnp and update appVersion and appMarketingVersion. Bump major version for major breaking changes, minor version for significant new features, and patch version for tiny features and bugfixes.
  3. Edit CHANGELOG.md and add a section with your new version.
  4. Run yarn build to build the UI.
  5. Run yarn build-server to build the backend.
  6. Run vagrant-spk dev and navigate around the app testing various functionality. This is generally good to do, but when you exit, this will also update .sandstorm/sandstorm-files.list with any new files.
  7. Run vagrant-spk pack build/[VERSION].spk (replacing [VERSION] with the version you chose in step 2)
  8. On a sandstorm instance, upload the packed app file and install it. Test it to ensure everything works properly and that all files were included.
  9. Commit any uncommitted changes and tag them v[VERSION].
  10. Run vagrant-spk publish build/[VERSION].spk

Acknowledgements

License

See LICENSE file.

davros's People

Contributors

ember-tomster avatar garrison avatar griff avatar jontheniceguy avatar mnutt avatar ocdtrekkie avatar timbertson avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

davros's Issues

Menu requires two taps on Android/Firefox

Opening the menu requires two taps, and selecting a menu item also requires two taps. It's most likely an ember-paper bug, but we should figure out a way to mitigate it.

Long uploads get interrupted frequently

I'm trying to upload about 30GB to a Davros share, and it seems as though the grain is getting shut down at regular intervals, possibly because the keepalive is expiring.

Fork Idea: Synchronized Text/Markdown Files

So I realize this is outside of the scope of Davros, but I've been pining for a really simple markdown editor that lets me edit from a folder on my computer and then backs it all up (Davros), but then ideally I could edit those files from within Sandstorm as well.

I suspect this is outside of the scope of Davros, but it would be cool to take the functionality that Davros supplies through a fork and focus it on something like this.

Thoughts?

Stretchgoals: Jekyll publishing to github, or something similar.

Question: How to dynamically list files in index.html?

Although I'm a server-side Java developer, I do appreciate some of the technical complexities of my question. I would presumably write some Javascript to hit an endpoint exposed by Davros to get the file list, if such a consumable endpoint even exists.

I'm using Davros on Sandstorm.

Thanks for your time. Please feel free to delete this question if it's not productive.

Upload not working in 0.14.0

Hey,

when I press the ''Upload" Button nothing happens. Tested on our own sandbox and in your demo server.

My JavaScript console says:
Error: Nothing handled the action 'nothing'. If you did handle the action, this error can be caused by returning true from an action handler in a controller, causing the action to bubble.

I'm using Firefox 41.0 on Ubuntu 14.04 AMD64.
Also tested Google Chrome Version 46.0.2490.80 (64-bit)

Thanks!
Nils Jürgens

Request: download directory as zip

I'd quite like this feature, for sharing files with users who don't want to set up a webdav client just to grab a batch of files.

I was looking at the code, and it seems like it would be nice to use the dav server's tree functionality, but to build a zip on the fly rather than whatever stream dav pumps out. But I couldn't see a good way to access the dav server's functionality directly, it only seems to expose a low-level HTTP handler.

So then I looked into adding zip support by using something like https://github.com/archiverjs/node-archiver to add a download endpoint independent of the DAV stuff. But that feels pretty dodgy, as you then have to double up the security / validation (which I'm hoping DAV does) for issues like requesting paths outside of the DAV folder, or zipping up a directory with symlinks to /etc/passwd

Do you have any plans for this feature, or advice on what the best approach might be?

Davros not really working in v0.170

Since the weekend update of Sandstorm, we can't really use Davros anymore; it stays hanging, uploads functions some times, sometimes they don't; then the whole thing is starting to function great for a while and then back to hanging.

How well does Davros handle special characters in file names?

And does it have or cause any issues if used to sync between different OS-es like Win/OSX? Just asking as I even had some issues using rsync as I had not known beforehand that UTF8 != UTF8 (Found out the hard way the difference between UTF-8 NFC & UTF-8 NFD)

"just now ago"

When modifying a file, the date of the modification is first reported as "just now ago"

Support for other DAV clients

Hi all, tried to connect 2 different clients to Davros, but no luck

  1. mount.davfs2 - Linux
  2. ExpanDrive for Windows/Mac

I suspect the issues are client-specific and has nothing to do with Davros, but want to raise them here anyway to see if anyone has already found a solution or root cause.

mount.davfs2 seems to think the https://sandstorm:<...>@api.example.com URL is invalid

ExpanDrive seems to fail with some SSL errors (it uses Python ssl lib under the hood)

Unable to connect Windows ownCloud client to Oasis Sandstorm grain

Hi,

I try to connect the ownCloud client on Win 7 64 bit to a Davros grain hosted on Oasis Sandstorm. After entering the credentials that Davros offers at the /clients tab the ownCloud client replies with "Connection closed".

This does not seem to be #12 which is limited to iOS.

Any Idea what I can do to narrow that down?

uploading images on mobile causes images to glitch

Uploading images on mobile (whether through ownCloud mobile auto-upload from the camera or through selecting files to uplaod in the ownCloud mobile client) causes images to be glitched/scrambled. Here's a glitched image on Oasis:

image

And also when synced to desktop:

image

.jsdav file visible in folder

While syncing is in progress between owncloud-client and Davros, I see this .jsdav file, which I probably shouldn't:

screenshot1

Desktop client "open in browser" doesn't work

It can't actually open the grain, due to the grain not having knowledge of its own URL. And I'm not sure that it's even something configurable within the desktop client, it may just be trying to go to the root of the webdav URL.

This is unfortunately probably not fixable.

Javascript errors running in vagrant-spk dev

Aside from a few teething problems setting up the development environment (#46), I'm getting console errors now that the davros UI is running:

screenshot from 2016-04-28 21-24-14

(the UI itself is just a blank screen)

Any ideas?

renaming folders doesn't work

When I create a folder locally in my Davros/OwnCloud folder (on my Mac), it temporarily creates a folder called "untitled folder," which, by the time I've typed in the folder name I intend to call it, creates an error and is unable to sync with my Davros grain (which already knows about "untitled folder").

My current workaround is to create a folder elsewhere (e.g., on my desktop), make sure it's named correctly, and drag/drop it into my Davros/OwnCloud folder. When I do that, it works fine. :)

Files uploaded via web UI do not appear in files list immediately

After uploaded a file via the web UI, the file does not appear in the files list immediately. The page must be refreshed.

Environment: Sandstorm Oasis, Firefox 42, Fedora 23

Steps to reproduce:

  • Select Upload from the web UI menu
  • Select a file from the file browser, click "open"

When the upload finishes, the file list does not show the new file, but it will appear on refresh. Server returns a 200 "ok" response after upload. The console shows a syntax parsing ember-cli-live-reload.js:

It looks like the request for ember-cli-live-reload.js is actually returning HTML:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <title>Davros</title>
    <meta name="description" content="">
    <meta name="viewport" content="width=device-width, initial-scale=1">

    <base href="/" />
<meta name="davros/config/environment" content="%7B%22modulePrefix%22%3A%22davros%22%2C%22environment%22%3A%22development%22%2C%22baseURL%22%3A%22/%22%2C%22locationType%22%3A%22auto%22%2C%22EmberENV%22%3A%7B%22FEATURES%22%3A%7B%7D%7D%2C%22contentSecurityPolicy%22%3A%7B%22default-src%22%3A%22%27none%27%22%2C%22script-src%22%3A%22%27self%27%22%2C%22font-src%22%3A%22%27self%27%22%2C%22connect-src%22%3A%22%27self%27%22%2C%22img-src%22%3A%22%27self%27%20data%3A%22%2C%22media-src%22%3A%22%27self%27%22%2C%22style-src%22%3A%22%27self%27%22%7D%2C%22APP%22%3A%7B%22name%22%3A%22davros%22%2C%22version%22%3A%220.0.0+03670927%22%7D%2C%22contentSecurityPolicyHeader%22%3A%22Content-Security-Policy-Report-Only%22%2C%22exportApplicationGlobal%22%3Atrue%7D" />
<script src="/ember-cli-live-reload.js" type="text/javascript"></script>
<div id='paper-wormhole'></div>

    <link rel="stylesheet" href="assets/vendor.css">
    <link rel="stylesheet" href="assets/davros.css">


  </head>
  <body>


    <script src="assets/vendor.js"></script>
    <script src="assets/davros.js"></script>


  </body>
</html>

Wrong disk usage reported

When syncing for the first time, davros reports the entire disk usage as preparing to sync. When you're expecting to sync 1MB of files and it tells you you're about to sync 3.7GB, that's unsettling to say the least.

Also, in at least one instance df is returning back a wonky value:

/opt/app/node_modules/jsDAV/lib/DAV/backends/fs/directory.js:224
                (statvfs.blocks - statvfs.bfree),// * statvfs.bsize,
                        ^

TypeError: Cannot read property 'blocks' of null
    at /opt/app/node_modules/jsDAV/lib/DAV/backends/fs/directory.js:224:25
    at /opt/app/server/dav/statvfs-shim.js:18:16
    at ChildProcess.exithandler (child_process.js:194:7)
    at emitTwo (events.js:87:13)
    at ChildProcess.emit (events.js:172:7)
    at maybeClose (internal/child_process.js:818:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)
** HTTP-BRIDGE: App server exited with status code: 1
** SANDSTORM SUPERVISOR: App exited with status code: 1

Uploading/deleting from menu not working

I'm using 0.14.1 (on Microsoft Edge), a freshly created grain on the Sandstorm Alpha server. I opened each preincluded file, and selected "Delete File" from the top right menu. I got the message prompt, but the files did not delete. I also tried selecting Upload File from the home folder, and nothing happened.

Interestingly, trying to delete it again is giving me this in console: HTTP404: NOT FOUND - The server has not found anything matching the requested URI (Uniform Resource Identifier).
(XHR): DELETE - https://alpha-37f9f984da43c5ad8f413d2bccc17466.sandstorm.io/remote.php/webdav/voyager.jpg

As if the browser/grain has not noticed I deleted it.

When uploading large files, it would be nice to have a progress bar or other indicator of progress

Steps to reproduce:

  • Create a davros grain
  • Use the menu to select a file and upload it to the grain

Expected behavior:

  • See some indication that upload is in progress

Actual behavior:

  • The only indication is that the grain size is increasing, which is an indicator, I suppose, but a very subtle one! (-:

References:

Rebuild package with Sandstorm 0.166

Sandstorm build 0.166 updates sandstorm-http-bridge to support flow control, so that large downloads don't buffer in RAM in the front-end causing OOM crashes, which are currently the biggest production issue for Sandstorm Oasis (causing brief outages about 2-3 times per week).

Since sandstorm-http-bridge is bundled into app packages, apps need to be updated for this change to take effect. Davros is one of the apps that is often used for large file downloads. Can you please rebuild Davros against the latest Sandstorm and push an update?

Thanks!

404 Not Found

The first page when I opened my grain was 404 Page Not Found. There is no way to see files uploaded to it. And once I go to "Clients", any attempt to click on "Files" just fails. But I did manage to authenticate my ownCloud client to it, and I think it uploaded. I just can't see the files.

Wishlist: Preview PDFs

I was kinda sad when I uploaded two PDFs I often share to people to a Davros grain that they weren't previewable. I could probably link them using web publishing, I suppose, but I found it odd to get "No Preview" for PDF format in the Davros UI.

Rebuild package with Sandstorm 0.168

Sorry to do this again.

Unfortunately, the sandstorm-http-bridge in version 0.166 has a subtle memory corruption bug. I have not been able to reproduce the bug with Davros, but it is observable with FileDrop by repeatedly refreshing the grain frame -- eventually, the app stops responding to requests.

This bug has been fixed in v0.168, which I just pushed now (run sandstorm update if you don't have it yet).

'#' in file and folder names causes problems

If I enter a folder name in the web interface containing #, everything after # is ignored (treated as a comment?). If a file or folder is synced from the local folder using owncloud, it shows up in the web interface, but clicking on it leads to a 404.

The server version 5.0.17 is old and unsupported

Owncloud client 2.2.0 on Ubuntu 16.04 displays error message:

Connected to (myip) as sandstorm. The server version 5.0.17 is old and unsupported! Proceed at your own risk.

It auto-pauses synchronization and I'm a little wary to unpause in case of problems.

Sync with unison (feature request)

Might it be possible to allow one to sync with a Davros repository using the Unison file synchronization tool? This would allow intelligent offline support, able to propagate changes in both directions and resilient to failure.

Personally, I have been very happy/impressed with Unison in the past and am now looking to move to Davros or another Sandstorm app for each of my file collections, instead of having them all on a server I can access over unison via ssh. Intelligent offline support is very important to me, so I am hesitant to move away from Unison and adopt something else entirely. But if there exist offline WebDAV tools that are just as intelligent as unison, knowing of them may negate the need for this feature.

WebDAV at API root for all non-Owncloud clients

We serve webdav at /remote.php/webdav for Owncloud's benefit, but it can be confusing to other clients. Many let you specify a starting path, but often have a "parent directory" link which leads nowhere since we don't serve anything at just /remote.php. Instead, clients that don't identify themselves as Owncloud should get files served at /.

"Server do not support X-OC-MTime" from owncloudcmd

While using owncloudcmd to sync files to a Davros grain, I get a message in my owncloudcmd log that says:

Server do not support X-OC-MTime ""

I suspect it would be wise for us to add support to this to Sandstorm. Presumably my client is going to submit duplicated requests to Davros based on not having access to remote mtimes, so I thought I should file this.

Slightly longer log snippet:

!!! OCC::PUTFileJob created for "https://api.rose.sandcats.io/" + "/pub/pub/scrapy-talk/_static/up.png"
void OCC::PropagateUploadFileQNAM::slotPutFinished() QUrl("https://[email protected]/remote.php/webdav/pub/pub/scrapy-talk/_static/up.png") FINISHED WITH STATUS QNetworkReply::NetworkError(NoError) "" QVariant(int, 201) QVariant(QString, "Created")
Server do not support X-OC-MTime ""
*==* duration UPLOAD 363 0 1166
OCC::SyncJournalFileRecord::SyncJournalFileRecord(const OCC::SyncFileItem&, const QString&) "./pub/pub/scrapy-talk/_static/up.png" Retrieved inode  331892 (previous item inode:  331892 )
"INSERT OR REPLACE INTO metadata (phash, pathlen, path, inode, uid, gid, mode, modtime, type, md5, fileid, remotePerm, filesize, ignoredChildrenRemote) VALUES (?1 , ?2, ?3 , ?4 , ?5 , ?6 , ?7,  ?8 , ?9 , ?10, ?11, ?12, ?13, ?14);" -1275537046966359755 34 "pub/pub/scrapy-talk/_static/up.png" 331892 0 "1374686074" "0" "e06a5c2d79aa917614dcd2977bbbd27c" "" "" 363 0
"DELETE FROM uploadinfo WHERE path=?1" "pub/pub/scrapy-talk/_static/up.png"
void OCC::SyncJournalDb::commitInternal(const QString&, bool) Transaction commit  "upload file start" and starting new transaction
void OCC::SyncEngine::slotItemCompleted(const OCC::SyncFileItem&, const OCC::PropagatorJob&) "pub/pub/scrapy-talk/_static/up.png" INSTRUCTION_NEW 4 ""
!!! OCC::PUTFileJob created for "https://api.rose.sandcats.io/" + "/pub/pub/scrapy-talk/_static/view-source.img"

IE11: 404 Not Found at root

Below is a fresh Davros grain running on IE11. If I share the grain, it works correctly in other browsers.

screenshot from 2016-02-22 17-54-43

Feature Request: Drag files into folders from web UI.

Since I can drag things into the Davros window to upload them, it felt natural that I should be able to drag files into folders in the web interface. But I can't. So I think this would be an awesome feature, since I kinda just expected it, and others may as well.

Maybe for moving files to higher level directories there can be a "./" type drag target. Or maybe the "home" link at the top, could be something you could drag a file onto to move it up to a higher level.

WARNING: etag of Apr is ` `. This must not happen.

When synchronizing, I get a message like this sometimes:

!!! OCC::LsColJob created for "https://api.rose.sandcats.io/" + "//////2008"
WARNING: etag of Apr is   This must not happen.
WARNING: etag of Aug is   This must not happen.
WARNING: etag of Dec is   This must not happen.
WARNING: etag of Feb is   This must not happen.
WARNING: etag of Jan is   This must not happen.
WARNING: etag of Jul is   This must not happen.
WARNING: etag of Jun is   This must not happen.
WARNING: etag of Mar is   This must not happen.
WARNING: etag of May is   This must not happen.
WARNING: etag of Nov is   This must not happen.
WARNING: etag of Oct is   This must not happen.
WARNING: etag of Sep is   This must not happen.
void OCC::DiscoveryMainThread::singleDirectoryJobResultSlot(const QList<OCC::FileStatPointer>&) Have 13 results for  "//////2008"
static csync_vio_handle_t* OCC::DiscoveryJob::remote_vio_opendir_hook(const char*, void*) OCC::DiscoveryJob(0x55a02ff733d0) /////2008 ...Returned from main thread
[12/19 17:19:10.503436, 9] csync_walker:  directory: ownclouds://sandstorm:(removed)@api.rose.sandcats.io/remote.php/webdav//////2008/Apr [file_id=]
[12/19 17:19:10.503493, 8] sqlite_profile:  _SQL_ SELECT * FROM metadata WHERE phash=?1: 0
[12/19 17:19:10.503508, 8] _csync_detect_update:  file: /////2008/Apr, instruction: INSTRUCTION_NEW <<=
[12/19 17:19:10.503512, 4] csync_ftw:  URI without fuzz for ownclouds://sandstorm:(removed)@api.rose.sandcats.io/remote.php/webdav//////2008/Apr is "/////2008/Apr"
static csync_vio_handle_t* OCC::DiscoveryJob::remote_vio_opendir_hook(const char*, void*) OCC::DiscoveryJob(0x55a02ff733d0) /////2008/Apr Calling into main thread...
!!! OCC::LsColJob created for "https://api.rose.sandcats.io/" + "//////2008/Apr"
WARNING: etag of 22 is   This must not happen.
void OCC::DiscoveryMainThread::singleDirectoryJobResultSlot(const QList<OCC::FileStatPointer>&) Have 2 results for  "//////2008/Apr"
static csync_vio_handle_t* OCC::DiscoveryJob::remote_vio_opendir_hook(const char*, void*) OCC::DiscoveryJob(0x55a02ff733d0) /////2008/Apr ...Returned from main thread
[12/19 17:19:10.699804, 9] csync_walker:  directory: ownclouds://sandstorm:(removed)@api.rose.sandcats.io/remote.php/webdav//////2008/Apr/22 [file_id=]

Note the etag warning. Should I be worried?

Also, here's a read-only sharing link to my grain: https://rose.sandcats.io/shared/qO_9RFvfldvC46x4Mj1DGuOxrnHtB3HYRuuhk9tUrBo

Of note:

  • owncloudcmd is trying to access /////2008 not just plain-old /2008. That strikes me as very weird. Maybe that's a bug somewhere?

upload files from mobile

When using davros on a mobile phone via web you can see the upload button but nothing happens when you tap it.

image1

Tested with iOS 9.2 on an iPhone 6S

Current behavior: nothing happens

Expected behavior: dialogue asking for access to pictures on the phone. possibility to upload one or more pictures.

I'm not sure if that works and you can get access to the files with the mobile safari browser.

Grain size incorrect again

I needed to urgently transfer an ISO from one PC to another so I thought I'd give Davros a try.
I created a grain, downloaded the client and configured owncloud. I then started copying the 2.9GB ISO into the owncloud folder and was quite impressed when the upload began immediately, while the file was still copying into the owncloud folder.

Once the upload was finished my grain looked like this (compare the 3.13Gb VS the 2.0GB)
davros12

And on the target machine, when I installed owncloud and downloaded I ended up with a 169MB file where owncloud said: successfully synced.

All this doesn't inspire too much confidence in entrusting my files to Davros :-(

Write permissions

I'm using the owncloud client with Davros to sync files between computers running OSX and Ubuntu 14.04. Files created on one have only read permissions set on the other, rather than read and write.

Preview .odt, .doc, .pages

A user (the same one who wanted PDF support) said it would be nice if .odt and .pages files could also be previewed. Obviously this would be a whole lot more complicated, but I imagine it would be feasible to find some open-source tool that can convert various formats to .pdf for display.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.