jayrbolton / dat-wot Goto Github PK
View Code? Open in Web Editor NEW[WIP] A decentralized public key network with encryption utilities for data collaboration
[WIP] A decentralized public key network with encryption utilities for data collaboration
Using sodium-universal
makes it easier to use dat-pki
in browser environments.
I was thinking that metadat could have two types of "device networks"
A "device network" would simply be redundant devices that pull and share all the exact same stuff over the network. Since users will likely span multiple devices, you can replicate their full metadat directory over the two devices. Whenever you create a dat on one device, it gets auto-pulled and shared on the other device.
(If one device in a user device network is stolen, would we need to revoke the pgp key? The PGP private key would be encrypted using the user's passphrase, and all their data could be encrypted, so maybe it'd be ok.)
Likewise, goups may want to provide more bandwidth or storage for their dats. These "group device networks" would pull all the dats from a group, as soon as they are created for that group, and store and share the data. These group networks would not need to store any user information, but would just replicate the group dats.
https://en.wikipedia.org/wiki/Web_of_trust
A major initial problem with authority-less key sharing (ie adding someone as a trusted contact), is being able to verify that someone is who they say they are. Pgp has a system where people can sign each other's keys, so you can verify that someone is who they say they are based on other contacts you have in common. It would be cool to have a UI that showed this web of trust for new contacts
Can swap out our lil lib file with https://github.com/joehand/dat-download
the createDat
function could take an option for who you want to share it with, which could be an array of user ids. That dat address should be placed in a dats.json
file inside each push-relationship dat for each contact.
So metadat is setup for
However, it would be nice to add push-based messaging. This would allow user A to message user B without user B knowing user A. The push messages could be handled by the receiver according to a hardcoded set of message types built into metadat (eg "initiate-contact", "send-dat", etc). Maybe this can be done with hypercore
In order to find and follow cool public metadats (eg "NPR's dats""), simply enter a public metadat address. With a UI, that address could have a clickable link.
To follow someone, you enter the address of their metadat, which gets saved locally/privately. Then all their public dats can be listed.
Adding a contact allows you to give them private access to certain dats. To create this relationship, both contacts need to initiate a contact request with the other.
On the creation of a new contact requset, a new dat gets generated that will hold the relationship metadata ("relationship metadat"). Once one contact initiates the contact request, that relationship metadat address gets encrypted using the other contact's public key and placed in their public metadat. Once both contacts read each other's relationship metadat addresses from each other's public metadats, then that encrypted address can be immediately deleted from their public metadats.
For each contact, two relationship metadats should be saved, one that contact A writes to and contact B reads from, and the other for contact B to write to and contact A to read from.
Once both relationship metadats are established, then contact A can start privately sharing dats with contact B, and vice versa.
When you create or edit a dat, you can explicitly set which among your contacts have access to that dat. For each contact that you give access, the dat address will get saved into the relationship metadat that you have for each contact. If you make the dat public, then the dat address simply gets listed in your public metadat instead.
When you set permissions on a dat, it might be handy to give access to a whole group of contacts all at once, without having to select individually.
After a contact is established, you can simply put a group label within the relationship metadat. Each contact could request to be in a group from the other contact. When you set permission on a dat, you can select groups as well as individual contacts.
Similar to creating a regular contact, you create a fully-trusted relationship between two devices. Any private dat address on one device would be immediately put in the "relationship dat" and listed and downloadable on the paired device (eg your laptop and your phone). This could be thought of as a kind of "full-trust device network". For each dat you create on any device, you could choose which other devices in your trust network should auto-download the full dat.
Since everything is based around PGP keys, we could encrypt all metadata in all metadats. Even though the dats themselves are private, and are shared over SSL, if your device was stolen, then someone could find all your metadat information. Encrypting everything locally would ensure that simply stealing the device wouldn't expose all your dat addresses. All dat addresses that are shared with other contacts are encrypted using their pubkey.
Could have some simple device-to-device messaging by placing a pgp-encrypted message in your relationship metadat for someone else using their pubkey.
If you download a dat and want to edit it, there could be a simple method for initializing a new dat with the edits and notifying the initial dat creator of your edited dat address, kind of like a fork.
Say you have a dat with files in a certain format (.csv
, .md
, etc). When you click the file, you could choose a "renderer" for that file. The renderer could be a JS/HTML application that takes the file as input . You could simply provide the .git
url of the renderer app when opening the file (and in a UI, the "renderers" would be pre-listed). This would open up potential to build open-source renderers for spreadsheets, wiki data, etc. The renderer itself does not need to be part of the dat, and can just be pulled off github or similar.
Since pgp is built into it, it would be easy to support pgp encryption of arbitrary files inside dats. For regular dats holding any data, metadat could handle encrypting the file and saving the encrypted version of the file to the dat, rather than any plaintext. If the dat is shared among users and groups, then metadat could automatically handle multi-key encryption of the files using the pubkeys of all the other users who have access to the dat.
Since dat addresses are private, and data is sent over an encrypted connection, the only advantage of this kind of encryption is if your device is stolen or if someone steals your login.
Could potentially have a commenting and message-threading system on dats and on dat files, all encrypted using each user's pubkey
So the only downside is that there's a temporary file (the encrypted address) that could indicate to the public that a contact request is pending (maybe it could be pretty obfuscated though). The upside is no-one will ever be able to read the full list of your contacts, and nobody would ever have a clue about what you're sharing with whom.
the total non-recoverability of keypair passphrases is a little rough for user experience. Probably the biggest reason people lose private keys is because they lose the device it's on, or the filesystem gets wiped. We could copy encrypted private keys for every device to every other device in their device network. Any UI built on top of this library should encourage users to add many devices to prevent key loss.
This may be way out of scope or unnecessary, but wanted to leave this here...
Users should be able to share a dat anonymously. Other users who download this dat would not be able to trace (or easily trace) the source device of the dat share. Could possibly do some kind of onion routing style thing over dat
Maybe related:
https://www.net.t-labs.tu-berlin.de/papers/M-UORWP2PA-08.pdf
https://www.freehaven.net/anonbib/cache/conf/iwsec/PalmieriP14.pdf
First answer here is pretty good: https://security.stackexchange.com/questions/91704/which-strategy-to-encrypt-data-accessed-by-multiple-users
Summary of group encryption with asymmetric keys:
People suggest we use ECC keys instaed of 2048-bit (or 4096) RSA, which can be generated with libsodium
The password hashing functions and some other things are slow, and the full test suite takes almost 60s to run. Maybe we could split the tests into separate files and run them in parallel. The trick will be to aggregate exit codes in all the child processes. Maybe there is a package for doing this? Possibly this: https://github.com/DiegoRBaquero/bogota
@lukeburns : I saw your post here and was excited to hear about your potential use cases.
I initially started this project with UI in mind for collaborative data backups, but hopefully the public/private key sharing system could be generic enough that it could support any kind of public and private user-to-user identity and communication over dat. Check out this rough high-level overview I wrote our here: https://github.com/jayrbolton/dat-pki/wiki/How-it-Works. If you have any specific feature requests, ideas, etc for your use case, please post them here.
The awesome dat page is not updated for a while.
Time to collect all these cool projects that have popped up since.
Care to add an entry to it?
This library will probably be used in other applications and so should be async. There are some synchronous fs calls
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.