Giter VIP home page Giter VIP logo

gogs's Introduction

Gitter Docker Automated build Build Status

About Gin Gogs

gin-gogs is the web interface for the GIN (G-Node INfrastructure) services.

What is gin?

Management of scientific data, including consistent organization, annotation and storage of data, is a challenging task. Accessing and managing data from multiple workplaces while keeping it in sync, backed up, and easily accessible from within or outside the lab is even more demanding.

To minimize the time and effort scientists have to spend on these tasks, we develop the GIN (G-Node Infrastructure) services, a free data management system designed for comprehensive and reproducible management of scientific data.

Why should I use GIN?

Manage your data from anywhere

  • Upload your data on a repository based structure: you can create as many repositories as you like.
  • Access your data from anywhere: once the data is at the main repository service you can securely access your data from anywhere you like.
  • Synchronize your data: you can download complete or partial repositories on any workplace you like, work on them locally and upload the changes back to the main repository.

Version your data

  • When changing your files and uploading them to the server, the history is automatically kept, you can always go back to a previous version.

Share your data

  • Make your data public: if you want to make your data accessible to the world, just make your repository publicly available. The data will be accessible but only you will be able to change it.
  • Share your data with collaborators: you can also share repositories with other users of the GIN service making it easy to jointly work on a project.
  • Make your data citable: through the gin DOI service you can obtain registered identifiers for your public datasets.

Choose how you want to use our service

  • Register with the GIN services and use the provided infrastructure.
  • Set up and host your own in-house instance - our software is open source, you can use it for free.

Enhanced search of your repositories in development

By indexing the repository contents it's easy to find the files you are looking for. When using the NIX data format for scientific data and metadata, even the contents of these files will be indexed and searchable, making it easy for you to identify the data you are looking for.

Acknowledgments

GIN is based on Gogs

License

This project is under the MIT License. See the LICENSE file for the full license text.

gogs's People

Contributors

0rax avatar 0xbaadf00d avatar achilleas-k avatar acusti avatar andreynering avatar bkcsoft avatar cgars avatar chriswatt avatar codeskyblue avatar compressed avatar crohr avatar crystaldust avatar dependabot-preview[bot] avatar fanningert avatar fnkr avatar fuxiaohei avatar gibheer avatar lstahlman avatar lunny avatar makhov avatar manfer avatar mpsonntag avatar nanoant avatar nussjustin avatar nussjustin-hmmh avatar psmit avatar sapk avatar slene avatar strk avatar unknwon avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

gogs's Issues

add type to datacite info

corresponding to resourceType / resourceTypeGeneral to indicate whether it is a Dataset, Software, Model, DataPaper, etc.

Explore pager

The pager in the explore section of the gin webpage keeps on going even if there are no more repositories listed resulting in a completely empty page.

WebDAV on GIN

Respective GIN issue: https://gin.g-node.org/G-Node/Info/issues/35

The current plan is to keep WebDAV access disabled while we work on bringing our fork up to date with upstream GOGS. The update will bring file content access through the web API which can serve similar purposes.

In the future, we may provide WebDAV access to repositories using a separate service.

Extra slash in git clone URL

Our git clone URLs, for the default port, are in the form:

[email protected]:/user/repo

This differs from most git services and also introduces some issues with git module linking on the web frontend.
I'd like to revert the change that caused this and slowly phase out supporting this form.

More general references in datacite.yml

The references field in the datacite file should be more general with respect to the reference ID.

references:
    - 
       id: "xxxxxx"
       reftype: "IsSupplementTo"
       description: "Publication name or description"

The explanation text should explain that ID could be a DOI, URL, ISBN, arXiv, or PMID. They should be provided in the form type:ID e.g., doi:10.xxx/zzzz, pmid:xxxyyy, etc.

Update the validator in G-Node/gin-doi to accept this form.

Simplification of collaboration features

Psychological simplification of data collaboration features.

Situation: When a researchers uses the GIN services and decides to share his/her data with others. Traditionally, other collaborators would be required to 1. Visit the GIN homepage 2. Create & verify an account. 3. Use GIN online or set it up locally. 4. Collaborate with the researcher.

Problem: This process is seen as tedious, as the researchers aims to minimise any potential hurdles for the collaboration with others.

Solution: Implement an 'Invite others via email' feature. The researcher can invite other collaborators via email. These will receive a link to the GIN data via email and are prompted with a log-in screen with their email already filled in as their username. What remains to do is only to decide and enter a password. After doing so, they are directly forwarded to the desired collaboration repository.

Advantages: This eliminates the first step of the situation and thereby a huge step of a potential psychological collaboration hurdle. The collaborator no longer needs to decide on whether or not he/she wants to sign up and collaborate. The decision is already made for them. It is only required to decide on a password. This solution therefore enhances collaboration without the risk or compromising security.

Show DOIfy button for re-registering new version

After PR #76 is merged, we'll be able to support repositories with multiple DOIs, so we should also make it easier for users to register new versions of already-registered repos.

The DOIfy button should still be hidden (or disabled) when the latest valid DOI matches the master commit though, so it should appear once the user makes changes to the repository.

Allow git-annex to recognise repo as an annex-capable remote

Its own test fails like this:

$ git annex init --debug       
...
[2018-09-26 11:53:56.198970849] Request {
  host                 = "web.gin.g-node.org"
  port                 = 443
  secure               = True
  requestHeaders       = [("Range","bytes=0-"),("Accept-Encoding","identity")]
  path                 = "/INT/multielectrode_grasp/config"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}

download failed: Not Found

Grant all non-owner members of an organization the right to create repositories

I was recently talking to a professor about hosting the data of his lab on GIN. While discussing rights management, he opined that it would be good if all people added to an organization could create repositories in the organization. Without this, every time a lab member wants a new repository, either the professor or a designated lab member with admin rights would have to create a repository and set appropriate member rights for it. Currently, the only way to allow everyone to create repositories in the organization is to add everyone to the "Owners" team, which gives everyone admin rights to all repositories, which could lead to someone messing things up. If individual non-owner members could create repositories, they would get admin rights to the repository in addition to the owners of the organization. They can then continue to push and share the data of that repository without requiring the involvement of anybody else.

Configuration overhaul

Following PR #88 the configuration and app.ini have been overhauled. We need to review our app.ini before deploying.

Initial impressions/questions: Default FAQ, ssh urls,

I am going through the https://web.gin.g-node.org/G-Node/Info/wiki/InHouseWithoutDocker . Thank you for the nice documentation -- it seems has worked out for me quite splendidly! Here I want to outline a few points which I have observed and which you might (or might not) want to address or just explain (decided not to separate into separate issues for now, added empty checkboxes to signal "something to address"). Some issues might relate more to upstream gogs .

  • There is no default FAQ page (http://localhost:3000/G-Node/Info/wiki/FaqTroubleshooting gives 404 page)
  • no options to initialize git-annex while creating a repo In above FAQ I was hoping to find on how to initialize "git annex" repository. I could not find an obvious setting somewhere while initializing a new repo or after to invoke "git annex init" on the server side. If such settings group existed (during creation) would be nice to be able to state desired backend for the annex (e.g. MD5) to partially mitigate those problems under windows. E.g. in datalad, by default, we just now use md5 backend:
$> datalad create /tmp/test; cat /tmp/test/.gitattributes
[INFO   ] Creating a new annex repo at /tmp/test 
create(ok): /tmp/test (dataset)                                                  
* annex.backend=MD5E

which should be sufficient for any scientific needs.

  • incomplete ssh urls I have configured to not use builtin ssh. ssh urls provided by web interface "aren't complete" and rely on ssh account having entering the shell at the level of ~/gogs-repositories. I guess, whenever no internal ssh server is configured, I guess ssh urls need to be adjusted to reflect that . Anyways, I guess I better need to try the deployment with builtin ssh server, since I guess otherwise no multiple users would be able to push data
  • http://... does not support git-annex I "git annex init"ed server side repo by adding an ssh remote, and letting git annex sync to init it, which doesn't work since
[Macaron] 2018-01-23 11:15:16: Started GET /yarikoptic/test1.git/config for [::1]
[Macaron] 2018-01-23 11:15:16: Completed GET /yarikoptic/test1.git/config 404 Not Found in 4.42299ms

i.e. git-annex trying to access that /config but it is not "interfaced" by the http/web interface. I wondered if it might become sufficient to make /config provided for git-annex so it could deduce uuid of that remote repo and request content straight via http?

I guess some of those issues are addressed via custom handling in gin cmdline client, but I thought it might be nice if GIN could work via native "git-annex" commands

If no objections, I will keep adding notes here as I go along. And please take those above points not as a critic of some kind, and just as my humble attempt on feedback to make GIN even better ;)

Make "People" and "Teams" on Organization pages clickable

The issue will be closed without any reasons if it does not satisfy any of following requirements:

  1. Please speak English.
  2. Please ask questions or config/deploy problems to our gitter channel: https://gitter.im/gnode-gin/Lobby
  3. Please take a moment to search that an issue doesn't already exist.
  4. Please give all relevant information below for bug reports; incomplete details considered invalid report.

You MUST delete above content including this line before posting; too lazy to take this action considered invalid report.

  • Git version:
  • Operating system:

Description

...

Breadcrumbs when exploring a repository

When exploring a repository, having breadcrumbs somewhere at the top of the page can be help in keeping track of the current position in the folder structure of the repository and easy relative navigation.

Repository access keys

Provide per Repo access via access keys.
Probably via. system users with access to a single repository

404 error when transferring ownership of repository

Transferring ownership of a private repository from my personal account to another personal or organization account results in a 404 error. As result, the repository is neither in the sender's account nor the receiver's. Nonetheless, the Dashboard has an entry " username transfered repository username/repo_name to receiver/repo_name".
My application version is: 0.11.91.0811

Update app.ini based on newest changes

Some settings such as CAPTCHA for viewing and downloading files (see below) doesn't map to any functionality any more and should be removed from the sample.

gogs/conf/app.ini

Lines 114 to 117 in 85a0212

; File size from which on raw view is protected by captures
RAW_CAPTCHA_MIN_FILE_SIZE = 50
; File size from which on repo file view is protected by captures
CAPTCHA_MIN_FILE_SIZE = 5

include "size" information for files/directories

With my datalad hat on, I find it very useful to know how many data files I do serve from http://datasets.datalad.org for any sub-directory/dataset -- thePROVIDED/TOTAL indicator in Size column. It would be nice if I could see in GIN/gogs also similar information -- currently I have no clue about total size of the dataset or for which subdirectories I did upload data (and for which forgot), see e.g. https://web.gin.g-node.org/yarikoptic/ds000001 and try to answer above question ;-)

Zip file download of annexed data without creating zip file

A zip function that reads annexed data already exists in libgin/archive. It might be possible to write the output of the zip function to the web response writer, feeding the data to the response without creating it on disk and without waiting for the zip to finish.

The downsides will be that the download can't be resumed on interruption and there's no way of knowing the final size beforehand, so no progress.

Team management API endpoints

Add Team management API endpoints for repository owners. Currently, only site admins are allowed to create and assign organisation teams through the API.

Normally, I would add this upstream and merge down, but a current project requires this quite fast so we'll have to do it the other way around this time.

Wait for annex uploads to complete before triggering hooks

If a git push is also followed by an upload of annexed data, any hooks that rely on annexed files being present will get triggered and then fail.

This is important for gin-valid, gin-proc, and the (upcoming) upload service for meta.g-node.org.

Revert templates to upstream and add our own to the custom directory

The upside of this is that keeping up to date with upstream is simpler, since it won't create merge conflicts. The downside is that if something changes in the upstream templates with respect to the information they have (variable name change, for instance), it will break our templates without warning.

One solution to the downside is to keep up to date with GOGS more regularly and follow the changes closely, instead of merging blindly.

We could also write tests that check if all pages render properly.

merge upstream changes?

The issue will be closed without any reasons if it does not satisfy any of following requirements:

  1. Please speak English.
  2. Please take a moment to search that an issue doesn't already exist.
  3. Please give all relevant information below for bug reports; incomplete details considered invalid report.
yoh@hopa:~/gocode/src/github.com/G-Node/gogs> git show v0.11.34 | head
commit 6f2347fc71f17b5703a9b1f383a2d3451f88b741
Author: Unknwon <[email protected]>
Date:   Wed Nov 22 14:46:14 2017 -0500

    locale: sync from Crowdin
    
    Fixes #4811

diff --git a/conf/locale/locale_fr-FR.ini b/conf/locale/locale_fr-FR.ini
index c04c931c..4754f168 100644

yoh@hopa:~/gocode/src/github.com/G-Node/gogs> git describe --tags 
v0.11.19-187-ga37c1225

Hide buttons for disabled features

Previously we had a gin@home fork and container image that was modified for on-premise deployments of GIN. The modifications disabled features and UI elements that require extra (micro)services that we run ourselves:

  • Data search (gindex)
  • Dataset publication (gin-doi)

Maintaining a fork with these changes is cumbersome. Instead we should hide the UI elements (e.g., the Request DOI button) when the service is not configured.

Login redirect is not resolved

When a page requires a login, the target page of a redirect still shows up in the address bar before a login, but the redirect is not resolved after the login is done.

Tag with a DOI "registered" releases of the datasets

ATM registered releases of the datasets get a DOI based landing page (like https://doi.gin.g-node.org/10.12751/g-node.70e11f/), which links to a dataset under https://gin.g-node.org/doi (like https://gin.g-node.org/doi/AIDA_dataset), which is a fork of the original dataset (like https://gin.g-node.org/pallastn/AIDA_dataset). But neither in the original dataset or the one under doi/, there is a tag for that specific DOI. I wondered if it is just that nobody thought to add it yet, or there is some tech limitations of bogs etc? At least locally git would be happy to tag with that DOI, e.g.

$> git tag -m "describe the release/DOI'ed version" 10.12751/g-node.70e11f doi/master

$> git describe master
10.12751/g-node.70e11f-11-geb9ce5c

Button for download with annexes from website

I think that the download button for the repo should actually download the full repository with the annex data. Currently it seems to only download the repo with the linkers to the file, and I can't find any way to download the full content without going file by file.

Fix search API endpoints

Since PR #47 the API endpoints for search and suggest no longer work. The web search uses internal functions for searching and filtering with the user's cookie. The API endpoints should be rewritten to use access tokens and reuse functionality.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.