standardnotes / syncing-server Goto Github PK

View Code? Open in Web Editor NEW

209.0 15.0 48.0 848 KB

[Deprecated: Use our new Node server: https://github.com/standardnotes/syncing-server-js]

Home Page: https://standardnotes.org

License: GNU Affero General Public License v3.0

Ruby 94.85% CSS 0.07% HTML 2.75% Dockerfile 0.18% Shell 1.73% Haml 0.43%

syncing-server's Introduction

⛔️ Deprecation Notice

This codebase is deprecated in favor of our new Node.js server. For self-hosting, use standardnotes/standalone.

Standard Notes Syncing Server

You can run your own Standard Notes server and use it with any Standard Notes app. This allows you to have 100% control of your data. This server is built with Ruby on Rails and can be deployed in minutes.

Requirements

Docker

Data persistency

Your MySQL Data will be written to your local disk in the data folder to keep it persistent between server runs.

Getting started

Clone the project:

git clone --branch master https://github.com/standardnotes/syncing-server.git

Setup the server by running:

./server.sh setup

Run the server by typing:

./server.sh start

Your server should now be available under http://localhost:3000

Logs

You can check the logs of the running server by typing:

./server.sh logs

Stopping the Server

In order to stop the server type:

./server.sh stop

Updating to latest version

In order to update to the latest version of our software please first stop the server and then type:

./server.sh update

Checking Status

You can check the status of running services by typing:

./server.sh status

Cleanup Data

Please use this step with caution. In order to remove all your data and start with a fresh environment please type:

./server.sh cleanup

Tests

The syncing-server uses RSpec for tests.

To execute all of the test specs, run the following command at the root of the project directory:

bundle exec rspec

Code coverage report is available within the coverage directory.

syncing-server's People

Contributors

Stargazers

Watchers

syncing-server's Issues

App not working after restart - docker-compose

I've successfully started the syncing server using docker-compose up -d.

After a reboot the docker image syncing-server_app keeps restarting

7fb5acb2882d syncing-server_app "bash -c 'bundle exe…" 2 days ago Restarting (1) 2 seconds ago syncing-server_app_1
75eb82f6285f mysql:5.7 "docker-entrypoint.s…" 2 days ago Up 13 minutes 3306/tcp, 33060/tcp syncing-server_db_1

Only a manual stop using docker-compose down followed by a restart docker-compose up -d will start the app again

CSRF token authenticity warning

With every request we are seeing this warning entry in logs:

Can't verify CSRF token authenticity.

This basically doubles the amount of logs the app produces so we need to either fix the issue or get rid of the message.

bcrypt 3.1.13 installation error

I'm running Arch Linux server on Raspberry Pi 3 and bundle install fails when installing bcrypt 3.1.13, as reported here. I overcome this by editing this line and setting version to 3.1.12.

Finally, rerun bundle install. It should build succesfully.

Reverts back to the default server address-Wrong Credentials

I try to add the IMAP note accounts (similar to Apple notes) to the app. The server is Namecheap Private Email (powered by Open-Xchange). https://mail.privateemail.com

But the issue is that on the Standard note app, I can not save the server, and each time it reverts back to the default one. Another issue is I cannot log in to the server at all. It notifies me of the wrong credentials.

Yarn executable was not detected in the system

Trying to setup StandardNotes with docker platform following the instructions form https://docs.standardnotes.org/self-hosting/self-hosting-with-docker

While attempting to execute the app by running
$ docker-compose exec app bundle exec rake assets:precompile

I get

Yarn executable was not detected in the system.
Download Yarn at https://yarnpkg.com/en/docs/install

Host system has yarn but I'm unaware of the docker build.

OS: Debian 10
Docker: Docker version 19.03.5, build 633a0ea838
Docker-Compose: docker-compose version 1.25.1, build a82fef07

Serverless container?

I would love to run this in a serverless fashion. I tried building the docker container without the DB, and it doesn't work so well. Have you considered creating putting a container on docker hub that just needs the DB information as environment variables?

Able to log in, but POST updates to syncing-server are returning 500

Hello,

I am running Standard Notes behind an nginx reverse proxy -> Standard Notes server.

I am able to authenticate and pull in notes from my Standard Notes server to the client, but about 80% of the time, it is unable to update those notes back in the server. Sometimes it works, sometimes it doesn't. The result is conflict merge issues, with notes just continuing to duplicate on my client.

The access logs for Nginx reverse proxy show 200's and then sometimes 500's, which correspond with sync failures. Error logs show nothing at all.

Any ideas?

Here's the access log sample:

xxx.xxx.xxx.xxx - - [23/Jul/2020:18:21:51 +0000] "POST /items/sync HTTP/1.1" 200 3689 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) StandardNotes/3.4.1 Chrome/80.0.3987.158 Electron/8.2.0 Safari/537.36"
xxx.xxx.xxx.xxx - - [23/Jul/2020:18:22:09 +0000] "POST /items/sync HTTP/1.1" 500 654 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) StandardNotes/3.4.1 Chrome/80.0.3987.158 Electron/8.2.0 Safari/537.36"

Reverse proxy nginx config:

server {
        listen 443 ssl;
	listen [::]:443;

	server_name xxx;
	
	ssl on;
	ssl_certificate /etc/letsencrypt/live/xxx/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/xxx/privkey.pem;
	
	location / {
		proxy_pass http://10.0.0.100;
	}
}

server {
    listen 80;
    listen [::]:80;

    server_name xxx;

    return 302 https://$server_name$request_uri;
}

Standard Notes server systemctl service file:

[Unit]
Description=Standard Notes
Requires=network.target
 
[Service]
Type=simple
WorkingDirectory=/home/xxx/standardnotes/syncing-server
ExecStart=/usr/local/bin/bundle exec rails server -e production -b 0.0.0.0 -p 80
TimeoutSec=30
RestartSec=15s
Restart=always
 
[Install]
WantedBy=multi-user.target

syncing-server_app_1 keeps restarting Raspberry Pi

followed instructions for docker compose for Raspberry Pi, and getting the app restarting after 20-25 seconds, here is the end of the logs from the logs

Installing bcrypt 3.1.13 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /usr/local/bundle/gems/bcrypt-3.1.13/ext/mri
/usr/local/bin/ruby -I /usr/local/lib/ruby/2.6.0 -r
./siteconf20200818-7-16tfqhw.rb extconf.rb
creating Makefile

current directory: /usr/local/bundle/gems/bcrypt-3.1.13/ext/mri
make "DESTDIR=" clean

current directory: /usr/local/bundle/gems/bcrypt-3.1.13/ext/mri
make "DESTDIR="
compiling bcrypt_ext.c
compiling crypt_blowfish.c
gcc  -D__SKIP_GNU     -c -o x86.o x86.S
x86.S: Assembler messages:
x86.S:202: Error: junk at end of line, first unrecognized character is `,'
make: *** [<builtin>: x86.o] Error 1

make failed, exit code 2

Gem files will remain installed in /usr/local/bundle/gems/bcrypt-3.1.13 for
inspection.
Results logged to
/usr/local/bundle/extensions/arm-linux/2.6.0/bcrypt-3.1.13/gem_make.out

An error occurred while installing bcrypt (3.1.13), and Bundler cannot continue.
Make sure that `gem install bcrypt -v '3.1.13' --source 'https://rubygems.org/'`
succeeds before bundling.

In Gemfile:
  bcrypt

Re-Add Docker instructions

Hello,
The instructions on standardnotes.org, need preconfigured docker-compose and Dockerfiles in the repository, but in this new repo they're not there.

It would be really appreciated, if these instructions and files could be updated/added, so self-hosted servers can be migrated.

Best Regards.

Raspberry pi docker-compose version mismatch

git pull then set my .env file then ran the docker-compose command then was met with

docker-compose -f docker-compose.yml -f docker-compose.raspberry-pi.yml up -d
ERROR: Version mismatch: file ./docker-compose.yml specifies version 3.8 but extension file ./docker-compose.raspberry-pi.yml uses version 3.0

changing the raspberrypi.yml version seems to fix

Cannot connect to hostname using iOS, IP works (HTTP)

When trying to connect to a hostname, I get "Oops, a server error occurred when trying to sign in. Please try again." When I switch to the IP address that that hostname resolves to, sign in works.

Now, I would prefer to use HTTPS, but currently I'm attempting this over HTTP as I'm unclear about how to get an SSL certificate installed in the docker container.

This is leveraging the MacOS app as well as iOS; same behavior. Using docker-compose

Disable registrations not working

I've set the variable in my .env file and restarted the server but I am still able to register?

An item without a content and content-type

Using my API I seem to have accidentally created an item both without content and without a content type. As you can see by the timestamps this is from last September, so maybe a lack of input validation that's since been fixed?

(gosn.DecryptedItem) {
 UUID: (string) (len=36) "5f4f0b99-ea6b-4a8d-982e-88ca87dd7932",
 Content: (string) "",
 ContentType: (string) "",
 Deleted: (bool) false,
 CreatedAt: (string) (len=24) "2019-09-22T22:00:53.933Z",
 UpdatedAt: (string) (len=24) "2019-09-22T22:00:53.933Z"
}

This is an account I used for testing my code in CircleCI and I'm now using a new one so, if it's of interest, please have a look and then delete the account:
email: redacted
password: redacted

Docker: rails db:migrate step failing

Hello,

first off, I'm not familiar with ruby or rails.
I tried getting the docker container up and running with docker-compose and also with docker build. I'm facing the issue that the build process fails at Step 10/11 which is RUN rails db:migrate. I'm getting the Error /bin/sh: 1: rails: not found.
Now I checked the mentioned requirements and installed them. I am on a Debian distribution and I tried installing ruby and rails using apt but also using rvm with the referenced versions.
I launched /bin/sh and tried running the command and it is working. But when I try building with docker I still get the Error.

I'm sure I'm missing something really stupid here but since I can't find any more to go on googling I need to ask here.

Any help is appreciated.

/var/lib/mysql does not exist

I've successfully set up the syncing-server using docker-compose

The README states that the database will be written to /var/lib/mysql.

That directory does not exist on my server?!

docker-compose.yml is unchanged as

22 volumes:
23 - std_notes_db:/var/lib/mysql

Reimplement Failing test

One test is failing on a fresh local filesystem setup and CI. It was marked as skipped in this PR #52 and should be reimplemented in a different fashion.

Error output from the test run:

.......................Not enough or too many segments
 
..............................FUnable to disable MFA; user has email recovery disabled.
 
.MFA has been disabled.
 
...Successfully disabled email backups.
 
..
 

 
Failures:
 

 
  1) User download_backup 
 
     Failure/Error: File.open("tmp/#{email}-restore.txt", 'w') { |file| file.write(body) }
 
     
 
     Errno::ENOENT:
 
       No such file or directory @ rb_sysopen - tmp/[email protected]
 
     # ./app/models/user.rb:37:in `initialize'
 
     # ./app/models/user.rb:37:in `open'
 
     # ./app/models/user.rb:37:in `download_backup'
 
     # ./spec/models/user_spec.rb:113:in `block (3 levels) in <top (required)>'
 

 
Finished in 9.01 seconds (files took 2.48 seconds to load)
 
60 examples, 1 failure

cc @johnny243

CloudLink extension doesn't work well

Hi,

I got problems with CloudLink extension and after several emails with your support team, they suggest opening an issue here. Since this is the first issue I submit in GitHub, please feel free to let me know if I miss anything.

Here is the whole thing:

I run a syncing-server on AWS LightSail and install CloudLink extension on macOs app according to https://docs.standardnotes.org/self-hosting/ec2-nginx . The backup feature doesn't work. After I contact the support team, I was told that I need to setup AWS SQS on my syncing-server. And they gave me some guides:

1. Set up your AWS credentials (see here: https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html#aws-ruby-sdk-setting-credentials)
2. Run bundle exec shoryuken sqs create dev_queue
3. Update SQS_QUEUE and AWS_REGION on your .env file
4. Start shoryuken with: bundle exec shoryuken -q hello -R

After running these commands (I only changed the queue name), I got it work but with some problems.

2.1 Backup can't run automatically. Only the first backup after installing it and backups which I click manually can work well. It can't backup automatically everyday. None of Dropbox, Google Drive, OneDrive can work automatically. With the syncing-server running, it's also running command bundle exec shoryuken -q hello -R in the background.

2.2 Too many queue messages. From March 26 to March 29, the queue created by shoryuken had more than 4 million messages to SQS. Please see the image, which was got on March 29. The final messages amount I checked yesterday is more than 4 million 700 thousand.

At the same time, I found it in the production.log that the server will trigger more queue messages as long as I type in the macOS app. Please see the image. Almost every letter I type will trigger this log. I think it means too many queue messages were created.

2.3 Email backup failed. Email backup doesn't work. I got error like this image. Before I submit this issue, I found from other issues that it seems like I miss SMTP settings configured but which I can't find in the document on StandardNotes website.

[ERROR] Error performing ActionMailer::DeliveryJob (Job ID: 7a6239eb-89a8-4835-9d20-5d9c8bd78cb9) from Shoryuken(cloud_backup_queue) in 79.13ms: Errno::ECONNREFUSED (Connection refused - connect(2) for nil port 25):
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/smtp.rb:539:in `initialize'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/smtp.rb:539:in `open'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/smtp.rb:539:in `tcp_socket'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/smtp.rb:549:in `block in do_start'
/home/ubuntu/.rbenv/versions/2.7.0/lib/ruby/2.7.0/timeout.rb:95:in `block in timeout'

I was also asked by your support team how I setup ActiveJob and if I switch to the built-in adapters: async or inline. I have no idea about these setup and I think I just keep the default setup. I changed nothing other than running the commands I got from your support team email.
The syncing-server code I'm running.

That's all I got.

Storing mysql root password is bad and not needed

I noticed DB_ROOT_PASSWORD used in .env file. This is always a bad idea, and bundle exec rails db:create db:migrate will prompt for password anyway.

docker-compose up error

I've updated to the latest version. When running docker-compose up -d I get

root@nuc:/opt/syncing-server# sudo docker-compose up -d
syncing-server_db_1 is up-to-date
Creating syncing-server_app_1 ... error

ERROR: for syncing-server_app_1 Cannot start service app: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: "start-local": executable file not found in $PATH": unknown

ERROR: for app Cannot start service app: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: "start-local": executable file not found in $PATH": unknown
ERROR: Encountered errors while bringing up the project.
root@nuc:/opt/syncing-server#

My .env file is

# Sample ENV setup Variables
# Copy this file and update as needed.
#
# $ cp env.sample .env
#
# Do not include this new file in source control
#

# Rails Settings
EXPOSED_PORT=3001
RAILS_ENV=development
RAILS_LOG_TO_STDOUT=false
#SQS_QUEUE=somequeue
#AWS_REGION=us-west1

# Database Settings
DB_PORT=3306
DB_HOST=db #=> If using docker-compose

DB_DATABASE=standard_notes_db
DB_USERNAME=std_notes_user
# Please change this!
DB_PASSWORD=mysecretpassword
# Please change this!
DB_ROOT_PASSWORD=mysecretpassword
DB_POOL_SIZE=30
DB_WAIT_TIMEOUT=180

# Secrets
# Use: bundle exec rake secret
# To generate required secrets below
#

SECRET_KEY_BASE=changeme123

# Disable user registration
#DISABLE_USER_REGISTRATION=true

# Datadog
DATADOG_ENABLED=false

Can you help me out? The server was working fine before the update

Unable to sync notes

docker-compose.yml:

version: "3"

networks:
  internal:
  caddy_external:
    external: true

services:    

  server:
    image: standardnotes/syncing-server
    networks:
      - internal
      - caddy_external
    expose:
      - 3000
    restart: always
    volumes:
      - /etc/localtime:/etc/localtime:ro
    depends_on:
      - db
    env_file:
      - .env
    #user: 1000:1000
    labels:
      - caddy=test.notes.$MY_DOMAIN
      - caddy.route=/api/*
      - caddy.route.0_uri=strip_prefix /api
      - caddy.route.1_reverse_proxy={{upstreams 3000}}

  db:
    image: linuxserver/mariadb
    networks:
      - internal
    restart: always
    volumes:
      - ./db:/config
    environment:
      - MYSQL_DATABASE=$DB_DATABASE
      - MYSQL_USER=$DB_USERNAME
      - MYSQL_PASSWORD=$DB_PASSWORD
      - MYSQL_ROOT_PASSWORD=$DB_ROOT_PASSWORD
      - PUID=1000
      - PGID=1000
      - TZ=CET

.env file:

MY_DOMAIN=example.com

# Rails Settings

EXPOSED_PORT=3000
RAILS_ENV=production
RAILS_LOG_TO_STDOUT=false

# Database Settings

DB_HOST=db
DB_PORT=3306

DB_DATABASE=standardnotes
DB_USERNAME=standardnotes
DB_PASSWORD=[redacted]
DB_ROOT_PASSWORD=[redacted]

#DB_POOL_SIZE=30
#DB_WAIT_TIMEOUT=180

# Standard Notes Settings

SECRET_KEY_BASE=[redacted]

DISABLE_USER_REGISTRATION=false
DATADOG_ENABLED=false

I can sign up on app.standardnotes.org using my sync server test.notes.example.com/api.
But any notes I create don't sync properly. It is stuck on Syncing:0/1, then creates duplicates of the note I created.

Logs for the syncing-server container:

server_1  | Starting Server...
server_1  | => Booting Puma
server_1  | => Rails 5.1.7 application starting in production
server_1  | => Run `rails server -h` for more startup options
server_1  | Puma starting in single mode...
server_1  | * Version 4.3.5 (ruby 2.6.5-p114), codename: Mysterious Traveller
server_1  | * Min threads: 5, max threads: 5
server_1  | * Environment: production
server_1  | * Listening on tcp://0.0.0.0:3000
server_1  | Use Ctrl-C to stop
server_1  | {"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":60.52,"view":0.99,"db":6.9,"ddsource":["ruby"],"time":"2020-08-31 19:09:49 +0000","params":{"limit":150,"items":[{"uuid":"ba50f164-2d49-49fd-923d-0ca820c8b8fe","content_type":"[FILTERED]","created_at":"2020-08-31T19:08:57.885Z","updated_at":"1970-01-01T00:00:00.000Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"}],"sync_token":"MjoxNTk4OTAwOTQ4LjE0MjM3MDI=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
server_1  | {"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":3.74,"view":0.18,"db":1.0,"ddsource":["ruby"],"time":"2020-08-31 19:09:50 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4OTAwOTg5Ljg3Nzg0Ng==\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
server_1  | {"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":3.71,"view":0.2,"db":0.98,"ddsource":["ruby"],"time":"2020-08-31 19:09:50 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4OTAwOTkwLjAxODQ2Mjc=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
server_1  | {"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":3.7,"view":0.21,"db":0.97,"ddsource":["ruby"],"time":"2020-08-31 19:09:53 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4OTAwOTkwLjM4ODg2NA==\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
server_1  | {"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":3.67,"view":0.19,"db":0.97,"ddsource":["ruby"],"time":"2020-08-31 19:09:55 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4OTAwOTkzLjk0OTM5Mjg=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}

client login failed

hi guys,
for me some clients do not work with the self hosted server.

running on ARMv8 architecture - RaspberryPi 3
working: win10 client, ios app
not working: web app, android app

Is it due to missing ssl encryption?

Docker-Compose App Unreachable

I'm running the Standard Notes docker-compose build on DigitalOcean Ubuntu 18.04, I've allowed the exposed port through the firewall but cannot reach the site that says to go to the app or website.

On the server, if I run wget http://localhost:Exposed_Port it says failed: Connection refused.

When I ran docker-compose ps, it alternates between:

       Name                      Command                 State             Ports
---------------------------------------------------------------------------------------
syncingserver_app_1   bash -c bundle exec rails  ...   Restarting
syncingserver_db_1    docker-entrypoint.sh mysqld      Up

And:

       Name                      Command               State           Ports
-------------------------------------------------------------------------------------
syncingserver_app_1   bash -c bundle exec rails  ...   Up      0.0.0.0:XXXXX->3000/tcp
syncingserver_db_1    docker-entrypoint.sh mysqld      Up

The db never shows restarting, but the app continuously does.

Any ideas?

DB Errors when Running docker-compose

Hi! When running docker-compose on my Synology, I get this error, would you know why this is occuring pls? Searched the net but not really sure :/

Mysql2::Error: Lost connection to MySQL server at 'handshake: reading inital communication packet', system error: 107
/usr/local/bundle/gems/mysql2-0.4.10/lib/mysql2/client.rb:89:in `connect'
/usr/local/bundle/gems/mysql2-0.4.10/lib/mysql2/client.rb:89:in `initialize'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/connection_adapters/mysql2_adapter.rb:21:in `new'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/connection_adapters/mysql2_adapter.rb:21:in `mysql2_connection'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:761:in `new_connection'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:805:in `checkout_new_connection'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:784:in `try_to_checkout_new_connection'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:745:in `acquire_connection'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:502:in `checkout'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:376:in `connection'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:933:in `retrieve_connection'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/connection_handling.rb:116:in `retrieve_connection'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/connection_handling.rb:88:in `connection'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/schema_migration.rb:20:in `table_exists?'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/schema_migration.rb:24:in `create_table'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/migration.rb:1125:in `initialize'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/migration.rb:1007:in `new'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/migration.rb:1007:in `up'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/migration.rb:985:in `migrate'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/tasks/database_tasks.rb:171:in `migrate'
/usr/local/bundle/gems/activerecord-5.1.7/lib/active_record/railties/databases.rake:58:in `block (2 levels) in <top (required)>'
/usr/local/bundle/gems/railties-5.1.7/lib/rails/commands/rake/rake_command.rb:21:in `block in perform'
/usr/local/bundle/gems/railties-5.1.7/lib/rails/commands/rake/rake_command.rb:18:in `perform'
/usr/local/bundle/gems/railties-5.1.7/lib/rails/command.rb:46:in `invoke'
/usr/local/bundle/gems/railties-5.1.7/lib/rails/commands.rb:16:in `<top (required)>'
bin/rails:4:in `require'
bin/rails:4:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)

error on nokogiri install

Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

current directory: /var/lib/gems/2.5.0/gems/nokogiri-1.10.9/ext/nokogiri

/usr/bin/ruby2.5 -r ./siteconf20200612-13184-15dwlpy.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h

extconf failed, exit code 1

Gem files will remain installed in /var/lib/gems/2.5.0/gems/nokogiri-1.10.9 for inspection.
Results logged to /var/lib/gems/2.5.0/extensions/x86_64-linux/2.5.0/nokogiri-1.10.9/gem_make.out

An error occurred while installing nokogiri (1.10.9), and Bundler cannot continue.
Make sure that gem install nokogiri -v '1.10.9' --source 'https://rubygems.org/' succeeds before bundling.

In Gemfile:
rails was resolved to 5.1.7, which depends on
actioncable was resolved to 5.1.7, which depends on
actionpack was resolved to 5.1.7, which depends on
actionview was resolved to 5.1.7, which depends on
rails-dom-testing was resolved to 2.0.3, which depends on
nokogiri

getting an error, when trying to install just the base nokogiri gem (no version) it also fails

JSON gem deprecation warnings

We should get rid of those warnings:

/usr/local/bundle/gems/json-1.8.6/lib/json/common.rb:155: warning: Using the last argument as keyword parameters is deprecated,
/usr/local/bundle/gems/activesupport-5.1.7/lib/active_support/notifications/fanout.rb:19: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead,
/usr/local/bundle/gems/actionpack-5.1.7/lib/action_dispatch/middleware/stack.rb:35: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call,
/usr/local/bundle/gems/actionpack-5.1.7/lib/action_dispatch/middleware/static.rb:109: warning: The called method `initialize' is defined here

Best way to upgrade selfhost from ruby-server?

I am still using the "ruby-server" implementation of the Standard Notes server. Should I upgrade to the new "syncing-server" and if so, what is the best way to do this without losing notes.

Supress tcp connections errors on unreachable hosts

Sentry Issue: RUBY-SERVER-5Z

SocketError: getaddrinfo: Name does not resolve
  app/jobs/extension_job.rb:33:in `perform'
    response = http.request(req)
...
(73 additional frame(s) were not displayed)

SocketError: Failed to open TCP connection to preview.extensions.standardnotes.org:443 (getaddrinfo: Name does not resolve)
  app/jobs/extension_job.rb:33:in `perform'
    response = http.request(req)
...
(72 additional frame(s) were not displayed)

Unable to deploy to heroku

Link: https://dashboard.heroku.com/new?button-url=https%3A%2F%2Fgithub.com%2Fstandardnotes%2Fsyncing-server&template=https%3A%2F%2Fgithub.com%2Fstandardnotes%2Fsyncing-server

-----> Ruby app detected

-----> Installing bundler 2.0.2

-----> Removing BUNDLED WITH version in the Gemfile.lock

-----> Compiling Ruby/Rails

-----> Using Ruby version: ruby-2.6.6

-----> Installing dependencies using bundler 2.0.2
       Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
       You are trying to install in deployment mode after changing
       your Gemfile. Run `bundle install` elsewhere and add the
       updated Gemfile.lock to version control.
       
       You have deleted from the Gemfile:

       * device_detector
       Bundler Output: You are trying to install in deployment mode after changing
       your Gemfile. Run `bundle install` elsewhere and add the
       updated Gemfile.lock to version control.
       
       You have deleted from the Gemfile:
       * device_detector

 !
 !     Failed to install gems via Bundler.
 !
 !     Push rejected, failed to compile Ruby app.
 !     Push failed

unable to connect to self hosted backend

My syncing server repo is on the master branch but I'm still getting an error about version mismatch. Not sure what else I can do to get it to work.

Missing ids error on extension job

Durign the weekend we had quite a few multiline errors with missing ids passed to extension job:

Implement using database read replicas in Rails

In Rails 6 db read replicas come as a built-in feature. Since we are at 5.1 we need to add the octopus gem to utilize delegating read queries to read replicas.

Sources:

Can't deploy to heroku because of sqlite3 gem

Can't deploy to heroku. I get the following error:

       An error occurred while installing sqlite3 (1.4.1), and Bundler cannot continue.
       Make sure that `gem install sqlite3 -v '1.4.1' --source 'https://rubygems.org/'`
       succeeds before bundling.
       
       In Gemfile:
         mailcatcher was resolved to 0.2.4, which depends on
           sqlite3-ruby was resolved to 1.3.3, which depends on
             sqlite3
 !
 !     Failed to install gems via Bundler.
 !     Detected sqlite3 gem which is not supported on Heroku:
 !     https://devcenter.heroku.com/articles/sqlite3
 !
 !     Push rejected, failed to compile Ruby app.
 !     Push failed

Problem adding selfhosted extension windows app

Hello standardnotes-team,
I struggle at the moment with installing my selfhosted-extensions in the desktop app (Win).
In the web app (app.standardnotes.org) i logged in with sync-URL (Advanced Options) and User, add the extensions URL and after that, every extensions can be installed flawlessly.
On the desktop app i got this error messages: "Error installing locally: error downloading https://github.com/sn-extensions".
Activating the developer tools in the window app i got these message installing for example the Folder Component Extensions.

i just started using standardnotes as a possibility to organize my knowledgebase and to check, if i can do all my stuff.

Has somebody an idea?

Session tokens develop branch

I wanted to try to understand how authentication and sessions worked here. The Sessions.md file is very much appreciated, since I do not know ruby well.
What I figure is that the access token and the refresh token are stored as they are.

Therefore if an attacker were to gain access to the db silently or backups(depending on how their stored) then they would be able to get hold of anybody's session tokens and use them.
Also that searching for the token in the db can perhaps be open to comparison attacks.

I am reading this book: Password authentication for web and mobile apps by Dmitry Chestnykh, https://dchest.com/authbook/

And it recommends to uniquely generate two parts for a single token, an Identifier and a Verifier. You would then concatenate both of those values and send it to the client. You would store and index the identifier, and hash(SHA-256) of the verifier and store that separately. To authenticate the user, search for the identifier and compare the hash of the verifier in the request to the hash of the verifier in the database using constant time comparison.

So I guess you would do this for both the access and refresh tokens. Is this realistic? Or am I missing something?

Can't register using domain name, working locally

Hello,

I have problems setting up Standard Notes Syncing-Server. I am running Debian 10. Using a self-hosted web client, I am able to connect to my Syncing-Server using http://127.0.0.1:3000 (3000 being the port where is running the syncing-server when issuing rails s -b=0.0.0.0 -p 3000), but I can neither access It with http://domain.com nor https://domain.com. Trying to connect on https://app.standardnotes.org/ using https://domain.com also fails.

I think It must be a Passenger issue, but I don't know how to solve It. Indeed, when accessing http://127.0.0.1:3000 (3000 being the port where is running the syncing server when issuing rails s -b=0.0.0.0 -p 3000), I can see the page "You're not supposed to be here" with a normal presentation (Image 1). However, accessing the syncing server with http://domain.com or https://domain.com results in a page with "You're not supposed to be here" without any page layout (Image 2).

I have installed Ruby, RVM and Passenger using the official tutorial on Passenger website : https://www.phusionpassenger.com/library/walkthroughs/deploy/ruby/ownserver/nginx/oss/stretch/install_passenger.html.

Here are the logs when I try to access the syncing-server using http://domain.com or https://domain.com. The line 6 to 10 seems to be the main problem. :

^[[0;37m2020-02-09 16:38:19.906^[[0m [^[[32mINFO ^[[0m] Started GET "/" for **FILTERED** at 2020-02-09 16:38:19 +0100 (pid:776)
^[[0;37m2020-02-09 16:38:19.907^[[0m [^[[32mINFO ^[[0m] Processing by ApplicationController#home as HTML (pid:776)
^[[0;37m2020-02-09 16:38:19.908^[[0m [^[[32mINFO ^[[0m] Rendering application/home.html (pid:776)
^[[0;37m2020-02-09 16:38:19.908^[[0m [^[[32mINFO ^[[0m] Rendered application/home.html (0.1ms) (pid:776)
^[[0;37m2020-02-09 16:38:19.909^[[0m [^[[32mINFO ^[[0m] Completed 200 OK in 2ms (Views: 0.8ms) (pid:776)
^[[0;37m2020-02-09 16:38:19.964^[[0m [^[[32mINFO ^[[0m] Started GET "/assets/home.css" for **FILTERED** at 2020-02-09 16:38:19 +0100 (pid:776)
^[[0;37m2020-02-09 16:38:19.964^[[0m [^[[31mFATAL^[[0m]  (pid:776)
^[[0;37m2020-02-09 16:38:19.964^[[0m [^[[31mFATAL^[[0m] ActionController::RoutingError (No route matches [GET] "/assets/home.css"): (pid:776)
^[[0;37m2020-02-09 16:38:19.964^[[0m [^[[31mFATAL^[[0m]  (pid:776)
^[[0;37m2020-02-09 16:38:19.964^[[0m [^[[31mFATAL^[[0m] actionpack (5.1.7) lib/action_dispatch/middleware/debug_exceptions.rb:63:in `call'
actionpack (5.1.7) lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
railties (5.1.7) lib/rails/rack/logger.rb:36:in `call_app'
railties (5.1.7) lib/rails/rack/logger.rb:26:in `call'
actionpack (5.1.7) lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
actionpack (5.1.7) lib/action_dispatch/middleware/request_id.rb:25:in `call'
rack (2.0.7) lib/rack/method_override.rb:22:in `call'
rack (2.0.7) lib/rack/runtime.rb:22:in `call'
activesupport (5.1.7) lib/active_support/cache/strategy/local_cache_middleware.rb:27:in `call'
actionpack (5.1.7) lib/action_dispatch/middleware/executor.rb:12:in `call'
rack (2.0.7) lib/rack/sendfile.rb:111:in `call'
rack (2.0.7) lib/rack/deflater.rb:34:in `call'
rack-cors (1.0.3) lib/rack/cors.rb:95:in `call'
secure_headers (6.1.1) lib/secure_headers/middleware.rb:11:in `call'
sentry-raven (2.12.0) lib/raven/integrations/rack.rb:51:in `call'
railties (5.1.7) lib/rails/engine.rb:522:in `call'
/usr/lib/ruby/vendor_ruby/phusion_passenger/rack/thread_handler_extension.rb:97:in `process_request'
/usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:149:in `accept_and_process_next_request'
/usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:110:in `main_loop'
/usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler.rb:415:in `block (3 levels) in start_threads'
/usr/lib/ruby/vendor_ruby/phusion_passenger/utils.rb:113:in `block in create_thread_and_abort_on_exception' (pid:776)

My Standard notes config file for nginx is located in /etc/nginx/conf.d/standardnotes.conf all the other conf files for my other services and is the same as your tutorial :

server {
    listen 443 ssl default_server;
    ssl_certificate /etc/letsencrypt/live/domain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/domain.com/privkey.pem;
    server_name domain.com;
    passenger_enabled on;
    passenger_app_env production;
    root /home/ec2-user/syncing-server/public;
  }

Please help me, I love this app and want to use It every day ! Thank you !

How to run sync server with nginx under other location

Hi,

I tried to run the sync server with nginx and want to use other location

Snippet from my site config:

location /notes/ {
                proxy_pass http://127.0.0.1:3000;
                root /MY/FOLDER/syncing-server/public;
        }

But I got every time an error that the POST path "/notes/auth" was not found.
So can I change somewhere in the sync server the default path or should I use a sub domain for the sync server?

docker-compose MySQL error

I'm trying to run the server using docker-compose

My .env file is:

1 # Sample ENV setup Variables
2 # Copy this file and update as needed.
3 #
4 # $ cp env.sample .env
5 #
6 # Do not include this new file in source control
7 #
8
9 # Rails Settings
10 EXPOSED_PORT=3000
11 SECRET_KEY_BASE=changeme123
12 RAILS_ENV=development
13 #SQS_QUEUE=somequeue
14 #AWS_REGION=us-west1
15
16 # Database Settings
17 DB_PORT=3306
18 DB_HOST=127.0.0.1
19 #DB_HOST=db #=> If using docker-compose
20
21 DB_DATABASE=standard_notes_db
22 DB_USERNAME=std_notes_user
23 DB_PASSWORD=changeme123
24 DB_ROOT_PASSWORD=changeme123

The container keeps restarting and the logfile says:

app_1 | Mysql2::Error: Can't connect to MySQL server on '127.0.0.1' (107)

I'm especially unsure about lines 18/19 in the .env file. Can you please elaborate further?
Thanks a lot!

Docker Hub image

I originally posted this on r/StandardNotes where u/sn-jaspal requested I create this enhancement request.

Your Dockerfile is a ready to build. Is there a reason you are not utilizing Docker Hub to auto-compile, build, host, and easily distribute your image? I believe this is ultimately the same request dslovin is making in issue #14

Here are the Docker Hub provided instructions. I have forked/cloned this repository and have it auto-compiling without issue using Docker Hub at danofun/syncing-server. The only change I see to your code would be changing docker-compose.yml line #4 to reflect the new Docker Hub image.
image: standardnotes/syncing-server

Installing on Pi Not Working

Hello! I have been playing around with setting up Standard Notes and, maybe, paying for it for the extensions if it worked out. As the moile/desktop apps look great!

I have been trying to setup this via Docker Compose on my Pi but its not working. From the looks of things, it appears something in there isn't designed for ARM processors. Any guidance on how to resolve this issue, if possible pls?

[Feature Suggestion] Cold Storage

Hey @mobitar,

I had a feature suggestion (sadly no pull request though) for the syncing server; would a cold storage option be in the cards in the near future? I have around 750 notes, and it takes a little while in order to sync everything. Most of the notes I archive successfully, but the archive remains synced, and I don't fetch notes from the archive often. It'd be really cool if after 30 days in archive without user access, notes could go into "cold storage" where they are not synced by default, and are only pulled when manually requested and refreshed.

Unfortunately I'm not too familiar with Ruby...and I think this would be a full-stack issue, and since I don't have a pull request, I'm just putting this out there as a possible performance improvement.

Thanks so much! I never thought I'd get to 750 notes with Standard Notes, but I'm here and I'm so happy I made the purchase :)
Ying

docker-compose fails to run

HEAD: d8b2416

Building app
Step 1/10 : FROM ruby:2.6.5-alpine
 ---> 59d4ec426ed6
Step 2/10 : ARG optimize_for_raspberry_pi
 ---> Using cache
 ---> dca633596c4b
Step 3/10 : RUN apk add --update --no-cache     alpine-sdk     mariadb-dev     tzdata
 ---> Using cache
 ---> 903efbdb3b1c
Step 4/10 : WORKDIR /syncing-server
 ---> Using cache
 ---> de572fbe20d5
Step 5/10 : COPY Gemfile Gemfile.lock /syncing-server/
 ---> Using cache
 ---> 2bfa7f97aea7
Step 6/10 : RUN if [ "$optimize_for_raspberry_pi" = true ] ; then sed -i 's/bcrypt (3.1.13)/bcrypt (3.1.12)/g' Gemfile.lock; fi
 ---> Using cache
 ---> 22fdbbb9dac7
Step 7/10 : RUN gem install bundler && bundle install
 ---> Running in 58530ac0c3b0
Successfully installed bundler-2.1.4
1 gem installed
Fetching https://github.com/mobitar/capistrano-shoryuken.git
Fetching gem metadata from https://rubygems.org/.........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies...
Fetching rake 13.0.1
Installing rake 13.0.1
Fetching concurrent-ruby 1.1.6
Installing concurrent-ruby 1.1.6
Fetching i18n 1.8.2
Installing i18n 1.8.2
Fetching minitest 5.14.0
Installing minitest 5.14.0
Fetching thread_safe 0.3.6
Installing thread_safe 0.3.6
Fetching tzinfo 1.2.6
Installing tzinfo 1.2.6
Fetching activesupport 5.1.7
Installing activesupport 5.1.7
Fetching builder 3.2.4
Installing builder 3.2.4
Fetching erubi 1.9.0
Installing erubi 1.9.0
Fetching mini_portile2 2.4.0
Installing mini_portile2 2.4.0
Fetching nokogiri 1.10.9
Installing nokogiri 1.10.9 with native extensions
Fetching rails-dom-testing 2.0.3
Installing rails-dom-testing 2.0.3
Fetching crass 1.0.6
Installing crass 1.0.6
Fetching loofah 2.4.0
Installing loofah 2.4.0
Fetching rails-html-sanitizer 1.3.0
Installing rails-html-sanitizer 1.3.0
Fetching actionview 5.1.7
Installing actionview 5.1.7
Fetching rack 2.2.3
Installing rack 2.2.3
Fetching rack-test 1.1.0
Installing rack-test 1.1.0
Fetching actionpack 5.1.7
Installing actionpack 5.1.7
Fetching nio4r 2.5.2
Installing nio4r 2.5.2 with native extensions
Fetching websocket-extensions 0.1.4
Installing websocket-extensions 0.1.4
Fetching websocket-driver 0.6.5
Installing websocket-driver 0.6.5 with native extensions
Fetching actioncable 5.1.7
Installing actioncable 5.1.7
Fetching globalid 0.4.2
Installing globalid 0.4.2
Fetching activejob 5.1.7
Installing activejob 5.1.7
Fetching mini_mime 1.0.2
Installing mini_mime 1.0.2
Fetching mail 2.7.1
Installing mail 2.7.1
Fetching actionmailer 5.1.7
Installing actionmailer 5.1.7
Fetching activemodel 5.1.7
Installing activemodel 5.1.7
Fetching arel 8.0.0
Installing arel 8.0.0
Fetching activerecord 5.1.7
Installing activerecord 5.1.7
Fetching public_suffix 4.0.3
Installing public_suffix 4.0.3
Fetching addressable 2.7.0
Installing addressable 2.7.0
Fetching net-ssh 5.2.0
Installing net-ssh 5.2.0
Fetching net-scp 2.0.0
Installing net-scp 2.0.0
Fetching sshkit 1.20.0
Installing sshkit 1.20.0
Fetching airbrussh 1.4.0
Installing airbrussh 1.4.0
Fetching ast 2.4.0
Installing ast 2.4.0
Fetching aws-eventstream 1.0.3
Installing aws-eventstream 1.0.3
Fetching aws-partitions 1.278.0
Installing aws-partitions 1.278.0
Fetching aws-sigv4 1.1.1
Installing aws-sigv4 1.1.1
Fetching jmespath 1.4.0
Installing jmespath 1.4.0
Fetching aws-sdk-core 3.90.1
Installing aws-sdk-core 3.90.1
Fetching aws-sdk-sqs 1.23.1
Installing aws-sdk-sqs 1.23.1
Fetching bcrypt 3.1.13
Installing bcrypt 3.1.13 with native extensions
Using bundler 2.1.4
Fetching byebug 11.1.1
Installing byebug 11.1.1 with native extensions
Fetching capistrano 3.12.0
Installing capistrano 3.12.0
Fetching capistrano-bundler 1.6.0
Installing capistrano-bundler 1.6.0
Fetching capistrano-passenger 0.2.0
Installing capistrano-passenger 0.2.0
Fetching capistrano-rails 1.4.0
Installing capistrano-rails 1.4.0
Fetching capistrano-rvm 0.1.2
Installing capistrano-rvm 0.1.2
Using capistrano-shoryuken 0.1.6 from https://github.com/mobitar/capistrano-shoryuken.git (at master@e935bad)
Fetching chronic 0.10.2
Installing chronic 0.10.2
Fetching msgpack 1.3.3
Installing msgpack 1.3.3 with native extensions
Fetching ddtrace 0.38.0
Installing ddtrace 0.38.0
Fetching diff-lcs 1.3
Installing diff-lcs 1.3
Fetching docile 1.3.2
Installing docile 1.3.2
Fetching dogstatsd-ruby 4.8.1
Installing dogstatsd-ruby 4.8.1
Fetching dotenv 2.7.5
Installing dotenv 2.7.5
Fetching method_source 0.9.2
Installing method_source 0.9.2
Fetching thor 1.0.1
Installing thor 1.0.1
Fetching railties 5.1.7
Installing railties 5.1.7
Fetching dotenv-rails 2.7.5
Installing dotenv-rails 2.7.5
Fetching erubis 2.7.0
Installing erubis 2.7.0
Fetching factory_bot 5.1.1
Installing factory_bot 5.1.1
Fetching factory_bot_rails 5.1.1
Installing factory_bot_rails 5.1.1
Fetching ffi 1.12.2
Installing ffi 1.12.2 with native extensions
Fetching temple 0.8.2
Installing temple 0.8.2
Fetching tilt 2.0.10
Installing tilt 2.0.10
Fetching haml 5.1.2
Installing haml 5.1.2
Fetching sexp_processor 4.14.1
Installing sexp_processor 4.14.1
Fetching ruby_parser 3.14.2
Installing ruby_parser 3.14.2
Fetching html2haml 2.2.0
Installing html2haml 2.2.0
Fetching haml-rails 2.0.1
Installing haml-rails 2.0.1
Fetching jaro_winkler 1.5.4
Installing jaro_winkler 1.5.4 with native extensions
Fetching jwt 2.2.1
Installing jwt 2.2.1
Fetching rb-fsevent 0.10.3
Installing rb-fsevent 0.10.3
Fetching rb-inotify 0.10.1
Installing rb-inotify 0.10.1
Fetching listen 3.2.1
Installing listen 3.2.1
Fetching request_store 1.5.0
Installing request_store 1.5.0
Fetching lograge 0.11.2
Installing lograge 0.11.2
Fetching mysql 2.9.1
Installing mysql 2.9.1 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /usr/local/bundle/gems/mysql-2.9.1/ext/mysql_api
/usr/local/bin/ruby -I /usr/local/lib/ruby/2.6.0 -r
./siteconf20200818-1-11pmds9.rb extconf.rb
checking for mysql_ssl_set()... yes
checking for rb_str_set_len()... yes
checking for rb_thread_start_timer()... no
checking for mysql.h... yes
creating Makefile

current directory: /usr/local/bundle/gems/mysql-2.9.1/ext/mysql_api
make "DESTDIR=" clean

current directory: /usr/local/bundle/gems/mysql-2.9.1/ext/mysql_api
make "DESTDIR="
compiling mysql.c
mysql.c: In function 'real_connect':
mysql.c:276:17: error: 'MYSQL' {aka 'struct st_mysql'} has no member named
'reconnect'
  276 |     myp->handler.reconnect = 0;
      |                 ^
mysql.c: In function 'real_connect2':
mysql.c:350:6: error: 'MYSQL' {aka 'struct st_mysql'} has no member named
'reconnect'
  350 |     m->reconnect = 0;
      |      ^~
mysql.c: In function 'reconnect':
mysql.c:920:27: error: 'MYSQL' {aka 'struct st_mysql'} has no member named
'reconnect'
  920 |     return GetHandler(obj)->reconnect ? Qtrue : Qfalse;
      |                           ^~
mysql.c: In function 'reconnect_set':
mysql.c:926:20: error: 'MYSQL' {aka 'struct st_mysql'} has no member named
'reconnect'
926 |     GetHandler(obj)->reconnect = (flag == Qnil || flag == Qfalse) ? 0 :
1;
      |                    ^~
mysql.c: In function 'stmt_bind_result':
mysql.c:1320:74: error: 'rb_cFixnum' undeclared (first use in this function)
1320 |  else if (argv[i] == rb_cNumeric || argv[i] == rb_cInteger || argv[i] ==
rb_cFixnum)
|                                                                         
^~~~~~~~~~
mysql.c:1320:74: note: each undeclared identifier is reported only once for each
function it appears in
mysql.c: In function 'reconnect':
mysql.c:921:1: warning: control reaches end of non-void function [-Wreturn-type]
  921 | }
      | ^
mysql.c: At top level:
cc1: warning: unrecognized command line option '-Wno-self-assign'
cc1: warning: unrecognized command line option '-Wno-parentheses-equality'
cc1: warning: unrecognized command line option '-Wno-constant-logical-operand'
make: *** [Makefile:245: mysql.o] Error 1

make failed, exit code 2

Gem files will remain installed in /usr/local/bundle/gems/mysql-2.9.1 for
inspection.
Results logged to
/usr/local/bundle/extensions/x86_64-linux/2.6.0/mysql-2.9.1/gem_make.out

An error occurred while installing mysql (2.9.1), and Bundler cannot continue.
Make sure that `gem install mysql -v '2.9.1' --source 'https://rubygems.org/'`
succeeds before bundling.

In Gemfile:
  mysql
ERROR: Service 'app' failed to build: The command '/bin/sh -c gem install bundler && bundle install' returned a non-zero code: 5

Alternative command for secret when using docker-compose

Maybe add another line here:

syncing-server/env.sample

Line 28 in 9b458f9

# Use: bundle exec rake secret

With something like this:

# or by using: openssl rand -hex 64

For users who have never used RoR (like me) and to prevent to "pollute" the prod server by installing RoR only for generate secrets.

Administration-Remove User

Hello,
I am not too sure if this is the right place, so please feel free to point me somewhere else.
I have installed the sync-server following your documentation, but could not find anywhere if any administration features where available.
As an example, how would one remove an existing user? I found traces of some admin controller in the repo code, and wanted to know if it was possible to get a bit more information on how to actually use it. Is there an API call that can be made?
In the negative, what would be the operation to perform to remove a user?

Many thanks and have a good rest of the day,

Best,

Sync duplicating notes

Hi,
Everything works as expected. However when I create a note it starts duplicating randomly. i.e, 1 note becomes 2,4,6,8 and so on and doesn't stop. Here are the logs

{"method":"POST","path":"/auth/sign_in","format":"*/*","controller":"Api::AuthController","action":"sign_in","status":200,"duration":148.02,"view":0.4,"db":1.98,"ddsource":["ruby"],"time":"2020-08-31 12:51:56 +0000","params":{"password":"[FILTERED]","email":"[email protected]","api":"20190520","controller":"api/auth","action":"sign_in","auth":{"password":"[FILTERED]","email":"[email protected]","api":"20190520"}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":67.2,"view":33.2,"db":10.75,"ddsource":["ruby"],"time":"2020-08-31 12:51:57 +0000","params":{"limit":150,"compute_integrity":true,"items":[{"uuid":"b7031050-278c-4e44-9c44-346e962c8b92","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:29.714Z","updated_at":"2020-08-31T12:51:37.946Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"},{"uuid":"c7d0c2bd-8395-4a52-bff4-13ce7dc2c2da","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:29.716Z","updated_at":"2020-08-31T12:51:37.946Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"},{"uuid":"1d1aa8fd-159b-40e6-931a-1e25a2830a5d","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:29.718Z","updated_at":"2020-08-31T12:51:37.946Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"},{"uuid":"0589bae7-42dd-4463-8ae6-8c5846ca4c87","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:29.719Z","updated_at":"2020-08-31T12:51:37.946Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"}],"sync_token":null,"cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":28.22,"view":1.14,"db":10.29,"ddsource":["ruby"],"time":"2020-08-31 12:51:59 +0000","params":{"limit":150,"items":[{"uuid":"b7031050-278c-4e44-9c44-346e962c8b92","content_type":"[FILTERED]","deleted":true,"created_at":"2020-08-31T12:51:29.714Z","updated_at":"2020-08-31T12:51:57.508Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"},{"uuid":"c7d0c2bd-8395-4a52-bff4-13ce7dc2c2da","content_type":"[FILTERED]","deleted":true,"created_at":"2020-08-31T12:51:29.716Z","updated_at":"2020-08-31T12:51:57.515Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"},{"uuid":"1d1aa8fd-159b-40e6-931a-1e25a2830a5d","content_type":"[FILTERED]","deleted":true,"created_at":"2020-08-31T12:51:29.718Z","updated_at":"2020-08-31T12:51:57.519Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"},{"uuid":"0589bae7-42dd-4463-8ae6-8c5846ca4c87","content_type":"[FILTERED]","deleted":true,"created_at":"2020-08-31T12:51:29.719Z","updated_at":"2020-08-31T12:51:57.523Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"}],"sync_token":"MjoxNTk4ODc4MzE3LjUyMzY1MQ==\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":3.29,"view":0.18,"db":0.78,"ddsource":["ruby"],"time":"2020-08-31 12:51:59 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4ODc4MzE5LjE4MDA5Ng==\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":3.35,"view":0.19,"db":0.82,"ddsource":["ruby"],"time":"2020-08-31 12:51:59 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4ODc4MzE5LjI5NzM4NQ==\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"html","controller":"Api::ItemsController","action":"sync","status":200,"duration":9.37,"view":1.04,"db":1.71,"ddsource":["ruby"],"time":"2020-08-31 12:52:01 +0000","params":{"limit":150,"compute_integrity":true,"items":[],"sync_token":"MjoxNTk4ODc4Mjk1LjMxNDQzMg==\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"html","controller":"Api::ItemsController","action":"sync","status":200,"duration":8.54,"view":0.22,"db":1.74,"ddsource":["ruby"],"time":"2020-08-31 12:52:03 +0000","params":{"limit":150,"compute_integrity":true,"items":[],"sync_token":"MjoxNTk4ODc4MzIxLjkxMjAwMzU=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"html","controller":"Api::ItemsController","action":"sync","status":200,"duration":3.26,"view":0.19,"db":0.78,"ddsource":["ruby"],"time":"2020-08-31 12:52:05 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4ODc4MzIzLjMwMTc2NTc=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"html","controller":"Api::ItemsController","action":"sync","status":200,"duration":3.9,"view":0.51,"db":0.81,"ddsource":["ruby"],"time":"2020-08-31 12:52:15 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4ODc4MzI1LjMwOTU5Nw==\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":9018.59,"view":0.34,"db":3.22,"ddsource":["ruby"],"time":"2020-08-31 12:52:07 +0000","params":{"limit":150,"items":[{"uuid":"3167b72f-582f-4710-860b-e985d6a1a5b4","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:58.978Z","updated_at":"1970-01-01T00:00:00.000Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"}],"sync_token":"MjoxNTk4ODc4MzE5Ljc0Njg5NTM=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"html","controller":"Api::ItemsController","action":"sync","status":200,"duration":4.75,"view":0.59,"db":1.07,"ddsource":["ruby"],"time":"2020-08-31 12:52:21 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4ODc4MzM1LjU3NTk5Mw==\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":9034.05,"view":0.8,"db":4.26,"ddsource":["ruby"],"time":"2020-08-31 12:52:16 +0000","params":{"limit":150,"items":[{"uuid":"1042ff76-f2d2-46a1-836a-a730b50806e6","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:58.978Z","updated_at":"1970-01-01T00:00:00.000Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"}],"sync_token":"MjoxNTk4ODc4MzI3LjE0MjU5ODI=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"html","controller":"Api::ItemsController","action":"sync","status":200,"duration":3.78,"view":0.21,"db":0.88,"ddsource":["ruby"],"time":"2020-08-31 12:52:25 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4ODc4MzQxLjk1NjU5Mg==\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":9034.07,"view":0.72,"db":3.64,"ddsource":["ruby"],"time":"2020-08-31 12:52:25 +0000","params":{"limit":150,"items":[{"uuid":"e0259f6d-ce04-4ef8-96e6-6d8e246405cb","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:58.978Z","updated_at":"1970-01-01T00:00:00.000Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"}],"sync_token":"MjoxNTk4ODc4MzM2LjM1NTg3NzY=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"html","controller":"Api::ItemsController","action":"sync","status":200,"duration":3.76,"view":0.55,"db":0.83,"ddsource":["ruby"],"time":"2020-08-31 12:52:35 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4ODc4MzQ1LjUwMzMyMjQ=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":9031.38,"view":0.85,"db":3.41,"ddsource":["ruby"],"time":"2020-08-31 12:52:35 +0000","params":{"limit":150,"items":[{"uuid":"da87e346-fcb1-43ae-86a4-b4b8fc93a626","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:58.978Z","updated_at":"1970-01-01T00:00:00.000Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"}],"sync_token":"MjoxNTk4ODc4MzQ1LjYxNTI2MDE=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":9031.99,"view":0.73,"db":3.09,"ddsource":["ruby"],"time":"2020-08-31 12:52:45 +0000","params":{"limit":150,"items":[{"uuid":"0345a53c-e782-48a1-91ce-44d62ca4a30c","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:58.978Z","updated_at":"1970-01-01T00:00:00.000Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"}],"sync_token":"MjoxNTk4ODc4MzU1Ljc3OTc1NDY=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":9034.69,"view":0.74,"db":6.85,"ddsource":["ruby"],"time":"2020-08-31 12:52:54 +0000","params":{"limit":150,"items":[{"uuid":"2f700994-c95e-459f-a664-6cf02e4f1cb6","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:58.978Z","updated_at":"1970-01-01T00:00:00.000Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"}],"sync_token":"MjoxNTk4ODc4MzY1LjAxMTcwNQ==\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"html","controller":"Api::ItemsController","action":"sync","status":200,"duration":4.72,"view":1.27,"db":0.91,"ddsource":["ruby"],"time":"2020-08-31 12:53:05 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4ODc4MzU1LjMxNDE3NDI=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":9031.65,"view":1.54,"db":3.7,"ddsource":["ruby"],"time":"2020-08-31 12:53:03 +0000","params":{"limit":150,"items":[{"uuid":"026b025f-8ef3-46ef-98b7-2294e86b4e4c","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:58.978Z","updated_at":"1970-01-01T00:00:00.000Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"}],"sync_token":"MjoxNTk4ODc4Mzc0LjI2MjI0MTg=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":9032.09,"view":0.72,"db":3.12,"ddsource":["ruby"],"time":"2020-08-31 12:53:12 +0000","params":{"limit":150,"items":[{"uuid":"b1f33c0f-f63f-4884-babb-feeb1faaacd7","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:58.978Z","updated_at":"1970-01-01T00:00:00.000Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"}],"sync_token":"MjoxNTk4ODc4MzgzLjQ5MjE5OQ==\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"*/*","controller":"Api::ItemsController","action":"sync","status":200,"duration":9040.0,"view":1.33,"db":4.12,"ddsource":["ruby"],"time":"2020-08-31 12:53:21 +0000","params":{"limit":150,"items":[{"uuid":"ad4be084-faa7-4824-aeb1-8fce0d27fef0","content_type":"[FILTERED]","created_at":"2020-08-31T12:51:58.978Z","updated_at":"1970-01-01T00:00:00.000Z","enc_item_key":"[FILTERED]","content":"[FILTERED]","auth_hash":"[FILTERED]"}],"sync_token":"MjoxNTk4ODc4MzkyLjczMzcwMzY=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}
{"method":"POST","path":"/items/sync","format":"html","controller":"Api::ItemsController","action":"sync","status":200,"duration":6.26,"view":1.23,"db":1.26,"ddsource":["ruby"],"time":"2020-08-31 12:53:35 +0000","params":{"limit":150,"items":[],"sync_token":"MjoxNTk4ODc4Mzg1LjMxMjg5NjU=\n","cursor_token":null,"api":"20190520","controller":"api/items","action":"sync","item":{}},"level":"INFO"}

Also populate `#SECRET_KEY_BASE=` in env example

While setting up a test server, I ran into issues #27 and #28. After fixing them, the app still wouldn't load as the secret_key_base variable wasn't populated. Easily fixed, but I think it would help if it were in the example env file.

Note History and Daily Email Backup

Today I set up the syncing server via docker-compose on a DigitalOcean VPS. Imported notes and syncing is working fantastic.

I do have a couple questions:

Note History - I can see the Session History of a note, but when I check the Note History Action Menu, it says there are no actions available, but the Note History Cloud link is installed. Does Note history not work on the syncing server?
Daily Email Backup - Is this not available in the syncing server?

Cheers,
Josh

Using docker to host on my home-server

Thanks for this interesting tool.
I'm trying to get this to run on my old laptop, which I use as a homeserver. I follow the instructions in the README:

$ git clone https://github.com/standardnotes/syncing-server/
$ cd syncing-server
$ cp env.sample .env  # And set DB_HOST=db. Everything else I leave as it is, even the "changeme123"
$ docker-compose up -d

Currently I'm at work, and can't easily reach my local network, however I ssh'ed into the server to run these commands, and when I do: $ wget 192.168.0.100:3000
I get the html below:

<!DOCTYPE html>
<html>
	<head>
		<meta charset="utf-8">
		<title> Standard File Server </title>
		<link rel="stylesheet" href="assets/home.css"/>
	</head>

	<body>
		<h1> Hi! You're not supposed to be here. </h1>

		<p> You might be looking for the <a href="https://app.standardnotes.org"> Standard Notes Web App</a> or the main <a href="https://standardnotes.org"> Standard Notes Website</a>. </p>
	</body>
</html>

Am I doing something wrong here? I'm also new to docker, so there might be something fundamental I don't understand.
Thanks for any help.

IPv6 Support on hosted syncing-server

Hey together,
I hope this is the right repository for my question.
It seems that the hosted syncing-server sync.standardnotes.org on AWS is not support IPv6.
Is there any plan to support that in the future? :)

HTTP parse error with nginx reverse proxy

Im using the syncing server on a raspberry pi, i have nginx setup for reverse proxy on another machine.

here is my nginx proxy config

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name notes.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

 location / {

 include /config/nginx/proxy.conf;
 proxy_pass http://192.168.1.16:3000; # IP of Raspberry pi

  }

}

when i navigate to 192.168.1.16:300 locally i get

which i expect, but navigating to my subdomain notes.domain.com and get this in the logs

I, [2020-10-20T10:07:58.977927 #73] INFO -- : {"method":"GET","path":"/","format":"html","controller":"ApplicationController","action":"home","status":200,"duration":1.68,"view":0.89,"dd":{"trace_id":"0","span_id":"0","env":"","service":"","version":""},"ddsource":["ruby"],"time":3939.59427887,"params":{"controller":"application","action":"home"},"level":"INFO"}

I get

here are the logs for this

2020-10-20 10:21:53 +0000 HTTP parse error, malformed request: #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>

all other services that use http or https work with this set up and a prox_pass the IP and port and it works fine, so not sure whats happening here.

Legal question

Hello,
If I were to develop a server based on the Standard Notes Spec, however not using the ruby implementation, but instead my own implementation in a different language, would that be subject to the same AGPL license?

standardnotes / syncing-server Goto Github PK

syncing-server's Introduction

⛔️ Deprecation Notice

Standard Notes Syncing Server

Getting started

Logs

Stopping the Server

Updating to latest version

Checking Status

Cleanup Data

Tests

syncing-server's People

Contributors

Stargazers

Watchers

Forkers

syncing-server's Issues

Recommend Projects

Recommend Topics

Recommend Org