Giter VIP home page Giter VIP logo

ssl4tor's Introduction

Self-signed SSL for https onion urls

License: AGPL v3

Note: there is a rewrite over here to separate the code into re-usable modules and improve logging, code quality and reduce the amount of duplicate code within this organisation.

Create your own httpS://example31415926535.onion websites, with your own self-signed https certificates, in a single command.

*The difference being that all those services are in the "trusted" list of all of your computers, phones and browsers, whereas your own certificate is not. That is why this project also adds your own root CA to those trusted lists.

Set up 2 onion domains and SSH on server

alt-text-1

Get the root CA and set up passwordless ssh into server

alt-text-2

Usage

Clone this repo on 2 devices;

  • server - a regular computer with Ubuntu, will host your onion websites
  • client - access the onion websites from this device

Setup Server

You can create 3 https onion domains, for 2 services, and an ssh access into the server, with:

./src/main.sh \
  --delete-onion-domain \
  --delete-projects-ssl-certs \
  --delete-root-ca-certs \
  --firefox-to-apt \
  --services 443:gitlab:443/9001:dash:9002/22:ssh:22 \
  --ssl-password somepassword \
  --get-onion-domain

This creates an ssh tunnel, and 2 dash plots that you can visit:

This is the default service that is created on your onion domain:

Get your public root ca certificate

You now have self-signed your SSL certificates for your onion domain(s) on your server.

Since it is not standard that you as a user have the authority to be an authority, you have to tell your devices to respect your new authority. This is done by adding your self-signed root ca (ca.crt) to your browser, or even computer. This is automated for you. You need the:

  • Ubuntu username of your server.
  • ssh onion of your server (shown in that last green line of the server setup gif).

Then run this on your client:

./src/main.sh \
 --1-domain-1-service \
 --setup-ssh-client \
 --get-root-ca-certificate \
 --set-server-username <Ubuntu username of your server> \
 --set-server-ssh-onion <server ssh onion>.onion

Passwordless SSH into server

As a side affect, you can now ssh into your server, from anywhere in the world, safely and passwordless:

torsocks ssh [email protected]

(It uses your unique, private & public ssh-key pair for authentication.)

Developer Requirements

(Re)-install the required submodules with:

chmod +x install-bats-libs.sh
./install-bats-libs.sh

Install:

sudo gem install bats
sudo apt install bats -y
sudo gem install bashcov
sudo apt install shfmt -y
pre-commit install
pre-commit autoupdate

Pre-commit

Run pre-commit with:

pre-commit run --all

Tests

Run the tests with:

bats test/*

Code coverage

bashcov bats test

CLI recorder

To update the two gifs in the above Readme, use:

./src/main.sh --record-cli ~/server
./src/main.sh --record-cli ~/client

To get the server.gif into your client, set up the server in qemu and ssh-copy it from your client with:

./src/main.sh \
 --setup-ssh-client \
 --set-server-username <Ubuntu username of your server> \
 --set-server-ssh-onion <server ssh onion>.onion \
 --get-server-gif

Starting QEMU (Optional)

Using qemu is not necessary, but it is a nice sandbox to give this code a try, keeping your own system nice and clean.

One can create an Ubuntu 22.10 virtual machine in qemu with:

sudo apt install qemu-system-x86 -y
mkdir -p ~/qemus
cd ~/qemus

# Download the ubuntu-22.04.2-desktop-amd64.iso into this directory.
#wget https://releases.ubuntu.com/jammy/ubuntu-22.04.2-desktop-amd64.iso
wget https://releases.ubuntu.com/lunar/ubuntu-23.04-desktop-amd64.iso

# Create the image in which the Ubuntu 22.04 VM will be created.
qemu-img create ubuntu22.img 30G

# Create the Ubuntu 2x.xx Ubuntu image.
qemu-system-x86_64 \
 --enable-kvm \
 -m 1024 \
 -machine smm=off \
 -cdrom $PWD/ubuntu-23.04-desktop-amd64.iso \
 -boot order=d ubuntu23.img

# Now manually install Ubuntu (TODO: automate, using kickstart).

And then run Ubuntu 22.04 in a sandbox on your device with:

qemu-system-x86_64 --enable-kvm -m 4096 -machine smm=off -boot order=d \
  ubuntu22*.img -smp 4 \
  -chardev qemu-vdagent,id=ch1,name=vdagent,clipboard=on \
  -device virtio-serial-pci \
  -device virtserialport,chardev=ch1,id=ch1,name=com.redhat.spice.0

Then press Ctrl+Alt+G to capture the keyboard (and mouse).

Help

Feel free to create an issue if you have any questions :)

How to help

An quick and easy way to contribute is to:

  • Reduce the output of the (main) script, make it more simple/silent.

And if you like this project, feel free to:

  • Pick an issue and fix it.
  • Create support for Windows and/or Mac.
  • Improve the test-coverage by writing more (meaningful) tests.
  • Move the developer instructions into a separate documentation, similar to: https://bats-core.readthedocs.io/en/stable/
  • Help make this repository Readme more simple. (To me this makes sense, but perhaps to a new person coming into this some context is missing. And I believe less is more in this case.)

ssl4tor's People

Contributors

a-t-0 avatar example123 avatar

Stargazers

 avatar

Watchers

 avatar  avatar

Forkers

g3n3s1sl4b

ssl4tor's Issues

Include argument to run the Python dash apps in the background.

  • Verify the local ports are available for Dash to run on.
  • If the project running on that port already is dash, close it.
  • Allow the user to automatically spin up a separate dash instance for each project (in the background).
  • Close the dash apps if the code is done running.
  • Optionally: Include graceful exit, if the code throws an error, still close the dash apps.

Add the root ca to Ubuntu (and/or Firefox) (and/or Brave).

Ubuntu

  • Store the root ca in the certificates/root/ directory of the client, instead of the root directory of the repository of the client.
  • Get the public root ca file from the server into the client and automatically add it to Ubuntu when it is retrieved.
  • Verify Ubuntu acknowledges the root CA

Then check if (apt and snap) Firefox and Brave browser and Tor browser automatically accept the root ca, If not:

APT Firefox

  • Automatically add the root CA file to apt firefox.
  • Verify apt Firefox acknowledges the root CA

Support multiple onion services for different local ports.

Source: https://stackoverflow.com/questions/64646146/how-can-i-have-multiple-tor-onions-pointing-to-the-same-listening-port

It is possible to create multiple onion services that point to the same port/application. To do this, create multiple HiddenService configurations with different data directories pointing to the same port:

torrc

# Hidden service instance 1
HiddenServiceDir /var/lib/tor/hs-1/
HiddenServiceVersion 3
HiddenServicePort 80 127.0.0.1:8080

# Hidden service instance 2
HiddenServiceDir /var/lib/tor/hs-2/
HiddenServiceVersion 3
HiddenServicePort 80 127.0.0.1:8080

For each HiddenServiceDir/HiddenServicePort combination you create that points to the same port, you will get a different onion address which ultimately point to the same service.

  • Allow the user to specify the projects in CLI args as: <local_port_nr>:<project_name>:<external_port_nr>-<local_port_nr>:<project_name>:<external_port_nr> etc.
  • Then split those into a list of projects.
  • Give the user the option: one domain per project (or not) as boolean cli arg.
  • If one domain per project: loop over the code, generate the hostnames/onion domains, and then
  • if one domain for all projects, generate the hostname with <project_name_0>_<project_name_1> etc.
  • add the accumulated data into torrc. (Distinguish in the project names end content based on the one-domain-per-service or not boolean).

Then also generate mutliple ssl certs per service.

  • Store the root CA in certificates/root
  • Store the <service_name> certificates in certificates/<service_name>.

Reduce client setup verbosity and resolve cat: write error: Broken pipe

Getting the root CA from the server into the client is still a verbose process.

  • Resolve the error message: cat: write error: Broken pipe.
  • remove the need to say "yes" to the question "Do you want to add your fingerprint"?
  • Silence all other output of these procedures.
  • Create a green message or red message that tells the user whether or not the target file has been retrieved from the server into the client successfully or not.

Automatically add root ca to snap Firefox.

Snap Firefox

  • Automatically add the root CA file to snap firefox.
  • Verify snap Firefox acknowledges the root CA

Perhaps also check whether adding to snap Brave solves the issue.

Reduce verbosity for server setup.

The following output is still generated upon running the main/single command:

dpkg-query: no packages found matching net-tools
dpkg-query: no packages found matching httping
Ensuring Firefox is installed with apt instead of snap.
src/firefox_version/firefox_version.sh: line 43: snap: command not found
src/firefox_version/firefox_version.sh: line 43: snap: command not found

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.


WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Package: *
Pin: release o=LP-PPA-mozillateam
Pin-Priority: 1001
Unattended-Upgrade::Allowed-Origins:: "LP-PPA-mozillateam:${distro_codename}";

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.


WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Firefox is installed successfully using ppa and apt.
Firefox is now installed with apt instead of snap.

dpkg-query: no packages found matching net-tools
dpkg-query: no packages found matching httping
Generating your onion domain for:gitlab
HiddenServiceDir /var/lib/tor/gitlab/
HiddenServicePort 8070 127.0.0.1:8050
Now starting tor, and waiting (max) 260 seconds to generate onion url locally.

Generating your onion domain for:dash
HiddenServiceDir /var/lib/tor/dash/
HiddenServicePort 9002 127.0.0.1:9001
Now starting tor, and waiting (max) 260 seconds to generate onion url locally.

Generating your onion domain for:ssh
HiddenServiceDir /var/lib/tor/ssh/
HiddenServicePort 22 127.0.0.1:22
Now starting tor, and waiting (max) 260 seconds to generate onion url locally.

Synchronizing state of ssh.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable ssh
TODO: assert_root_ca_files_exist
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Certificate request self-signature ok
subject=CN = yourcn
certificates/ssl_cert/gitlab/cert.pem: OK
Running dash in the background now.
Dash is running in the background for: gitlab at port:8050. Proceeding.

https://fzcrfwemptbv7fwfhqzqfhe2ifimckw5pyvsm4333aryrkrnxoweread.onion:8070
Auto enabling SSL due to https-URL
Certificate request self-signature ok
subject=CN = yourcn
certificates/ssl_cert/dash/cert.pem: OK
Running dash in the background now.
Dash is running in the background for: dash at port:9001. Proceeding.

https://eocgteprq3i4r4thgdnhmjiujqioslu3wlztgbszxxrk6bnjs4zrlaad.onion:9002
Auto enabling SSL due to https-URL
1681669453 PERROR torsocks[8493]: socks5 libc connect: Connection refused (in socks5_connect() at socks5.c:202)
Auto enabling SSL due to https-URL
1681669468 ERROR torsocks[8503]: Host unreachable (in socks5_recv_connect_reply() at socks5.c:539)
Auto enabling SSL due to https-URL
Certificate request self-signature ok
subject=CN = yourcn
certificates/ssl_cert/ssh/cert.pem: OK
Adding your SSL certificates to firefox.
  • Make the explicit statements that tell the user what the code is doing, purple.
  • Remove all other verbosity/output.

Include check to see if onion is accessible.

Source: HiveMinds/Collabora-Online#32

  • Allow starting tor without terminating tor.
  • First include check to see if onion domain is reachable:
curl --socks5-hostname 127.0.0.1:9050 https://check.torproject.org/

Already wrote that function search for "congratulations" in code.

  • Optionally, include check to see if tor log has reached 100% bootstrapping.
  • Then write function to check if own http://onion is available online over tor.
  • Then write function to check if own https://onion is available online over tor.

Include support for ssh keys

Using the Ubuntu username password over ssh is unsafe. I was surprised it works. I expect it is slightly less worse than having a normal ssh port open with password, because the onion domain is hidden by default. Still it is better to use ssh keys.

https://apple.stackexchange.com/a/285807

  • Automatically generate the relevant ssh private and public keys in the server and client to allow ssh without manual Ubuntu password entering.

Automatically add root ca to snap Brave

Brave Browser

  • Automatically add the root CA file to brave browser.
  • Verify brave browser acknowledges the root CA
    Perhaps also check whether adding root ca to snap Firefox provides insight in effective strategy.

RX record too long at https:localhost

qemu-system-x86_64 --enable-kvm -m 4096 -machine smm=off -boot order=d ubuntu22_1.img -smp 4 \
  -chardev qemu-vdagent,id=ch1,name=vdagent,clipboard=on \
  -device virtio-serial-pci \
  -device virtserialport,chardev=ch1,id=ch1,name=com.redhat.spice.0

After starting QEMU and creating the ssl cert for the onion with:

python3 src/*/*.py
# To access a local project running on localhost:8050 via: <code>.onion:443
./src/main.sh -mo -n gitlab -lpp 8050 -ppo 443
./src/main.sh -ms -n gitlab -sp somepassword
# Remove tor and accompanying files.
./src/main.sh -do -n gitlab

One gets the ssl record too long when one visits:
https:localhost:443

Resolve it, then test the onion domain again.

Simplify CLI usage.

The default usage should be:

./src/main.sh --services 8050:gitlab:8070/9001:dash:9002/22:ssh:22

And that should:

Checks

  • check if there already exist any onion domains.
  • Check if the ssl certificates for that/those services already exist.
  • check if the services are up and running.
  • check if the root ca already has been added to Ubuntu
  • check if the root ca already has been added to apt Firefox
  • check if the root ca already has been added to snap Firefox
  • check if the root ca already has been added to Brave
  • check if the root ca already has been added to Tor

Delete args

If (any) of those data types already exists, and the user did not specify the "delete" command for that type of data, throw an error.

One should be able to give 1 command -da, --delete-all to automatically delete all the pre-existing:

  • onion domains
  • ssl certificates

One should be able to give 1 command: -do, --delete-onions to automatically delete the pre-existing:

  • onion domains.

One should be able to give 1 command: -dsc --delete-service-certificates to automatically delete the pre-existing:

  • ssl certificates from the services that are given in the --services command. (Do not delete the root ca in this case).

One should be able to give 1 command: -drc --delete-root-certificate to automatically delete the pre-existing:

  • root ca certificates

Perform intended command

  • apply those certificates to the services
  • Add that certificate to Ubuntu
  • Add that certificate to firefox (if it is installed)
  • Add that certificate to Brave (if it is installed).
  • Add that certificate to Tor (if it is installed).

Disable passwordless ssh into server.

When re-running

./src/main.sh \
 --1-domain-1-service \
 --setup-ssh-client \
 --set-server-username <Ubuntu username of your server> \
 --set-server-ssh-onion <server ssh onion>.onion \
 --get-server-gif

The pre-requisites generate a new private-public key pair and add it to the server. To allow that, the server asks the client the server password. This may only be done once, and then the server should not allow password login anymore.

  • Actively disable password login on server after client has received ssh access with private-public key pair.

Integrate with nextcloud (and khal)

  • Integrate with: github.com/TruCol/Self-host-GitLab-CI-for-GitHub (In essence add the self-signed certificate to Nextcloud)
  • And verify the Nextcloud server is reachable over https locally and over tor.
  • And verify the Nextcloud server is reachable over https locally and over tor in paralell with GitLab.

CLI calendar and Phone

The Nextcloud installation also sets up a CLI calendar (I believe Khal) over tor, and sets up a phone sync over tor. Update both of these procedures to use the SSL certs generated using this repo.

Include support for vanity domains

  • Allow users to specify the desired first n characters of their onion domain, and let it run.
  • Ideally also create a safe storage for the respective private keys.

Make torrc file modular.

Allow multiple modular blocks/services to be added into the torrc file. Such that:

  • Nextcloud can be added.
  • Afterwards, GitLab can be added without overwriting/removing the Nextcloud block/entry/service from the torrc file.
  • Nextcloud can be updated afterwards without changing the GitLab block in the torrc file.

Include qr code with ssh onion domain for phones.

  • Also allow the user to copy their (ssh) onion domain through a qr code (in case copy pasting from server is difficult).
  • Write adb script to get the onion domain from qr code on usb connected phone, then get the root ca and add it to the phone's trusted root ca's.
  • Ideally, write an (android) app that directly gets the root ca from the qr code and adds it to the phone's trusted root ca's.

Include support for clearnet domains (via lets encrypt).

It may be convenient for users to be able to switch over their services from onion to clearnet in a single command.
In these cases, it may be desirable to have a root CA that is trusted by default/everyone, hence allow support for Lets encrypt for clearnet.

Inlcude CLI Gif on how to use

Move most of the prerequisites into the package itself. (e.g. pip instructions etc.).
Ideally from two separate Qemu instances.
Make sure the Readme is consistent with those instructions.

Write ssh copy commands to get updated server and client gifs.

Currently, if the behaviour/verbosity of the system is updated, the gifs are manually created and retrieved from fresh Ubuntu installations in Qemu, through their onion domain and ssh copy.

  • Write a command that (creates) and gets the server and client gif files automatically.

Ideally integrate this with (GitLab) CI and a production pipeline.

Support ssh into device running onion.

https://www.maths.tcd.ie/~fionn/misc/ssh_hidden_service/

HiddenServiceDir /var/lib/tor/ssh/
HiddenServicePort 22 127.0.0.1:22
HiddenServiceAuthorizeClient stealth clientname
  • Include instruction to get the root certificate from the virtual machine/server into the normal/client device.
scp [email protected]:foobar.txt /local/dir

https://stackoverflow.com/questions/9427553/how-to-download-a-file-from-server-using-ssh

Optional

  • Delete the special ssh onion_ssh keys if they already exists.
  • Enable a specific ssh named ssh. (E.g. generate public and private ssh key).
  • Specify a usename required for the ssh login.

Steps

  • Get the onion domain for some project from CLI. (E.g. for project ssh).
  • Give the command to ssh into the server. (Use ssl password for ssh access?)
  • Give the command to copy the public root CA cert file from the server to its client.
  • Simplify readme by using the example output from the single command to skip explaining how the user can get the torsocks ssh command.

Move add SSL certs to Nextcloud from Collabora Online into SSL4tor.

  • DO NOT Include:
set_nextcloud_port() {
  local nextcloud_port="$1"

  yellow_msg "\nConfiguring NextCloud:${nextcloud_port}, please wait...\n"
  sudo snap set nextcloud ports.http="${nextcloud_port}"
  # TODO: verify nextcloud port is set successfully.

  #The website should display:
  #Secure Connection Failed

  #An error occurred during a connection to localhost:81. SSL received a
  # record that exceeded the maximum permissible length.

  # Error code: SSL_ERROR_RX_RECORD_TOO_LONG

}

  • Include:
add_onion_to_nextcloud_trusted_domain() {

  local onion_address
  onion_address=$(sudo cat "$NEXTCLOUD_HIDDEN_SERVICE_PATH/hostname")

  # TODO: verify format of incoming onion address.

  #add Hidden Service address like a trusted domain in NextCloud instance
  sudo /snap/bin/nextcloud.occ config:system:set trusted_domains 1 --value="$onion_address"
  printf "\nThe Hidden Service address has been added like trusted domain successfully.\n"

  # TODO: verify output:
  sudo /snap/bin/nextcloud.occ config:system:get trusted_domains
}
  • Include:
dd_certs_to_nextcloud() {
  local ssl_public_key_filename="$1"
  local ssl_private_key_filename="$2"
  local merged_ca_ssl_cert_filename="$3"

  # First copy the files into nextcloud.
  # Source: https://github.com/nextcloud-snap/nextcloud-snap/issues/256
  # (see nextcloud.enable-https custom -h command).
  #sudo cp ca.pem /var/snap/nextcloud/current/ca.pem
  sudo cp "$ssl_public_key_filename" /var/snap/nextcloud/current/"$ssl_public_key_filename"
  sudo cp "$ssl_private_key_filename" /var/snap/nextcloud/current/"$ssl_private_key_filename"
  sudo cp "$merged_ca_ssl_cert_filename" /var/snap/nextcloud/current/"$merged_ca_ssl_cert_filename"
  read -p "Before enable"

  # CLI sudo /snap/bin/nextcloud.enable-https custom Says:
  sudo /snap/bin/nextcloud.enable-https custom "/var/snap/nextcloud/current/$ssl_public_key_filename" "/var/snap/nextcloud/current/$ssl_private_key_filename" "/var/snap/nextcloud/current/$merged_ca_ssl_cert_filename"
  #sudo /snap/bin/nextcloud.enable-https custom "/var/snap/nextcloud/current/cert.pem" "/var/snap/nextcloud/current/cert-key.pem" "/var/snap/nextcloud/current/fullchain.pem"
}

From here

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.