the zsh install.sh did not run yet and the 3 lines oh-my-zsh were removed. Everything else was okay. And the doctor refused to start because of missing dotfiles.

doctor version: 9f7d3d8

I thought that was a one time issue and new batches will not have it. But it keeps coming up. Investigate what the root cause is. And maybe let the doctor update the bundler at all times? Or just with the --fix flag.

gem update --system
bundle update --bundler

Should only be run if bundler --version is too old (matching a known good version by the doctor) otherwise it slows down the doctor on a healthy system.

Invalid emails should always show a warning even if there is no active github auth we can check the email against.
Could also warn on unknown tlds or email providers but not sure if that has too many false positives. But it would catch typos of email providers.

Could also check if the most recent commit that is in the fullstack challenges repo has the same email as the current git email. As in the student changed the email in the git config but did not push anything yet.

There are a few issues or warnings if the clock is not setup correctly and ntp broke.

For example this ruby -e 'require "sqlite3"' just to ensure the gem folder or ruby runtime is not corrupted?
Never seen this issue. I made it up. So not sure how much sense it makes.

• Disk space alert if less than 10GB are free
  • on mac/linux use something like df
  • on wsl df might work too if not ask mr powershell for some info
• installed C compiler?
  • there are package in apt for ubuntu which are needed to build py and ruby native extenions
  • mac kinda forces xcode on you at the very beginning
    but a macOS update might wipe or break it. So check that calling any of the xcode tools
    does not throw any errors.
• locale issues on linux and wsl
  • ✔️ possible non interactive fix

    sudo locale-gen "en_US.UTF-8"
    sudo dpkg-reconfigure locales
    

  • ensure this does not mess up setups for non english users
    only attempt to fix this if a proper error is detected
    and no locale is set. Never overwrite custom locales without user confirmation
  • ✔️ a way to check for issues is running locale
    a broken output can look like this

    locale: Cannot set LC_CTYPE to default locale: No such file or directory
    locale: Cannot set LC_MESSAGES to default locale: No such file or directory
    locale: Cannot set LC_ALL to default locale: No such file or directory
    LANG=en_US.UTF-8
    

  • can also affect postgres charsets
    so maybe suggest a postgres reinstall to the user after locale fix
    and or test if postgres is working properly
• WSL opening windows web browser fails check BROWSER env var and installed browsers in programs dir
  • [ -z "$BROWSER" ] && echo "ERROR: please define a BROWSER environment variable ⚠️" || echo "Browser defined 👌"
  • exec: "xdg-open,x-www-browser,www-browser,wslview": executable file not found in $PATH
• Stuff not being in the PATH or not being installed (command not found error. Classical candidates: brew, code, gh)
  • fixed by installing them or adding them to the path
  • alert if vscode on mac is not in the applications folder or found in the ~/Downloads folder
• brew
  • run brew doctor
  • install core if its missing git -C $(brew --repo homebrew/core) checkout master
• Wrong shell
  • call chsh if the current shell is not zsh ✔️
  • seems like on mac there is also the ui option which is a bit independent of chsh ?? (did not investigate or solve yet)
• WSL being root user (no user)
  • check if there is a user invoke powershell and set it as default. if there is none create one
• rbenv command not found
  • possibly dotfiles issue check ~/.zshrc
• broken rbenv

  • the tricky one idk how to even call it. But when homebrew is added to the path after rbenv init - is ran
    then it does not run and does not add the shims to the PATH
    which makes it tricky to debug since rbenv is in the path and brew is and everything but the shims arent
    so ruby wont be found in the rbenv dir and it falls back to the system ruby

    so make sure to check that homebrews bin path is either in the zprofile or in the zshrc above the rbenv init

• ssh/git fails (ssh -T [email protected] is unhappy)
  • check gh auth status and ~/.ssh/config
  • make sure ssh is used for git not http (check gh config and remotes of all the important git repos)
    Scratch that. The check would be grep "git_protocol: ssh" ~/.config/gh/config.yml but I am sure https is also fine so leave it
• manual config edit mistakes
  • ✔️ adding ssh-agent to ~/.zshrc
  • ✔️ adding redirect for Jupyter Notebooks
  • making ubuntu default profile on windows terminal (check the json)
• something is installed in a wrong version or there is a conflicting competitor
  • if the bundler is too old you might get NameError: uninitialized constant Gem::Source
    the fix is gem update bundler;bundle update --bundler
  • wrong rails version ✔️
  • wrong wsl version should be wsl 2 not wsl 1 ✔️
  • old lewagon/dotfiles (should maybe not auto run the older install.sh script)
    if a old webdev student switches to data he is missing the new data things for example ✔️
  • wrong ruby version ✔️
  • rvm is installed ✔️
  • asdf is installed ✔️
  • ✔️ gitsome's gh is installed instead of the proper github-cli
  • MacPorts ✔️
• missing vscode extensions
  • do not forget code --install-extension ms-vscode-remote.remote-wsl
• WSL internet or dns
  • well good luck getting the doctor on that system but if you do invoke powershell to fix dns stuff ✔️
  • check firewall and custom anti virus software that blocks wsl traffic ✔️
• missing or broken git repos
  • dotfiles (zsh install.sh)
  • fullstack-challenges (curl -s https://kitt.lewagon.com/camps/:batch/setup_script/:student | bash)
• wrong email or username
  • ✔️ cross check ~/code foldername with ssh -T [email protected] and git username etc
  • ✔️ ask gh cli for the email on github and compare it to ~/.gitconfig
  • ✔️ even though kitt is already very helpful with wrong email make sure that the kitt commands are properly copy pasted 🙈

  cd ~/code/*/fullstack-challenges
  git config --global user.email [email protected]
  git commit --allow-empty -m "New commit with fixed email"
  git push origin master
  

• Make sure ruby is installed for the right architecture on M1 macs
  • check if lipo ruby/brew matches the current architecture
• apply known fixes to ruby build errors by grepping the build log
  • test/v3ext.c:201:24: error: implicitly declaring library function 'memcmp' with type 'int (const void *, const void *, unsigned long)' [-Werror,-Wimplicit-function-declaration]
  • CFLAGS="-Wno-error=implicit-function-declaration" RUBY_CONFIGURE_OPTS='--with-readline-dir=/usr/local/opt/readline/' arch -x86_64 rbenv install 3.1.2
  • RUBY_CFLAGS="-w" CFLAGS="-w" rbenv install 3.1.2
  • RUBY_CONFIGURE_OPTS="--disable-install-doc --with-readline-dir=$(brew --prefix readline) --with-openssl-dir=$(brew --prefix openssl@3) --with-libyaml-dir=$(brew --prefix libyaml)" arch -arm64 rbenv install 3.1.2
• Student did not accept github invitation
  • detect username and search him here https://github.com/orgs/lewagon/people?query=search suggest going to https://github.com/orgs/lewagon/invitation to accept
  • only for web. data is not supposed to be in there
• Warn about unusal setups
  • custom zshrc config
    • less than x lines more than y lines (should always alert if zshrc exists but is empty for example)
    • less than x lines more than y lines after sort -u to detect issues where a echo PATHSTUFF >> .zshrc was run too much
    • detect lewagon based on lewagon specific diff to default
    • is rbenv there?
    • is oh my zsh there?
    • is nvm there?
    • is aliases loaded?
    • is evaling the plugins= line throwing a runtime error?
    • is the EDITOR variable set to code?
  • custom zprofile config
    • less than x lines more than y lines
    • check pyenv stuff being there if data bootcamp
  • custom ssh config ✔️
  • non ubuntu distro ✔️
  • old macos ✔️
  • multiple ssh keys ✔️
  • windows 8? ✔️
• if postgres is installed make sure its running and autostarted
  • ✔️ verify the role is created
  • ✔️ on windows is has to be in the zshrc to autostart
  • ✔️ is there a slim way to test rails db:create without creating a rails app?
    had it once that on wsl it hang on this sudo -u postgres psql --command "CREATE DATABASE TestDbDeleteMe;"
    not sure if that is common enough to test that with a timeout since testing stuff with timeouts is always annoying
    if its untested then it just buggs the doctor when this issue is detected
  • make sure the socket is healthy. Verify this command is safe to run without the --fix flag

  # old mac
  rm /usr/local/var/postgresql@15/postmaster.pid
  # m1
  rm /opt/homebrew/var/postgresql@15/postmaster.pid
  brew services restart postgresql@15
  

  as a fix for

  psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such file or directory
  Is the server running locally and accepting connections on that socket?
  

• probably covered by the official check script but make sure all gems are installed
  • missing gems like rake can throw some nasty errors later on
• permission issues (being root or running stuff as root)
  • no gem/yarn/pip install as root (check if that happend and broke something)
    ^ this makes no sense. running rbenv/nvm/pyenv as root will use the root zshrc file
    and thus also use the root paths or more likely not find the tool at all
  • if on mac brew ran as root once it might brick your system because some key files are owned by root them
    check those files for ownership (brew doctor also can do that)
  • ✔️ on windows when messing up the initial wsl launch the whole setup might run as root
    make sure the current user is not root and there is a user in wsl and it is also the default user
    if not invoke a powershell from within wsl that sets it using ubuntu config --default-user new_user_name (hope that works 🤞 )
• installing docker is no simple command to copy and paste but a link to the docker website
  • WSL user might install docker on the windows host system
• In the data setup there is a file download and a zshrc edit based on the file path
  • echo 'export GOOGLE_APPLICATION_CREDENTIALS=/path/to/the/SERVICE_ACCOUNT_JSON_FILE_CONTAINING_YOUR_SECRET_KEY.json' >> ~/.zshrc
  • especially on wsl this might be some \c\mnt\ whatever wsl windows host path
  • but even on mac that step is kinda tricky make sure the line is only once in the zshrc and the path points to a existing file
• System Integrity Protection is turned off csrutl status should say enabled
  • this is super edge case but there was a student once where in a shared laptop scenario
    a friend installed a tiling window manager on mac which requires you to turn off SIP
    which is a security risk but also makes the ruby build fail in version 3.1.2 because of this
• curl expired ssl certs some smart stuff on slack

do a test curl and check for this response
against github and lewagon urls

error 60: SSL certificate problem: certificate has expired

(60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a “bundle”
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn’t adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you’d like to turn off curl’s verification of the certificate, use
 the -k (or --insecure) option.
HTTPS-proxy has similar options --proxy-cacert and --proxy-insecure.

  mv /etc/ssl/cert.pem /etc/ssl/cert.pem.org 
  curl https://curl.se/ca/cacert.pem > /etc/ssl/cert.pem

test this and if it fixes the error show it as suggested fix or fix with --fix
do not just edit that automatically

• Now here is a weird one .. hear me out: IPV6

  • Its a common theme in my life that the unexpected solution to a weird problem was ipv4
    but I think one of the first times I have seen it on a students native ubuntu laptop (took forever to debug)

    So deactivating ipv6 might be worth hinting on certain problem descriptions.
    But it is not very lewagon setup specific and a rather drastic change. So I suggest doing something more sophisticated like
    properly checking if ipv6 is actually used by the current ISP. And then making sure it actually has a bad impact on the
    current system.
    One could quickly try pinging known important domains like the gem and yarn repo urls.
    Or running a full gem/yarn command to see how toggling ipv6 impacts it.

    Update: the doctor now has g_ipv6_ok which is 0 if ping or curl to ipv6 did not work
    the doctor does not show a warning unless --verbose if ipv6 does not work. Because it is far too common
    and not always an issue.
    But now if yarn or gem commands fail and $g_ipv6_ok == 0 then the doctor could suggest it might be ipv6

    Update: it happend again. We used networksetup -setv6off Wi-Fi on a mac that got timeouts on vscode extension installs with ipv6 addresses in the error messages. That fixed it just fine. The doctor correctly set g_ipv6_ok=0 in that case. So on the vscode extension step the doctor could retry with networksetup -setv6off Wi-Fi on failur if also is mac and is wifi. There is also networksetup -setv6off Ethernet. I assume the command is not permanent. So it should for sure still show a warning that ipv6 had to be turned off to get something to work. If it is permanent on the other hand that would also be bad. The doctor should not permanently break ipv6.

  • ipv4 fallback for rubygems.org rubygems/rubygems#4555 (comment)

Even better would be some kind of linting for old bash versions.
Can shellcheck do it?

The doctor should auto detect this and all similar git corruption errors: https://stackoverflow.com/questions/11706215/how-can-i-fix-the-git-error-object-file-is-empty

A few students now had broken git folders because their machines shut down.

There are a few safe commands the doctor should run if it detects any issues. Such as git fsck and then there are the more dangerous ones that should only be suggested or run when --fix is passed. Such as:

find .git/objects/ -type f -empty | xargs rm
git fetch -p
git fsck --full

cmd+shift+p "add to PATH"

can fail if the destination file which is at /usr/local/bin or somewhere like that is already set to an invalid sym link.
vscode does not automatically delete it. The doctor should do that even without --fix if a dead symlink is blocking the destination. Especially if the sym link is pointing to Desktop or Downloads instead of the Applications folder.

This can happen if vscode is installed to the PATH and then moved.

Somewhere in the logfile it says which role owns the database. If the postgres database was initialized on another macos user than the one in use now psql will not be authed by default. So one has to do psql -U username -d postgres and then create a role for it self.

Saw a system where the browser was configured to open json files
Then the "open settings json" button in the windows terminal would open the browser. Which does not allow to change the settings.json

The CI should fail if those are not matching official Le Wagon instructions

When verbosity is set to 2. Dump all postgres client and server binary paths. Show if port is used. Show all processes running that match pg. Show all brew installed pg and wether the service is active. Show all apple plist files. Show all pg pid files.

Possibly a --only=pg would make sense. Could also add --only=dns and similar. Which sets arg_only= and then conditionally only runs one part of the doctor to not pollute output with unrelated warnings.

This repo is about helper scripts for the setup provided by the LeWagon coding bootcamp

https://github.com/LeWagon/data-setup
https://github.com/LeWagon/setup

There are two is one scripts:

doc.sh

Short for doctor. Also known as "the doctor" to sound epic.
The doctor is a non interactive shell script that can detect and optionally fix known setup issues.
By default it will not do any changes and only alert if it found possible issues.
When passed a --fix flag it will actually attempt to fix the issues.

gwen.sh

There is no point in starting this if there is also chat gpt.

Short for Gwendoline. Also known as the robot TA.
Since the doctor is not interactive and only controlled by possibly complicated cli flags. Gwendoline acts as a fully interactive frontend.
Gwendoline tries to give non technical people a hand and help them talk to the doctor.

Gwendoline behaves similar to a techsupport chatbot. Ready for smalltalk and asking the user who he is and what he needs. And is then calling the doctor under the hood. At some point gwen should be able to fully replace all basic TA tasks. And properly responding to user input ranging from:

how to get all elements array?? HELP! (could use codetogo.io)

to

LoadError: cannot load such file -- rspec/core/rake_task

todo: move those to a new issue

+ I forgot my password on wsl launch powershell to set new passwd
+ unexpected end of input expecting end
+ rake works locally but not on kitt (if mac this can be due to filename casing)

Currently the doctor just references the setup instructions on how to get the dotfiles working. But it could also do it if it has a logged in github user. Make sure that it does not get stuck on the ssh (yes/no) prompt. And use some SSH flags or environment variables for the clone.

common issue on running bundle in rails projects

An error occurred while installing psych (5.1.1.1), and Bundler cannot continue.

WSL/Linux fix

sudo apt-get install libyaml-dev

Do not install libyaml-dev at all times. Also ideally do not install the psyche gem at all times. There has to be some non intrusive way to properly check this issue.

doc just calls powershell.exe but it might not be a command (seen that)

usage: uniq [-c | -d | -u] [-i] [-fields...

❌ possible fix:
echo $(($(printf 'a\na\nb\nb\nb' | sort | uniq -dc | awk '{print $1}' | paste -sd+)))

winsock

wsl --shutdown
netsh winsock reset
netsh int ip reset all
netsh winhttp reset proxy
ipconfig /flushdns
netsh winsock reset
shutdown /r 

dns

TODO

lxss

either start

Get-Service LxssManager | Restart-Service

or restart

sc start LxssManager

or better this but it needs cmd and does not work in powershell

sc config LxssManager start=auto

windows terminal nullbytes

also check for nullbytes in registry
https://gist.github.com/lhecker/393252fb1e95b3bc118d40ace802a505

wslservice

microsoft/WSL#11273

sc.exe config wslservice start= demand

For example the extension in this loop printf '%b\n' "$WANTED_VSCODE_EXTENSIONS_WEB\n$win_ext" | while IFS='' read -r ext

the pipe creates a new context so they are not tracked.

brew

Based on this I came up with the following china mirrow brew install command

HOMEBREW_BREW_GIT_REMOTE=http://mirrors.ustc.edu.cn/brew.git/ HOMEBREW_CORE_GIT_REMOTE=http://mirrors.ustc.edu.cn/homebrew-core.git/ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

The doctor could potentially check if behind the china firewall and then suggest running that command in case brew is detected as being unhealthy. This should not be needed for everybody.

ruby gems

It could also check the recommended china gem mirror from the official lewagon setup instructions.