Giter VIP home page Giter VIP logo

Comments (8)

mickg10 avatar mickg10 commented on June 10, 2024

I am guessing that it is maybe picking up the wrong interface to send things out on?

As it says even ipv4 addr is unreachable 176.58.93.154:443, and yet:

root@udmp2333:/data/tailscale# telnet 176.58.93.154 443
Trying 176.58.93.154...
Connected to 176.58.93.154.
Escape character is '^]'.
^]

telnet> quit
Connection closed.

from tailscale.

mickg10 avatar mickg10 commented on June 10, 2024

Also, for controlplane:

root@udmp2333:/data/tailscale# telnet 2a05:d014:386:202:f041:44b6:9559:668 443
Trying 2a05:d014:386:202:f041:44b6:9559:668...
telnet: Unable to connect to remote host: Network is unreachable
root@udmp2333:/data/tailscale# telnet controlplane.tailscale.com 443
Trying 3.125.149.81...
Connected to controlplane.tailscale.com.
Escape character is '^]'.
^]

telnet> quit
Connection closed.

So, A lookup and telnet to it works, but AAAA lookup and telnet to it does not.

from tailscale.

mickg10 avatar mickg10 commented on June 10, 2024

And full tailscale logs

Apr 18 07:06:09 udmp2333 systemd[1]: Starting Tailscale node agent...
░░ Subject: A start job for unit tailscaled.service has begun execution
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit tailscaled.service has begun execution.
░░ 
░░ The job identifier is 3912.
Apr 18 07:06:09 udmp2333 tailscaled[645386]: logtail started
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Program starting: v1.64.0-t7e9bebdb2-gf314c5be5, Go 1.22.2: []string{"/usr/sbin/tailscaled", "--state=/var/lib/tailscale/tailscaled.state", "--socket=/run/tailscale/tailscaled.sock", "--port=41641"}
Apr 18 07:06:09 udmp2333 tailscaled[645386]: RAW-STDERR: ***
Apr 18 07:06:09 udmp2333 tailscaled[645386]: RAW-STDERR: *** Lines prefixed with RAW-STDERR below bypassed logtail and probably come from a previous run of the program
Apr 18 07:06:09 udmp2333 tailscaled[645386]: RAW-STDERR: ***
Apr 18 07:06:09 udmp2333 tailscaled[645386]: RAW-STDERR:
Apr 18 07:06:09 udmp2333 tailscaled[645386]: RAW-STDERR: network is unreachable\n"}
Apr 18 07:06:09 udmp2333 tailscaled[645386]: LogID: 4fa038657983321a890074e071fbb242554a989ae17277d4ae57eb7cca485c66
Apr 18 07:06:09 udmp2333 tailscaled[645386]: logpolicy: using $STATE_DIRECTORY, "/var/lib/tailscale"
Apr 18 07:06:09 udmp2333 dbus-daemon[671]: [system] Activating via systemd: service name='org.freedesktop.resolve1' unit='dbus-org.freedesktop.resolve1.service' requested by ':1.25' (uid=0 pid=645386 comm="/usr/sbin/tailscaled --state=/var/lib/tailscale/ta")
Apr 18 07:06:09 udmp2333 dbus-daemon[671]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.resolve1.service': Unit dbus-org.freedesktop.resolve1.service not found.
Apr 18 07:06:09 udmp2333 tailscaled[645386]: dns: [rc=unknown ret=direct]
Apr 18 07:06:09 udmp2333 tailscaled[645386]: dns: using "direct" mode
Apr 18 07:06:09 udmp2333 tailscaled[645386]: dns: using *dns.directManager
Apr 18 07:06:09 udmp2333 tailscaled[645386]: deleting [-j ts-input] in filter/INPUT: running [/usr/sbin/iptables -t filter -D INPUT -j ts-input --wait]: exit status 2: iptables v1.8.7 (legacy): Couldn't load target `ts-input':No such file or directory
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Try `iptables -h' or 'iptables --help' for more information.
Apr 18 07:06:09 udmp2333 tailscaled[645386]: deleting [-j ts-forward] in filter/FORWARD: running [/usr/sbin/iptables -t filter -D FORWARD -j ts-forward --wait]: exit status 2: iptables v1.8.7 (legacy): Couldn't load target `ts-forward':No such file or directory
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Try `iptables -h' or 'iptables --help' for more information.
Apr 18 07:06:09 udmp2333 tailscaled[645386]: deleting [-j ts-postrouting] in nat/POSTROUTING: running [/usr/sbin/iptables -t nat -D POSTROUTING -j ts-postrouting --wait]: exit status 2: iptables v1.8.7 (legacy): Couldn't load target `ts-postrouting':No such file or directory
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Try `iptables -h' or 'iptables --help' for more information.
Apr 18 07:06:09 udmp2333 tailscaled[645386]: deleting [-j ts-input] in filter/INPUT: running [/usr/sbin/ip6tables -t filter -D INPUT -j ts-input --wait]: exit status 2: ip6tables v1.8.7 (legacy): Couldn't load target `ts-input':No such file or directory
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Try `ip6tables -h' or 'ip6tables --help' for more information.
Apr 18 07:06:09 udmp2333 tailscaled[645386]: deleting [-j ts-forward] in filter/FORWARD: running [/usr/sbin/ip6tables -t filter -D FORWARD -j ts-forward --wait]: exit status 2: ip6tables v1.8.7 (legacy): Couldn't load target `ts-forward':No such file or directory
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Try `ip6tables -h' or 'ip6tables --help' for more information.
Apr 18 07:06:09 udmp2333 tailscaled[645386]: [RATELIMIT] format("deleting %v in %s/%s: %v")
Apr 18 07:06:09 udmp2333 tailscaled[645386]: cleanup: list tables: netlink receive: invalid argument
Apr 18 07:06:09 udmp2333 tailscaled[645386]: wgengine.NewUserspaceEngine(tun "tailscale0") ...
Apr 18 07:06:09 udmp2333 systemd[1]: Started Tailscale node agent.
░░ Subject: A start job for unit tailscaled.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit tailscaled.service has finished successfully.
░░ 
░░ The job identifier is 3912.
Apr 18 07:06:09 udmp2333 tailscaled[645386]: setting link attributes: setsockopt: protocol not available
Apr 18 07:06:09 udmp2333 dbus-daemon[671]: [system] Activating via systemd: service name='org.freedesktop.resolve1' unit='dbus-org.freedesktop.resolve1.service' requested by ':1.25' (uid=0 pid=645386 comm="/usr/sbin/tailscaled --state=/var/lib/tailscale/ta")
Apr 18 07:06:09 udmp2333 dbus-daemon[671]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.resolve1.service': Unit dbus-org.freedesktop.resolve1.service not found.
Apr 18 07:06:09 udmp2333 networkd-dispatcher[690]: WARNING:Unknown index 29 seen, reloading interface list
Apr 18 07:06:09 udmp2333 tailscaled[645386]: dns: [rc=unknown ret=direct]
Apr 18 07:06:09 udmp2333 tailscaled[645386]: dns: using "direct" mode
Apr 18 07:06:09 udmp2333 tailscaled[645386]: dns: using *dns.directManager
Apr 18 07:06:09 udmp2333 tailscaled[645386]: link state: interfaces.State{defaultRoute= ifs={br0:[192.168.17.70/24 llu6] eth8:[68.174.248.*/20 llu6] tun1:[192.168.30.17/32 llu6] tun2:[10.0.3.1/24 llu6]} v4=true v6=false}
Apr 18 07:06:09 udmp2333 tailscaled[645386]: onPortUpdate(port=41641, network=udp6)
Apr 18 07:06:09 udmp2333 tailscaled[645386]: router: using firewall mode pref
Apr 18 07:06:09 udmp2333 tailscaled[645386]: router: default choosing iptables
Apr 18 07:06:09 udmp2333 tailscaled[645386]: router: v6 = true, v6filter = true, v6nat = true
Apr 18 07:06:09 udmp2333 tailscaled[645386]: onPortUpdate(port=41641, network=udp4)
Apr 18 07:06:09 udmp2333 tailscaled[645386]: magicsock: disco key = d:3d8ce7f92d1c3ed6
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Creating WireGuard device...
Apr 18 07:06:09 udmp2333 tailscaled[645386]: external route: up
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Bringing WireGuard device up...
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Bringing router up...
Apr 18 07:06:09 udmp2333 systemd-networkd[403]: tailscale0: Link UP
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Clearing router settings...
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Starting network monitor...
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Engine created.
Apr 18 07:06:09 udmp2333 systemd-networkd[403]: tailscale0: Gained carrier
Apr 18 07:06:09 udmp2333 systemd-networkd[403]: tailscale0: Gained IPv6LL
Apr 18 07:06:09 udmp2333 tailscaled[645386]: pm: using backend prefs for "profile-a976": Prefs{ra=true dns=true want=false routes=[0.0.0.0/0 ::/0 192.168.17.0/24] snat=true nf=on update=on Persist{lm=, o=, n=[Aj5af] u="*@*.net"}}
Apr 18 07:06:09 udmp2333 tailscaled[645386]: envknob: PORT="41641"
Apr 18 07:06:09 udmp2333 tailscaled[645386]: logpolicy: using $STATE_DIRECTORY, "/var/lib/tailscale"
Apr 18 07:06:09 udmp2333 tailscaled[645386]: got LocalBackend in 56ms
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Start
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Backend: logs: be:4fa038657983321a890074e071fbb242554a989ae17277d4ae57eb7cca485c66 fe:
Apr 18 07:06:09 udmp2333 tailscaled[645386]: control: client.Login(false, 0)
Apr 18 07:06:09 udmp2333 tailscaled[645386]: health("overall"): error: state=Stopped, wantRunning=false
Apr 18 07:06:09 udmp2333 tailscaled[645386]: control: doLogin(regen=false, hasUrl=false)
Apr 18 07:06:09 udmp2333 tailscaled[645386]: Switching ipn state NoState -> Stopped (WantRunning=false, nm=false)
Apr 18 07:06:09 udmp2333 tailscaled[645386]: wgengine: Reconfig: configuring userspace WireGuard config (with 0/0 peers)
Apr 18 07:06:09 udmp2333 tailscaled[645386]: wgengine: Reconfig: configuring router
Apr 18 07:06:09 udmp2333 tailscaled[645386]: wgengine: Reconfig: configuring DNS
Apr 18 07:06:09 udmp2333 tailscaled[645386]: dns: Set: {DefaultResolvers:[] Routes:{} SearchDomains:[] Hosts:0}
Apr 18 07:06:09 udmp2333 tailscaled[645386]: dns: Resolvercfg: {Routes:{} Hosts:0 LocalDomains:[]}
Apr 18 07:06:09 udmp2333 tailscaled[645386]: dns: OScfg: {}
Apr 18 07:06:09 udmp2333 tailscaled[645386]: control: trying bootstrapDNS("derp10d.tailscale.com", "192.73.240.132") for "controlplane.tailscale.com" ...
Apr 18 07:06:09 udmp2333 tailscaled[645386]: control: bootstrapDNS("derp10d.tailscale.com", "192.73.240.132") for "controlplane.tailscale.com" error: Get "https://derp10d.tailscale.com/bootstrap-dns?q=controlplane.tailscale.com": dial tcp 192.73.240.132:443: connect: network is unreachable
Apr 18 07:06:09 udmp2333 tailscaled[645386]: control: trying bootstrapDNS("derp4g.tailscale.com", "2a00:dd80:20::8f") for "controlplane.tailscale.com" ...
Apr 18 07:06:09 udmp2333 tailscaled[645386]: control: bootstrapDNS("derp4g.tailscale.com", "2a00:dd80:20::8f") for "controlplane.tailscale.com" error: Get "https://derp4g.tailscale.com/bootstrap-dns?q=controlplane.tailscale.com": dial tcp [2a00:dd80:20::8f]:443: connect: network is unreachable
Apr 18 07:06:09 udmp2333 tailscaled[645386]: control: trying bootstrapDNS("derp5d.tailscale.com", "43.245.48.250") for "controlplane.tailscale.com" ...
Apr 18 07:06:09 udmp2333 tailscaled[645386]: control: bootstrapDNS("derp5d.tailscale.com", "43.245.48.250") for "controlplane.tailscale.com" error: Get "https://derp5d.tailscale.com/bootstrap-dns?q=controlplane.tailscale.com": dial tcp 43.245.48.250:443: connect: network is unreachable
Apr 18 07:06:09 udmp2333 tailscaled[645386]: control: trying bootstrapDNS("derp21d.tailscale.com", "2607:f740:50::ca4") for "controlplane.tailscale.com" ...
Apr 18 07:06:09 udmp2333 tailscaled[645386]: control: bootstrapDNS("derp21d.tailscale.com", "2607:f740:50::ca4") for "controlplane.tailscale.com" error: Get "https://derp21d.tailscale.com/bootstrap-dns?q=controlplane.tailscale.com": dial tcp [2607:f740:50::ca4]:443: connect: network is unreachable
Apr 18 07:06:09 udmp2333 tailscaled[645386]: control: trying bootstrapDNS("derp25b.tailscale.com", "102.67.167.245") for "controlplane.tailscale.com" ...
Apr 18 07:06:09 udmp2333 tailscaled[645386]: [RATELIMIT] format("control: trying bootstrapDNS(%q, %q) for %q ...")

from tailscale.

mickg10 avatar mickg10 commented on June 10, 2024

IPV4 routing (with IP addr redacted)

ip route show table all  | grep -v local
default via 68.174.240.1 dev eth8 table 201.eth8 proto dhcp 
blackhole default table 251.blackhole proto PBR 
10.0.3.0/24 dev tun2 proto kernel scope link src 10.0.3.1 
68.174.240.0/20 dev eth8 proto kernel scope link src 68.174.248.* 
192.168.17.0/24 dev br0 proto kernel scope link src 192.168.17.70 
192.168.19.0/24 via 192.168.30.16 dev tun1 
192.168.30.16 dev tun1 proto kernel scope link src 192.168.30.17 

from tailscale.

irbekrm avatar irbekrm commented on June 10, 2024

Just to clarify- after a brief look at your logs, it looks like this device was previously successfully running at 1.64 and started failing after a restart, is that correct?

As you say, looks like it cannot reach control plane or the relay servers either on IPv4 or IPv6.
Just to double clarify, what is the output of ip route get 2a00:dd80:3c::3d5, ip route get 176.58.93.154 on this machine?

From a brief look I don't see recent changes in the logic to figure out the default route for connections to control plane/derp. I wonder if we should log more, i.e what interface was chosen.

from tailscale.

mickg10 avatar mickg10 commented on June 10, 2024

The device failed after upgrading form 1.62.x - not just after a restart.

root@udmp2333:~# ip route get 176.58.93.154
176.58.93.154 via 68.174.240.1 dev eth8 table 201.eth8 src 68.174.248.36 uid 0 
    cache 

root@udmp2333:~# ip route get 2a00:dd80:3c::3d5
RTNETLINK answers: Network is unreachable

Yeah - logging the interface makes sense. Or just allow a cmd-line that specifies the external interface - that can be hard to auto-detect in pathological cases anyhow..

from tailscale.

mickg10 avatar mickg10 commented on June 10, 2024

So - one more observation.

If running with --tun userspace-networking it works just fine - but can't work as a subnet router unfortunately. So probably a tunnel selection of some sort.

from tailscale.

mickg10 avatar mickg10 commented on June 10, 2024

Upgraded to 1.66 - same issue. Any updates on triaging this?

from tailscale.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.