allow docker host access to enclave ports under linux ? about kurtosis HOT 10 OPEN

I am not using additional param, just straight setup from this(basically default)

https://geth.ethereum.org/docs/fundamentals/kurtosis

it seems that the 'dora' service can communicate with the geth container so it must be some docker level setup issue but that being wrapped inside kurtosis, I don't know where to start to trace

from kurtosis.

ok, it seems that it is the package itself https://github.com/ethpandaops/ethereum-package/tree/main is having issue, the default quickstart package runs find as the proper network configurations are used for the various containers

need to bug the package maintainer :-)

from kurtosis.

Hey @garyng2000 ! How are you attempting to access the ports? eg. localhost:61234with 61234 being the port on the host mapped to the port of the service inside the enclave ?

from kurtosis.

curl -s http://localhost: as shown in 'kurtosis enclave inspect '

========================================== User Services ==========================================
UUID Name Ports Status
9115b4d644b6 cl-1-lighthouse-geth http: 4000/tcp -> http://127.0.0.1:32877 RUNNING
metrics: 5054/tcp -> http://127.0.0.1:32876
tcp-discovery: 9000/tcp -> 127.0.0.1:32875
udp-discovery: 9000/udp -> 127.0.0.1:32789
683d6f13726e cl-2-lighthouse-geth http: 4000/tcp -> http://127.0.0.1:32880 RUNNING
metrics: 5054/tcp -> http://127.0.0.1:32879
tcp-discovery: 9000/tcp -> 127.0.0.1:32878
udp-discovery: 9000/udp -> 127.0.0.1:32790
f5803bd182e3 cl-3-teku-geth http: 4000/tcp -> http://127.0.0.1:32883 RUNNING
metrics: 8008/tcp -> 127.0.0.1:32882
tcp-discovery: 9000/tcp -> 127.0.0.1:32881
udp-discovery: 9000/udp -> 127.0.0.1:32791
7691562c9fb5 dora http: 8080/tcp -> http://127.0.0.1:32886 RUNNING
4a24f863b367 el-1-geth-lighthouse engine-rpc: 8551/tcp -> 127.0.0.1:32862 RUNNING
metrics: 9001/tcp -> 127.0.0.1:32861
rpc: 8545/tcp -> http://127.0.0.1:32864
tcp-discovery: 30303/tcp -> 127.0.0.1:32860
udp-discovery: 30303/udp -> 127.0.0.1:32786
ws: 8546/tcp -> 127.0.0.1:32863
e90e9bb1ee16 el-2-geth-lighthouse engine-rpc: 8551/tcp -> 127.0.0.1:32867 RUNNING
metrics: 9001/tcp -> 127.0.0.1:32866
rpc: 8545/tcp -> http://127.0.0.1:32869
tcp-discovery: 30303/tcp -> 127.0.0.1:32865
udp-discovery: 30303/udp -> 127.0.0.1:32787
ws: 8546/tcp -> 127.0.0.1:32868
6c2021fe91ad el-3-geth-teku engine-rpc: 8551/tcp -> 127.0.0.1:32872 RUNNING
metrics: 9001/tcp -> 127.0.0.1:32871
rpc: 8545/tcp -> http://127.0.0.1:32874
tcp-discovery: 30303/tcp -> 127.0.0.1:32870
udp-discovery: 30303/udp -> 127.0.0.1:32788
ws: 8546/tcp -> 127.0.0.1:32873
68d7758bdc9c validator-key-generation-cl-validator-keystore RUNNING
4be8bbb05868 vc-1-geth-lighthouse metrics: 8080/tcp -> http://127.0.0.1:32884 RUNNING
22175e303736 vc-2-geth-lighthouse metrics: 8080/tcp -> http://127.0.0.1:32885 RUNNING

from kurtosis.

so in this case curl -s http://localhost:32864. I get reset by peer error
I can get into the the container shell and access it without issue

wondering if it has anything to do with this

host.docker.internal:host-gateway

which is needed for my other docker-compose setup

below is the exact input/output
root@hqprdubada:/mnt/data3/chatgpt# curl -X POST -H "Content-Type: application/json" --data '{"method":"eth_blockNumber","params":[],"id":1,"jsonrpc":"2.0"}' http://127.0.0.1:32864
curl: (56) Recv failure: Connection reset by peer

from kurtosis.

what are the network params you're running with? or are you using defaults? Also how are you seeing the host.docker.internal config/does it work if you unset it?

my hypothesis is that this setting host.docker.internal:host-gateway might make services in the enclave communicate on the host network which means when the el-1-geth-lighthouse attempts to connect with other peers on the IP address assigned to them on the enclave subnetwork, that IP address is invalid on the host network?

from kurtosis.

Any chance you can join our discord? I'm happy to continue assisting you there! https://discord.com/invite/HUapYX9RvV

from kurtosis.

thanks for the help, I would try to use that channel later.

I may have find the reason as inspecting the geth container itself shows me this about network

"Networks": {
"kt-private3151908net": {
"IPAMConfig": {
"IPv4Address": "172.16.0.13"
},
"Links": null,
"Aliases": [
"el-3-geth-teku"
],
"MacAddress": "02:42:ac:10:00:0d",
"NetworkID": "150bb87a432c106db2b39f86325828a0e210a8d264e137787858bc42d8f6664d",
"EndpointID": "2951b8e68dbfbce91a7a8454bdda0bd8a3af13c46837b4696ce0b20beae6fa7b",
"Gateway": "172.16.0.4",
"IPAddress": "172.16.0.13",
"IPPrefixLen": 22,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DriverOpts": null,
"DNSNames": [
"el-3-geth-teku--6c2021fe91ad46d5b8e12db3a8fc7c82",
"el-3-geth-teku",
"81f987de1343"
]
}
}

which may be conflicting with my hosting machine which is also using 172.16.0.x, so the question may then be anyway to configure the network setup in kurtosis for the 'private docker network' ? though just a WAG at the moment

from kurtosis.

That makes sense. Background - Kurtosis creates a Docker bridge subnetwork for each enclave so that services within each enclave can only communicate with other services in the enclave. Any custom config that might alter how Docker networking works might affect this.

from kurtosis.

something doesn't make sense, in the created brdige(which seems to be what kurtosis is using)

br-b4e4c03a202a: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1440
inet 172.16.0.2 netmask 255.255.252.0 broadcast 172.16.3.255
inet6 fe80::42:53ff:fe3f:35cc prefixlen 64 scopeid 0x20
ether 02:42:53:3f:35:cc txqueuelen 0 (Ethernet)
RX packets 215 bytes 26166 (26.1 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 51 bytes 6221 (6.2 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

so this is using a bridging network thus the host mapping probably is not working as my limited understanding of docker is it needs to be in 'host' network mode for the port mapping to work

from kurtosis.