Comments (5)
jnummelin
commented on June 24, 2024
Could you double check that the LB pods are created as expected with kubectl get pod <lb-pod-name> -o yaml, just to make sure all the needed bits are there.
If those still look as expected, maybe try to run similar pod as the LB pods would be and see what cat /proc/sys/net/ipv4/ip_forward actually dumps out. If ip_forward is enabled on the host, I don't immediately see why it would fail on pod with NET_ADMIN capability set.
from akrobateo.
dhoard
commented on June 24, 2024
@jnummelin I see where ip_forward is enabled ...
initContainers:
- command:
- sh
- -c
- sysctl -w net.ipv4.ip_forward=1
Here is the output for one of the pods ...
apiVersion: v1
kind: Pod
metadata:
annotations:
cni.projectcalico.org/podIP: 10.42.2.18/32
creationTimestamp: "2019-09-12T14:13:28Z"
generateName: akrobateo-tomcat-test-
labels:
akrobateo.kontena.io/svcname: tomcat-test
app: akrobateo-tomcat-test
controller-revision-hash: 6fb4d87bb6
pod-template-generation: "1"
name: akrobateo-tomcat-test-42wkq
namespace: default
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: DaemonSet
name: akrobateo-tomcat-test
uid: 7e193faa-d567-11e9-b711-5254007875e7
resourceVersion: "157713"
selfLink: /api/v1/namespaces/default/pods/akrobateo-tomcat-test-42wkq
uid: 7e26d155-d567-11e9-b711-5254007875e7
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchFields:
- key: metadata.name
operator: In
values:
- 172.20.1.156
containers:
- env:
- name: SRC_PORT
value: "80"
- name: DEST_PROTO
value: TCP
- name: DEST_PORT
value: "80"
- name: DEST_IP
value: 10.43.177.188
image: registry.pharos.sh/kontenapharos/akrobateo-lb:latest
imagePullPolicy: IfNotPresent
name: http
ports:
- containerPort: 80
hostPort: 80
name: http
protocol: TCP
resources: {}
securityContext:
capabilities:
add:
- NET_ADMIN
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-tjrs9
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
initContainers:
- command:
- sh
- -c
- sysctl -w net.ipv4.ip_forward=1
image: registry.pharos.sh/kontenapharos/akrobateo-lb:latest
imagePullPolicy: Always
name: sysctl
resources: {}
securityContext:
privileged: true
terminationMessagePath: /dev/termination-log
apiVersion: v1
kind: Pod
metadata:
annotations:
cni.projectcalico.org/podIP: 10.42.2.18/32
creationTimestamp: "2019-09-12T14:13:28Z"
generateName: akrobateo-tomcat-test-
labels:
akrobateo.kontena.io/svcname: tomcat-test
app: akrobateo-tomcat-test
controller-revision-hash: 6fb4d87bb6
pod-template-generation: "1"
name: akrobateo-tomcat-test-42wkq
namespace: default
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: DaemonSet
name: akrobateo-tomcat-test
uid: 7e193faa-d567-11e9-b711-5254007875e7
resourceVersion: "157713"
selfLink: /api/v1/namespaces/default/pods/akrobateo-tomcat-test-42wkq
uid: 7e26d155-d567-11e9-b711-5254007875e7
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchFields:
- key: metadata.name
operator: In
values:
- 172.20.1.156
containers:
- env:
- name: SRC_PORT
value: "80"
- name: DEST_PROTO
value: TCP
- name: DEST_PORT
value: "80"
- name: DEST_IP
value: 10.43.177.188
image: registry.pharos.sh/kontenapharos/akrobateo-lb:latest
imagePullPolicy: IfNotPresent
name: http
ports:
- containerPort: 80
hostPort: 80
name: http
protocol: TCP
resources: {}
securityContext:
capabilities:
add:
- NET_ADMIN
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-tjrs9
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
initContainers:
- command:
- sh
- -c
- sysctl -w net.ipv4.ip_forward=1
image: registry.pharos.sh/kontenapharos/akrobateo-lb:latest
imagePullPolicy: Always
name: sysctl
resources: {}
securityContext:
privileged: true
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-tjrs9
readOnly: true
nodeName: 172.20.1.156
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
``
from akrobateo.
dhoard
commented on June 24, 2024
Here is my test Deployment / Service yaml
---
apiVersion: v1
kind: Service
metadata:
name: tomcat-test
labels:
app: tomcat
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
selector:
app: tomcat
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: tomcat-test
labels:
app: tomcat
spec:
replicas: 1
selector:
matchLabels:
app: tomcat
release: test
template:
metadata:
labels:
app: tomcat
release: test
spec:
volumes:
- name: app-volume
emptyDir: {}
initContainers:
- name: war
image: ananwaresystems/webarchive:1.0
imagePullPolicy: IfNotPresent
command:
- "sh"
- "-c"
- "cp /*.war /app"
volumeMounts:
- name: app-volume
mountPath: /app
containers:
- name: tomcat
image: tomcat:7.0
imagePullPolicy: IfNotPresent
volumeMounts:
- name: app-volume
mountPath: /usr/local/tomcat/webapps
ports:
- containerPort: 8080
hostPort: 8009
livenessProbe:
httpGet:
path: /sample
port: 8080
initialDelaySeconds: 60
periodSeconds: 30
readinessProbe:
httpGet:
path: /sample
port: 8080
initialDelaySeconds: 60
periodSeconds: 30
failureThreshold: 6
resources:
{}
As a data point, this works with metallb.
from akrobateo.
jnummelin
commented on June 24, 2024
Well, MetalLB does not use custom pods with iptables for routing so it works completely differently alltogether. :)
So the LB pods do look like expected with the init containers trying to set net.ipv4.ip_forward. Would you be able to run a pure test pod on the system and try out couple of things. So to test out how ip forwarding looks like, run a simple test pod that mimics the LB pod:
apiVersion: v1
kind: Pod
metadata:
name: ip-forward-test
labels:
name: ip-forward-test
spec:
containers:
- name: test
command:
- sh
- -c
- sleep 60000
image: registry.pharos.sh/kontenapharos/akrobateo-lb:latest
imagePullPolicy: Always
name: sysctl
resources: {}
securityContext:
privileged: true
Then exec yourself into the pod with kubectl exec -t -i ip-forward-test sh and run some tests:
/ $ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1
/ $ sysctl -w net.ipv4.ip_forward=1
/ $ echo $?
0
/ $ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1
/ $ cat /proc/sys/net/ipv4/ip_forward
1
from akrobateo.
dhoard
commented on June 24, 2024
Working correctly with a fresh install of Kubernetes v1.15.5.
from akrobateo.
Related Issues (17)
- Add automated builds
- Add deployment yamls
- Docs
- Renaming HOT 4
- Select between daemonset and deployment
- Container bins report wrong version
- Tolerations & node selectors
- LB POD Crashloop HOT 7
- Named ports not supported HOT 2
- Daemonset Not Deleted HOT 3
- default deployment gives rbac error
- service doesn't terminate on sigterm HOT 3
- Release 0.1.1 HOT 1
- [FEATURE] Have the ability to disable the init container (which requires privileged permissions) HOT 1
- akrobateo-lb with single External-IP HOT 1
- Akrobateo registry (https://registry.pharos.sh) is out of service making it non-operable HOT 31
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from akrobateo.