hadoop's People
Forkers
baifendian typhoonzero yancey1989 wsxzhmzwdfxeds lupan2015 lapterchow anuragtomar hexiaofeng zbaolong view-sonic switchiihadoop's Issues
Is it possible to run HDFS in Docker?
RM can not find the host of hadoop-slave-xxx
I'm tring to run the example WordCount, but the app state alwasy ACCEPTED.
In RM log:
16/05/08 15:56:36 ERROR scheduler.SchedulerApplicationAttempt: Error trying to assign container token and NM token to an allocated container container_1462716844418_0001_01_000001
java.lang.IllegalArgumentException: java.net.UnknownHostException: hadoop-slave-qjzbu
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:374)
at org.apache.hadoop.yarn.server.utils.BuilderUtils.newContainerToken(BuilderUtils.java:256)
at org.apache.hadoop.yarn.server.resourcemanager.security.RMContainerTokenSecretManager.createContainerToken(RMContainerTokenSecretManager.java:220)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt.pullNewlyAllocatedContainersAndNMTokens(SchedulerApplicationAttempt.java:448)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.allocate(FairScheduler.java:921)
It looks like RM node can not connect the pod of NM node,such as hadoop-slave-xxxx.
So my question is :
I can use dns service with the component name such as hadoop-master, but when the hadoop-master want to connect the specific pod, such as hadoop-slave-xxx, it can not find the ip.
Is it possible for us to have Ceph on Kubernetes in addition to HDFS
with init script, pod can not connect the service.
Env:Vagrant + coreos * 2
K8s version: 1.2.0
CoreOS version: Linux version 4.6.0-coreos ([email protected]) (gcc version 4.9.3 (Gentoo Hardened 4.9.3 p1.3, pie-0.6.3) ) #2 SMP Fri Jun 3 23:44:22 UTC 2016
- svc info
yancey@ yancey-macbook k8s$kubectl get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hadoop-zookeeper 10.0.0.120 <none> 2181/TCP 1h
kubernetes 10.0.0.1 <none> 443/TCP 1h
- my pod can not telnet to the kubernets service
yancey@ yancey-macbook k8s$kubectl exec hadoop-zookeeper-x96rj telnet 10.0.0.1 443
telnet: can't connect to remote host (10.0.0.1): Connection refused
error: error executing remote command: Error executing command in container: Error executing in Docker Container: 1
- ifconfig && iptables info
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 10.1.4.1 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::42:bcff:fef3:35ab prefixlen 64 scopeid 0x20<link>
ether 02:42:bc:f3:35:ab txqueuelen 0 (Ethernet)
RX packets 159 bytes 12380 (12.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 21 bytes 1242 (1.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.2.15 netmask 255.255.255.0 broadcast 10.0.2.255
inet6 fe80::a00:27ff:fea7:5e9e prefixlen 64 scopeid 0x20<link>
ether 08:00:27:a7:5e:9e txqueuelen 1000 (Ethernet)
RX packets 97026 bytes 76268827 (72.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 45284 bytes 3009253 (2.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.17.8.102 netmask 255.255.255.0 broadcast 172.17.8.255
inet6 fe80::a00:27ff:feda:478f prefixlen 64 scopeid 0x20<link>
ether 08:00:27:da:47:8f txqueuelen 1000 (Ethernet)
RX packets 39909 bytes 4961258 (4.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 40458 bytes 4070934 (3.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 10.1.4.0 netmask 255.255.0.0 broadcast 0.0.0.0
inet6 fe80::b3:c3ff:fe13:4d94 prefixlen 64 scopeid 0x20<link>
ether 02:b3:c3:13:4d:94 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 8 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 2360 bytes 122960 (120.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2360 bytes 122960 (120.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
veth054ea6f: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet6 fe80::60c5:d4ff:fe2e:b234 prefixlen 64 scopeid 0x20<link>
ether 62:c5:d4:2e:b2:34 txqueuelen 0 (Ethernet)
RX packets 159 bytes 14606 (14.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 29 bytes 1890 (1.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
core-v2-02 init # iptables-save -t nat
...
-A KUBE-SEP-OGNOLD2JUSLFPOMZ -s 10.0.2.15/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ
-A KUBE-SEP-OGNOLD2JUSLFPOMZ -p tcp -m comment --comment "default/kubernetes:https" -m tcp -j DNAT --to-destination 10.0.2.15:6443
-A KUBE-SERVICES -d 10.0.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
-A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS
...
- I think the address in iptables 10.0.2.15 should be insted of eth1(172.17.8.102), but I can not find the solution.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.