Comments (2)
I've been looking in details at your plugins and it looks really great.
Thanks! Sorry for the lag in following up. Replying to each section:
the plugin is relying on L2 mode (efficient and simple) but would probably work great with L3 mode (even if it will require some ip rules to force traffic through the appropriate ENI). Is there a particular reason for this choice?
When requesting a secondary private IP address on an ENI, the only guarantee is that the IP will be within the subnet assigned to the ENI. All traffic that leaves the ENI must have the same MAC address. There's no ability to assign a contiguous / routable block of IP addresses suitable for using l3 mode on an ENI, unless you dedicate an entire VPC subnet to a single ENI. If you do that, you're going to quickly run out of subnets.
traffic not going to the VPC CIDR block is masqueraded on the main interface. Is that necessary?
Having the pod IP address in VPC flowlogs could be interesting and it could also be interesting over VPC peerings (of course that would involve NAT to access the internet)
Traffic not going to the VPC CIDR block is routed back over the default namespace so we can make use of Amazon’s Public IPv4 addressing attribute feature when egressing over the primary private IP of the boot ENI. I'll file a bug to add VPC peering routes to the Pod route table so we'll use the IPvlan interface for that traffic. That should be a simple change.
In addition, it would be great if different ENIs could have different security groups and pods could be assigned to ENIs based on security requirements (using annotations for instance)
We use security groups to segment the control plane, which always runs on the boot ENI, from Pods running on non-boot ENIs. We don't currently have a use-case for scheduling against different ENIs, although we wouldn't be opposed to a PR if there's a straightforward way to handle that case. We're planning to add NetworkPolicy support to our CNI stack using netfilter rules and let the kernel enforce restrictions between Pods instead of relying on security groups to control Pod to Pod comms.
Let us know if you have any other questions.
from cni-ipvlan-vpc-k8s.
Sorry, I also lagged to answer back:
- I saw the new feature for peered VPC, this is great!
- I get your point for public IPs. It could be interesting to have a flag to choose if the default route is NATted over the primary interface or routed to the pod ENI (for setups where all nodes only have private IP addresses). If you're opened to it, I could work on a PR (I don't think I will have time in the coming days, but I could work on it in a few weeks)
- For L3/L2, I actually got IPVLAN to work in L3 mode with IP in the same subnet. I'm not exactly sure of differences between L2 and L3 in that case but I will dive into it. Here is an example that works for me:
sudo ip link add link eth1 name ipvltest1 type ipvlan mode l3
sudo ip link add link eth1 name ipvltest2 type ipvlan mode l3
sudo ip link set dev ipvltest1 netns test1
sudo ip link set dev ipvltest2 netns test2
sudo ip netns exec test1 ip addr add 172.29.0.100/24 dev ipvltest1
sudo ip netns exec test1 ip link set ipvltest1 up
sudo ip netns exec test1 ip route add default via 172.29.0.1
sudo ip netns exec test2 ip addr add 172.29.0.101/24 dev ipvltest2
sudo ip netns exec test2 ip link set ipvltest2 up
sudo ip netns exec test2 ip route add default via 172.29.0.1
echo "1 eni1" |sudo tee -a /etc/iproute2/rt_tables
sudo ip route add 172.29.0.1/32 scope link dev eth1 table eni1
sudo ip route add default via 172.29.0.1 dev eth1 table eni1
sudo ip rule add from 172.29.0.100/32 table eni1
sudo ip rule add from 172.29.0.101/32 table eni1
Where test1 and test2 are two different container namespaces and 172.29.0.100/172.29.0.101 IP addresses associated with the secondary ENI (eth1).
This setup is very similar to a L2 mode, but requires the addition of the IP rule to force traffic from container namespaces through the additional ENI. The traffic between namespaces works even if it not routed by the IPVLAN primary interface because both sub-interfaces are in the same subnet.
from cni-ipvlan-vpc-k8s.
Related Issues (20)
- Clarify configuration and setup instructions
- The CNI plugin only works on OS where interfaces are called ethX
- Additional interfaces could be in the same subnet HOT 8
- Plugin does not work with containerd runtime HOT 6
- missing routes on first pod for an interface HOT 3
- Nodeports not working properly HOT 8
- Incorrect handling of errors in IPAM HOT 1
- Inconsistency between ENI allocated IPs and OS configuration HOT 16
- CNI plugin: `portmap` when chained with cni-ipvlan-vpc-k8s does not work
- Listing VPC CIDRs to create routes in pods does not filter on association state
- New release HOT 1
- Disabling ipMasq
- Cannot connect to pod outside VPC HOT 4
- Pods loosing connection to each other on particular node from time to time HOT 13
- Build binaries for 0.5.0 release HOT 1
- aws-vpc-cni-k8s vs lyft cni HOT 3
- IP Batching changes the default behavior HOT 1
- Brother, I compile for a day. Compilation is unsuccessful. Please find a compiled package.
- Binary Release for 0.6.1 HOT 3
- support for containerd
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cni-ipvlan-vpc-k8s.