You can create multiple virtual networks to isolate different portions of your organization, for instance, development, marketing, and production.
DC/OS uses iptables to set up virtual network isolation. iptables are a high-speed, built-in mechanism for filtering traffic in Linux systems. We recommend configuring filtering by deploying a homogenous set of rules to all nodes in your infrastructure. To simplify this, we also recommend using the ipset feature of iptables.
Important: These commands should be run on all cluster nodes.
Set up your own chain that jumps from the
FORWARD chain. You can do this by running the following command:
iptables -N dcos-isolation
Now set up a default deny or a default accept policy between filtered overlays.
To set up default deny, run the following:
iptables -A dcos-isolation -j REJECT
To set up default accept, run the following:
iptables -A dcos-isolation -j RETURN
To make troubleshooting easier, use the
REJECT directive as opposed to the
DROP directive. The default is to allow all.
Use ipset to get onto the isolation chain. Create a
hash:net type ipset named
overlays that has all of the virtual networks that you want to restrict traffic from, or to. Then insert the rule:
iptables -I FORWARD -m set --match-set overlays src -m set --match-set overlays dst -j dcos-isolation
This rule says that if a given packet is from any of the overlays and is destined to any other overlay, send it to the
dcos-isolation rule. In most environments, the system does not prevent an virtual network’s outbound packets from reentering the same virtual network. To prevent this, add an exception set of type
hash:net,net and add entries for networks that should not be filtered. Modify the rule to:
iptables -I FORWARD -m set --match-set overlays src -m set --match-set overlays dst -m set ! --match-set src,dst overlay-exceptions -j dcos-isolation
The actual iptables rules that live on the
dcos-isolation chain are simple rules. For organization, use ipsets of type
hash:net and refer to
src sets and
Note: Future versions of DC/OS may automatically create the overlay ipsets. Network names prefixed with
mesos- are therefore reserved and should not be used.
In this example, the user has created two virtual networks, “IT” and “HR”, and wants isolation according to the following rules:
- HR apps can connect to IT apps.
- IT apps cannot connect to HR apps.
- All IT apps can communicate amongst themselves.
- All HR apps can communicate amongst themselves.
IT only runs apps on port 80. Assume an HR overlay with the agent subnets carved from
192.168.0.0/16 and an IT subnet carved from
First, create the sets you need:
iptables -N dcos-isolation iptables -A dcos-isolation -j REJECT # Changes it to default reject ipset create it hash:net ipset create hr hash:net ipset create overlays list:set
Next, define the subnets and policies:
ipset add it 10.250.0.0/16 ipset add hr 192.168.0.0/16 ipset create simple_allowed hash:net,net ipset create complex_allowed hash:net,port,net iptables -I FORWARD -m set --match-set overlays src -m set --match-set overlays dst -j dcos-isolation iptables -A dcos-isolation -m set --match-set simple_allowed src,dst -j RETURN
Then, allow traffic going from HR and allow bidirectional connections:
iptables -A dcos-isolation -m set --match-set complex_allowed src,dst,dst -j RETURN iptables -A dcos-isolation -m conntrack --ctstate RELATED,ESTABLISHED -j RETURN
Create hairpin exception rules:
iptables -I dcos-isolation -m set --match-set it src -m set --match-set it dst -j RETURN iptables -I dcos-isolation -m set --match-set hr src -m set --match-set hr dst -j RETURN ipset add simple_allowed 192.168.0.0./16,192.168.0.0./16 ipset add simple_allowed 10.250.0.0/16,10.250.0.0/16 ipset add complex_allowed 192.168.0.0/16,80,10.250.0.0/16 #this allows traffic from HR to IT on port 80
Debug with these commands:
iptables -L -v -n iptables -I dcos-isolation -j TRACE