FRR学习第九天--完整的数据中心网络模型

网络拓扑

拓扑说明

实验环境是一台16G内存的主机。上面使用vmware运行了三个虚拟机，运行的系统为ubuntu-19.04。三个虚拟机采用host-only模式连接。

• spine，leaf1，leaf2三个设备均为ubuntu-19.04.上面运行了FRR程序。
• host1，host2，host3，host4为网络命名空间。
• underlay网络采用的是二层模式(局限于实验条件)

整个实验是一个比例缩小的数据中心spine-leaf模型。leaf2还需要作为边界网关，使用默认路由将流量发送到公网，同时作为firewall(这里只进行了nat)。

leaf作为border和vtep的功能细化图

实验功能说明

• 整个数据中心一个租户，使用vni：100作为租户的l3vni
• 租户使用了三个subnet。1.1.1.0/24子网有两个虚机，分布在两个vtep下，使用10作为l2vni。2.2.2.0/24和3.3.3.0/24都只有一个虚机。5.5.5.0/24作为relay子网用于连接default-vrf和evpn-vrf。
• 整个实验需要实现租户内所有主机互通，同时主机可以访问公网。(暂时不能实现公网访问虚机，需要申请floating-ip才可以，申请了公网IP后，可以在default-vrf中做1:1 nat实现互访)

spine配置

bgp evpn配置

router bgp 7677
 bgp router-id 192.168.59.130
 bgp bestpath as-path multipath-relax
 neighbor fabric peer-group
 neighbor fabric remote-as external
 neighbor 192.168.59.128 peer-group fabric
 neighbor 192.168.59.129 peer-group fabric
 !
 address-family l2vpn evpn
  neighbor fabric activate
 exit-address-family
!

leaf1配置

接口配置

#开启转发
sudo sysctl -w net.ipv4.ip_forward=1  
sudo sysctl -p

#添加host1
sudo ip netns add host1
sudo ip link add veth1 type veth peer name eth0 netns host1
sudo ip netns exec host1 ip link set lo up
sudo ip netns exec host1 ip link set eth0 up
sudo ip netns exec host1 ip addr add 1.1.1.1/24 dev eth0
sudo ip netns exec host1 ip route add default via 1.1.1.254 dev eth0

sudo ip link add br10 type bridge
sudo ip link add vxlan10 type vxlan id 10 local 192.168.59.128 dstport 4789 nolearning
sudo ip link set br10 up
sudo ip link set veth1 up
sudo ip link set vxlan10 up
sudo ip link set veth1 master br10
sudo ip link set vxlan10 master br10
sudo ip addr add 1.1.1.254/24 dev br10
sudo ip link set dev br10 address 00:00:01:02:03:10 #分布式二层网关，mac需要一致

#添加host2
sudo ip netns add host2
sudo ip link add veth2 type veth peer name eth0 netns host2
sudo ip netns exec host2 ip link set lo up
sudo ip netns exec host2 ip link set eth0 up
sudo ip netns exec host2 ip addr add 2.2.2.2/24 dev eth0
sudo ip netns exec host2 ip route add default via 2.2.2.254 dev eth0

sudo ip link add br20 type bridge
sudo ip link set br20 up
sudo ip link set veth2 up
sudo ip link set veth2 master br20
sudo ip addr add 2.2.2.254/24 dev br20

#添加vni 100，作为l3vni
sudo ip link add br100 type bridge
sudo ip link add vxlan100 type vxlan id 100 local 192.168.59.128 dstport 4789 nolearning
sudo ip link set br100 up
sudo ip link set vxlan100 up
sudo ip link set vxlan100 master br100  
#sudo ip addr add 5.5.5.254/24 dev br100 切记，作为l3vni的svi接口不能配置IP，否则收到type-5路由不会安装。
sudo ip link set dev br100 address 00:00:01:02:03:04 #这个是路由mac

#添加vrf
sudo ip link add evpn-vrf type vrf table 100
sudo ip link set evpn-vrf up
sudo ip link set br100 master evpn-vrf  
sudo ip link set br10 master evpn-vrf 
sudo ip link set br20 master evpn-vrf 

bgp evpn配置

vrf evpn-vrf
 vni 100
 exit-vrf
!
router bgp 7675
 bgp router-id 192.168.59.128
 bgp bestpath as-path multipath-relax
 neighbor fabric peer-group
 neighbor fabric remote-as external
 neighbor 192.168.59.130 peer-group fabric
 !
 address-family l2vpn evpn
  neighbor fabric activate
  advertise-all-vni
 exit-address-family
!
router bgp 7675 vrf evpn-vrf
 !
 address-family ipv4 unicast
  network 2.2.2.0/24
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!

注：

vrf evpn-vrf
 vni 100
 exit-vrf

这段指令表示指定了一个l3vni

router bgp 7675 vrf evpn-vrf
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!

这段指令advertise ipv4 unicast表示宣告RT-5路由。

切记：一定不能为l3vni对应的svi添加IP地址，否则type5路由不能正确下内核。

leaf2配置

接口配置

#开启转发
sudo sysctl -w net.ipv4.ip_forward=1  
sudo sysctl -p

#添加host3
sudo ip netns add host3
sudo ip link add veth3 type veth peer name eth0 netns host3
sudo ip netns exec host3 ip link set lo up
sudo ip netns exec host3 ip link set eth0 up
sudo ip netns exec host3 ip addr add 3.3.3.3/24 dev eth0
sudo ip netns exec host3 ip route add default via 3.3.3.254 dev eth0 

# 添加网桥，将veth3加入网桥
sudo ip link add br30 type bridge
sudo ip link set br30 up
sudo ip link set veth3 up
sudo ip link set veth3 master br30
sudo ip addr add 3.3.3.254/24 dev br30

#添加host4
sudo ip netns add host4
sudo ip link add veth4 type veth peer name eth0 netns host4
sudo ip netns exec host4 ip link set lo up
sudo ip netns exec host4 ip link set eth0 up
sudo ip netns exec host4 ip addr add 1.1.1.2/24 dev eth0
sudo ip netns exec host4 ip route add default via 1.1.1.254 dev eth0

sudo ip link add br40 type bridge
sudo ip link add vxlan10 type vxlan id 10 local 192.168.59.129 dstport 4789 nolearning
sudo ip link set vxlan10 up
sudo ip link set vxlan10 master br40
sudo ip link set br40 up
sudo ip link set veth4 up
sudo ip link set veth4 master br40
sudo ip addr add 1.1.1.254/24 dev br40
sudo ip link set dev br40 address 00:00:01:02:03:10 #分布式二层网关，mac需要一致

#添加vni 100，作为l3vni
sudo ip link add br100 type bridge
sudo ip link add vxlan100 type vxlan id 100 local 192.168.59.129 dstport 4789 nolearning
sudo ip link set br100 up
sudo ip link set vxlan100 up
sudo ip link set vxlan100 master br100  
#sudo ip addr add 5.5.5.253/24 dev br100 切记一定不能添加IP地址，否则type5路由不能正确下内核
sudo ip link set dev br100 address 00:00:01:02:03:05  #这个是rmac，即路由mac

#添加vrf
sudo ip link add evpn-vrf type vrf table 100
sudo ip link set evpn-vrf up
sudo ip link set br100 master evpn-vrf  
sudo ip link set br30 master evpn-vrf 
sudo ip link set br40 master evpn-vrf 

#访问外网

#添加连接evpn-vrf到默认vrf的vtep接口
sudo ip link add ext1 type veth peer name ext
sudo ip link set ext1 up
sudo ip link set ext up
#其中ext1在evpn-vrf，ext在default
sudo ip link set ext1 master evpn-vrf
#使用网段5.5.5.0/24作为relay网段
sudo ip addr add 5.5.5.253/24 dev ext1
sudo ip addr add 5.5.5.254/24 dev ext

#在evpn中加入默认路由，默认让流量访问公网,这个网段所有租户共用，由管理员负责分配，不能冲突
sudo ip route add default via 5.5.5.254 dev ext1 table 100

#配置snat，让私网流量改smac后范文公网
sudo nft add table nat
sudo nft add chain nat prerouting { type nat hook prerouting priority 0 \; }
sudo nft add chain nat postrouting { type nat hook postrouting priority 100 \; }
sudo nft add rule nat postrouting oifname ext1  counter masquerade
sudo nft add rule nat postrouting oifname ens33  counter masquerade

bgp evpn配置

vrf evpn-vrf
 vni 100
 exit-vrf
!
router bgp 7676
 bgp router-id 192.168.59.129
 bgp bestpath as-path multipath-relax
 neighbor fabric peer-group
 neighbor fabric remote-as external
 neighbor 192.168.59.130 peer-group fabric
 !
 address-family l2vpn evpn
  neighbor fabric activate
  advertise-all-vni
 exit-address-family
!
router bgp 7676 vrf evpn-vrf
 !
 address-family ipv4 unicast
  network 3.3.3.0/24
  network 0.0.0.0/0
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!

查看bgp信息

leaf1

• 查看路由信息

ubuntu# show bgp l2vpn evpn 
BGP table version is 7, local router ID is 192.168.59.128
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: ip 2.2.2.254:2

*> [5]:[0]:[24]:[2.2.2.0]
                    192.168.59.128           0         32768 i
Route Distinguisher: ip 5.5.5.253:2

*> [5]:[0]:[0]:[0.0.0.0]
                    192.168.59.129                         0 7677 7676 i
*> [5]:[0]:[24]:[3.3.3.0]
                    192.168.59.129                         0 7677 7676 i
Route Distinguisher: ip 192.168.59.128:3

*> [2]:[0]:[48]:[46:48:a2:5e:e2:2f]
                    192.168.59.128                     32768 i
*> [2]:[0]:[48]:[46:48:a2:5e:e2:2f]:[32]:[1.1.1.1]
                    192.168.59.128                     32768 i
*> [3]:[0]:[32]:[192.168.59.128]
                    192.168.59.128                     32768 i
Route Distinguisher: ip 192.168.59.129:3

*> [3]:[0]:[32]:[192.168.59.129]
                    192.168.59.129                         0 7677 7676 i

Displayed 7 out of 7 total prefixes
ubuntu# 

leaf2

• 查看路由信息

ubuntu# show bgp l2vpn evpn 
BGP table version is 9, local router ID is 192.168.59.129
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: ip 2.2.2.254:2

*> [5]:[0]:[24]:[2.2.2.0]
                    192.168.59.128                         0 7677 7675 i
Route Distinguisher: ip 5.5.5.253:2

*> [5]:[0]:[0]:[0.0.0.0]
                    192.168.59.129           0         32768 i
*> [5]:[0]:[24]:[3.3.3.0]
                    192.168.59.129           0         32768 i
Route Distinguisher: ip 192.168.59.128:3

*> [2]:[0]:[48]:[46:48:a2:5e:e2:2f]
                    192.168.59.128                         0 7677 7675 i
*> [2]:[0]:[48]:[46:48:a2:5e:e2:2f]:[32]:[1.1.1.1]
                    192.168.59.128                         0 7677 7675 i
*> [3]:[0]:[32]:[192.168.59.128]
                    192.168.59.128                         0 7677 7675 i
Route Distinguisher: ip 192.168.59.129:3

*> [3]:[0]:[32]:[192.168.59.129]
                    192.168.59.129                     32768 i

Displayed 7 out of 7 total prefixes
ubuntu#