5.配置slave并加入team
配置从属并将其添加到team。
例如:
# nmcli con add type team-slave con-name team0-slave0 ifname enp0s3 master team0 Connection 'team0-slave0' (XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX) successfully added. # nmcli con add type team-slave con-name team0-slave1 ifname enp0s8 master team0 Connection 'team0-slave1' (XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX) successfully added.
# nmcli conn NAME UUID TYPE DEVICE team0 XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX team team0 team0-slave0 XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX ethernet enp0s3 team0-slave1 XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX ethernet enp0s8
# nmcli dev DEVICE TYPE STATE CONNECTION team0 team connected team0 enp0s3 ethernet connected team0-slave0 enp0s8 ethernet connected team0-slave1 lo loopback unmanaged -
NetworkManager 创建以下组从接口配置文件:
# cat /etc/sysconfig/network-scripts/ifcfg-team0-slave0 1 NAME=team0-slave0 2 UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX 3 DEVICE=enp0s3 4 ONBOOT=yes 5 TEAM_MASTER=team0 6 DEVICETYPE=TeamPort
# cat /etc/sysconfig/network-scripts/ifcfg-team0-slave1 1 NAME=team0-slave1 2 UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX 3 DEVICE=enp0s8 4 ONBOOT=yes 5 TEAM_MASTER=team0 6 DEVICETYPE=TeamPort
当至少有一个从属设备被添加到组中时,接口 (team0) 就会出现并变得可访问。
# ip addr ... 2: enp0s3: [BROADCAST,MULTICAST,UP,LOWER_UP] mtu 1500 qdisc fq_codel master team0 state UP group default qlen 1000 link/ether [MAC1] brd ff:ff:ff:ff:ff:ff 3: enp0s8: [BROADCAST,MULTICAST,UP,LOWER_UP] mtu 1500 qdisc fq_codel master team0 state UP group default qlen 1000 link/ether [MAC1] brd ff:ff:ff:ff:ff:ff 6: team0: [BROADCAST,MULTICAST,UP,LOWER_UP] mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether [MAC1] brd ff:ff:ff:ff:ff:ff inet [IP3]/24 brd [IP6] scope global dynamic noprefixroute team0 valid_lft 86045sec preferred_lft 86045sec inet6 fe80::5b1f:554a:1928:8575/64 scope link noprefixroute valid_lft forever preferred_lft forever
请注意,所有组/组接口都使用相同的 MAC 地址,例如:组中第一个组从成员的 MAC 地址。
2. 删除现有的网络连接
删除现有的 enp0s3 和 enp0s8 连接,如下所示。
这些将在以下步骤中重新创建为团队从属。
# nmcli con show NAME UUID TYPE DEVICE enp0s3 XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX ethernet enp0s3 enp0s8 XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX ethernet enp0s8
# nmcli con del XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX Connection 'enp0s3' (XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX) successfully deleted. # nmcli con del XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX Connection 'enp0s8' (XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX) successfully deleted.
# nmcli dev DEVICE TYPE STATE CONNECTION enp0s3 ethernet disconnected - enp0s8 ethernet disconnected - lo loopback unmanaged -
# ls -l /etc/sysconfig/network-scripts/ total 0 #
8. 为网络接口启用混杂模式
某些虚拟化技术(例如 Oracle VM VirtualBox)需要在分配给来宾的网络接口上以及来宾内启用混杂模式,以便从属故障转移/故障回复正常运行。
可能还需要在物理系统上的网络接口上启用混杂模式。
对于 Oracle VM VirtualBox,可以为来宾接口启用混杂模式,如下所示:
Oracle VM VirtualBox Manager > [GUEST] > 设置 > 网络 > 适配器 1|... ] 高级] 混杂模式:全部允许
可以使用自定义服务在 CentOS/RHEL 7 系统中动态和静态启用混杂模式,如下所示:
动态的、非持久的:
# ip link set enp0s3 promisc on # ip link set enp0s8 promisc on
静态的,持久的:
创建一个包含以下内容的自定义 systemd 单元文件,例如:
# cat /usr/lib/systemd/system/promiscuous.service 1 [Unit] 2 Description=Bring up network interfaces in promiscuous mode upon boot 3 After=network.target 4 5 [Service] 6 Type=oneshot 7 ExecStart=/usr/sbin/ip link set dev enp0s3 promisc on 8 ExecStart=/usr/sbin/ip link set dev enp0s8 promisc on 9 ExecStop=/usr/sbin/ip link set dev enp0s3 promisc off 10 ExecStop=/usr/sbin/ip link set dev enp0s8 promisc off 11 TimeoutStartSec=0 12 RemainAfterExit=yes 13 14 [Install] 15 WantedBy=default.target
通知 systemd 新服务。
例如:
# systemctl daemon-reload
启用并启动新的服务/单元,例如:
# systemctl enable promiscuous Created symlink /etc/systemd/system/default.target.wants/promiscuous.service → /usr/lib/systemd/system/promiscuous.service.
# systemctl start promiscuous
# systemctl status promiscuous ● promiscuous.service - Bring up network interfaces in promiscuous mode upon boot Loaded: loaded (/usr/lib/systemd/system/promiscuous.service; enabled; vendor preset: disabled) Active: active (exited) since Fri 2020-08-21 16:14:53 AEST; 17s ago Process: 8088 ExecStart=/usr/sbin/ip link set dev enp0s8 promisc on (code=exited, status=0/SUCCESS) Process: 8086 ExecStart=/usr/sbin/ip link set dev enp0s3 promisc on (code=exited, status=0/SUCCESS) Main PID: 8088 (code=exited, status=0/SUCCESS) Aug 21 16:14:53 [HOST] systemd[1]: Starting Bring up network interfaces in promiscuous mode upon boot... Aug 21 16:14:53 [HOST] systemd[1]: Started Bring up network interfaces in promiscuous mode upon boot.
验证在所有从接口上启用了混杂模式。
例如:
# ip addr | grep enp 2: enp0s3: [BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP] mtu 1500 qdisc fq_codel master team0 state UP group default qlen 1000 3: enp0s8: [BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP] mtu 1500 qdisc fq_codel master team0 state UP group default qlen 1000
9. 测试team网络连接弹性 - 从属故障转移/故障回复
A. 从远程客户端,启动已配置网络绑定的服务器的连续 ping(8),例如:
[CLIENT]$ ping [SERVER] PING [IP3] ([IP3]) 56(84) bytes of data. 64 bytes from [IP3]: icmp_seq=1 ttl=64 time=0.025 ms 64 bytes from [IP3]: icmp_seq=2 ttl=64 time=0.034 ms 64 bytes from [IP3]: icmp_seq=3 ttl=64 time=0.039 ms ...
B. 断开当前活动的team从设备。
暂时断开连接到从接口的物理网线,例如enp0s3.
请注意,以编程方式禁用接口不会模拟实际的物理链路连接丢失。
# teamdctl team0 state setup: runner: activebackup ports: enp0s3 link watches: link summary: down instance[link_watch_0]: name: ethtool link: down down count: 1 enp0s8 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up down count: 0 runner: active port: enp0s8
一旦断开连接,teamdctl 表示从接口 enp0s3 为关闭,而以前的备份从接口 enp0s8 现在是当前活动的从接口。
尽管从属故障转移,来自远程客户端的 ping(8) 继续不间断。
运行 ip(8) 命令进一步确认到team的所有网络流量都是通过新的活动从设备 enp0s8 发生的,即:
# ip -s link ... 2: enp0s3: [BROADCAST,MULTICAST,PROMISC] mtu 1500 qdisc fq_codel master team0 state DOWN mode DEFAULT group default qlen 1000 link/ether [MAC1] brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped overrun mcast 3007606561 5014092 0 27008 0 91263 TX: bytes packets errors dropped carrier collsns 787749 7568 0 0 0 0 3: enp0s8: [BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP] mtu 1500 qdisc fq_codel master team0 state UP mode DEFAULT group default qlen 1000 link/ether [MAC1] brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped overrun mcast 3829373093 6116137 0 26680 0 111948 TX: bytes packets errors dropped carrier collsns 179163 1337 0 0 0 0 7: team0: [BROADCAST,MULTICAST,UP,LOWER_UP] mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether [MAC1] brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped overrun mcast 2106463511 1969187 0 49615 0 60559 TX: bytes packets errors dropped carrier collsns 176341 1378 0 0 0 0
C. 断开新的当前活动team从属设备。
暂时断开连接到新提升的活动从接口的物理网线,例如enp0s8.
一旦断开连接,teamdctl 将两个从接口 enp0s3 和 enp0s8 表示为关闭,即:
# teamdctl team0 state setup: runner: activebackup ports: enp0s3 link watches: link summary: down instance[link_watch_0]: name: ethtool link: down down count: 1 enp0s8 link watches: link summary: down instance[link_watch_0]: name: ethtool link: up down count: 1 runner: active port:
此时,在两个团队从属设备都断开连接的情况下,来自远程客户端的 ping(8) 停止,例如
... 64 bytes from [IP3]: icmp_seq=1253 ttl=64 time=0.207 ms 64 bytes from [IP3]: icmp_seq=1254 ttl=64 time=0.131 ms 64 bytes from [IP3]: icmp_seq=1255 ttl=64 time=0.227 ms 64 bytes from [IP3]: icmp_seq=1256 ttl=64 time=0.218 ms 64 bytes from [IP3]: icmp_seq=1257 ttl=64 time=0.198 ms From [IP3] icmp_seq=1258 Destination Host Unreachable From [IP3] icmp_seq=1259 Destination Host Unreachable ...
D. 重新连接断开的从接口
将物理网线重新连接到从属接口之一,例如enp0s3.
重新连接后,teamdctl 表示从接口 enp0s3 为 up,即:
# teamdctl team0 state setup: runner: activebackup ports: enp0s3 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up down count: 1 enp0s8 link watches: link summary: down instance[link_watch_0]: name: ethtool link: down down count: 1 runner: active port: enp0s3
此时,使用 enp0s3 当前活动的从机,来自远程客户端的 ping(8) 恢复,例如
... From [IP3] icmp_seq=1392 Destination Host Unreachable From [IP3] icmp_seq=1393 Destination Host Unreachable From [IP3] icmp_seq=1394 Destination Host Unreachable From [IP3] icmp_seq=1395 Destination Host Unreachable 64 bytes from [IP3]: icmp_seq=1396 ttl=64 time=1258180 ms 64 bytes from [IP3]: icmp_seq=1397 ttl=64 time=1257180 ms 64 bytes from [IP3]: icmp_seq=1398 ttl=64 time=1256181 ms 64 bytes from [IP3]: icmp_seq=1399 ttl=64 time=1255181 ms ...
E. 重新连接剩余的断开连接的从接口。
将物理网线重新连接到剩余的从接口,例如enp0s8.
一旦重新连接,teamdctl 将剩余的从接口 enp0s8 表示为 up,即:
# teamdctl team0 state setup: runner: activebackup ports: enp0s3 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up down count: 1 enp0s8 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up down count: 1 runner: active port: enp0s3
1. 配置前的配置
以下是配置network teaming前的配置:
# dnf list | grep team | grep anaconda NetworkManager-team.x86_64 1:1.22.8-4.el8 @anaconda libteam.x86_64 1.29-1.el8 @anaconda teamd.x86_64 1.29-1.el8 @anaconda
# lspci | grep -i eth 00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02) 00:08.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)
# nmcli dev DEVICE TYPE STATE CONNECTION enp0s3 ethernet connected enp0s3 enp0s8 ethernet connected enp0s8 lo loopback unmanaged -
# nmcli con NAME UUID TYPE DEVICE enp0s3 XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX ethernet enp0s3 enp0s8 XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX ethernet enp0s8
# ls -l /etc/sysconfig/network-scripts/* -rw-r--r-- 1 root root 334 Aug 21 13:29 /etc/sysconfig/network-scripts/ifcfg-enp0s3 -rw-r--r-- 1 root root 334 Aug 21 13:30 /etc/sysconfig/network-scripts/ifcfg-enp0s8
# cat /etc/sysconfig/network-scripts/ifcfg-enp0s3 1 MACADDR=[MAC_ADDR1] 2 MTU=1500 3 TYPE=Ethernet 4 PROXY_METHOD=none 5 BROWSER_ONLY=no 6 BOOTPROTO=dhcp 7 DEFROUTE=yes 8 IPV4_FAILURE_FATAL=no 9 IPV6INIT=yes 10 IPV6_AUTOCONF=yes 11 IPV6_DEFROUTE=yes 12 IPV6_FAILURE_FATAL=no 13 IPV6_ADDR_GEN_MODE=stable-privacy 14 NAME="enp0s3" 15 UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX 16 DEVICE=enp0s3 17 ONBOOT=yes
# cat /etc/sysconfig/network-scripts/ifcfg-enp0s8 1 MACADDR=[MAC_ADDR2] 2 MTU=1500 3 TYPE=Ethernet 4 PROXY_METHOD=none 5 BROWSER_ONLY=no 6 BOOTPROTO=dhcp 7 DEFROUTE=yes 8 IPV4_FAILURE_FATAL=no 9 IPV6INIT=yes 10 IPV6_AUTOCONF=yes 11 IPV6_DEFROUTE=yes 12 IPV6_FAILURE_FATAL=no 13 IPV6_ADDR_GEN_MODE=stable-privacy 14 NAME="enp0s8" 15 UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX 16 DEVICE=enp0s8 17 ONBOOT=yes
# ip addr ... 2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether <MAC1> brd ff:ff:ff:ff:ff:ff inet <IP1>/24 brd <IP6> scope global dynamic noprefixroute enp0s3 valid_lft 86059sec preferred_lft 86059sec inet6 fe80::ca99:46d3:1765:f02b/64 scope link noprefixroute valid_lft forever preferred_lft forever 3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether <MAC2> brd ff:ff:ff:ff:ff:ff inet <IP2>/24 brd <IP6> scope global dynamic noprefixroute enp0s8 valid_lft 86121sec preferred_lft 86121sec inet6 fe80::36d0:6bd3:5152:83dc/64 scope link noprefixroute valid_lft forever preferred_lft forever
4. 可选择为team分配静态 IP 地址
可选择为team连接分配静态 IP 地址、网关、DNS 等。
例如:
# nmcli con mod team0 ipv4.addresses [IP3]/24 # nmcli con mod team0 ipv4.gateway [IP4] # nmcli con mod team0 ipv4.dns [IP5] # nmcli con mod team0 ipv4.method bananaal # nmcli con mod team0 connection.autoconnect yes
NetworkManager 修改如下组接口配置文件:
# cat /etc/sysconfig/network-scripts/ifcfg-team0 1 TEAM_CONFIG="{\"runner\": {\"name\": \"activebackup\"}, \"link_watch\": {\"name\": \"ethtool\"}}" 2 PROXY_METHOD=none 3 BROWSER_ONLY=no 4 BOOTPROTO=none 5 DEFROUTE=yes 6 IPV4_FAILURE_FATAL=no 7 IPV6INIT=yes 8 IPV6_AUTOCONF=yes 9 IPV6_DEFROUTE=yes 10 IPV6_FAILURE_FATAL=no 11 IPV6_ADDR_GEN_MODE=stable-privacy 12 NAME=team0 13 UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX 14 DEVICE=team0 15 ONBOOT=yes 16 DEVICETYPE=Team 17 IPADDR=[IP3] 18 PREFIX=24 19 GATEWAY=[IP4] 20 DNS1=[IP5]
如果没有分配静态 IP 地址,team将使用 DHCP。
7. 识别当前活动/非活动从接口
使用 teamdctl(8) 识别当前活动和非活动从属接口。
例如:
# teamdctl team0 state setup: runner: activebackup ports: enp0s3 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up down count: 0 enp0s8 link watches: link summary: up instance[link_watch_0]: name: ethtool link: up down count: 0 runner: active port: enp0s3
网络接口组合(Network interface teaming)是从 CentOS/RHEL 7 引入的,作为网络绑定( network bonding)的更具扩展性的替代方案。
本文说明了如何在 CentOS/RHEL 7/8 上配置网络绑定(network teaming)。
提供的示例基于使用 NetworkManager 的具有两个网络接口的 Oracle Linux 8.2 系统(Oracle VirtualBox 6.1 来宾虚拟机)。
具体来说,网络管理器命令行界面 (nmcli) 主要与以缩写/缩写形式指定的方法一起使用。
3.创建network teaming连接
创建network teaming连接。
例如:
# nmcli con add type team con-name team0 ifname team0 config '{"runner": {"name": "activebackup"}, "link_watch": {"name": "ethtool"}}' Connection 'team0' (XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX) successfully added.
上面,team运行模式(runner)是activebackup,团队链接监视器/观察者(link_watch)是ethtool。
替代的 runner 和 link_watch 值包括:
- runner : 负载平衡、循环、lacp、广播、随机。
- link_watch : arp_ping, nsna_ping
NetworkManager 创建以下接口配置文件:
# cat /etc/sysconfig/network-scripts/ifcfg-team0 1 TEAM_CONFIG="{\"runner\": {\"name\": \"activebackup\"}, \"link_watch\": {\"name\": \"ethtool\"}}" 2 PROXY_METHOD=none 3 BROWSER_ONLY=no 4 BOOTPROTO=dhcp 5 DEFROUTE=yes 6 IPV4_FAILURE_FATAL=no 7 IPV6INIT=yes 8 IPV6_AUTOCONF=yes 9 IPV6_DEFROUTE=yes 10 IPV6_FAILURE_FATAL=no 11 IPV6_ADDR_GEN_MODE=stable-privacy 12 NAME=team0 13 UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX 14 DEVICE=team0 15 ONBOOT=yes 16 DEVICETYPE=Team
# nmcli con NAME UUID TYPE DEVICE team0 XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX team team0
# nmcli dev DEVICE TYPE STATE CONNECTION team0 team connecting (getting IP configuration) team0 enp0s3 ethernet disconnected - enp0s8 ethernet disconnected - lo loopback unmanaged
6. 重启team
重启team使静态IP地址等生效。
例如:
# nmcli con down team0 && nmcli con up team0