k8s service-node-port-range ip_local_reserved_ports 记录

k8s ServiceNodePortRange 是什么

在 Kubernetes 中，ServiceNodePortRange 是一个用于指定 NodePort 服务端口范围的参数。该参数定义了可以分配给 NodePort 服务的端口范围。默认情况下，NodePort 服务使用的端口范围是 30000 到 32767。

如何查看 k8s ServiceNodePortRange

k8s master 节点上的 kube-apiserver 进程启动时，会指定参数 --service-node-port-range=xxx-xxx，该参数的值就是 ServiceNodePortRange 的值

使用 net.ipv4.ip_local_reserved_ports 配置保留端口范围

文档

ip_local_reserved_ports - list of comma separated ranges
    Specify the ports which are reserved for known third-party
    applications. These ports will not be used by automatic port
    assignments (e.g. when calling connect() or bind() with port
    number 0). Explicit port allocation behavior is unchanged.

    The format used for both input and output is a comma separated
    list of ranges (e.g. "1,2-4,10-10" for ports 1, 2, 3, 4 and
    10). Writing to the file will clear all previously reserved
    ports and update the current list with the one given in the
    input.

    Note that ip_local_port_range and ip_local_reserved_ports
    settings are independent and both are considered by the kernel
    when determining which ports are available for automatic port
    assignments.

    You can reserve ports which are not in the current
    ip_local_port_range, e.g.:

    $ cat /proc/sys/net/ipv4/ip_local_port_range
    32000    60999
    $ cat /proc/sys/net/ipv4/ip_local_reserved_ports
    8080,9148

    although this is redundant. However such a setting is useful
    if later the port range is changed to a value that will
    include the reserved ports.

    Default: Empty

ref: https://www.kernel.org/doc/Documentation/networking/ip-sysctl...

net.ipv4.ip_local_reserved_ports 是 Linux 内核参数，用于指定保留的本地端口范围，这些端口不会被随机分配给普通用户程序

防止冲突
防止普通用户程序占用 NodePort 服务端口范围，导致 NodePort 服务无法正常使用

net.ipv4.ip_local_reserved_ports="30000–32768"

这么写在我的系统上会报错 sysctl: setting key "net.ipv4.ip_local_reserved_ports": Invalid argument

这么写是可以的

sysctl -w net.ipv4.ip_local_reserved_ports=31000,32222,30080

修改NodePort端口范围时必须十分谨慎。务必保证NodePort端口范围与集群节点上Linux内核提供的net.ipv4.ip_local_port_range参数中的端口范围不冲突。该内核参数ip_local_port_range控制了Linux系统上任意应用程序可以使用的本地端口号范围。ip_local_port_range的默认值为32768~60999。

查看已经使用的 NodePort

kubectl get svc -A -o jsonpath='{range .items[*]}{.spec.ports[*].nodePort}{","}'

ref

• https://www.cnblogs.com/charlieroro/p/17387611.html
• https://www.cnblogs.com/zhangmingcheng/p/15031143.html
• https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/...
• https://www.kernel.org/doc/Documentation/networking/ip-sysctl...
• https://github.com/kubernetes/kubernetes/pull/115374
• https://github.com/kubernetes/kubernetes/issues/111144