kubectl 通过 Kubepug 实现集群升级版本检查(废弃API资源检查)

前言

Kubepug 工具可用于k8s版本升级前对API 对象进行检查，并给出反馈。

因内网环境无法使用魔法，故此只能kubepug 离线安装，配置 kubectl 插件

k8s 的版本迭代很快，虽然主要版本一直没有变化，但是次要版本一直在迭代，同时不同的次要版本之间 API 资源一直在变化，有新加入的，也有废弃删除的。不同版本的 api 资源版本也有不同，往往不是向下兼容的，比如在低版本中 API 资源版本为 v1beta1，而高版本可能升级为 v1。但是在高版本中不能运行低版本的API资源。

所以在 k8s 版本升级的时候，需要对之前废弃的和删除的 API 资源 做出清理，需要升级的做升级，或者替换为其他的 API 资源。在这之前，需要一个工具来检查 API 资源对象，那些是已经废弃的删除的，那些将要废弃。

Kubepug 即是这样一个工具，一个升级前检查器，可帮助在迁移到新的主要版本之前在 Kubernetes 资源中找到已弃用和已删除的 API。

Kubepug 安装-离线

kubepug下载地址：https://github.com/kubepug/kubepug/releases 此次部署版本为最新的v1.7.1

[root@k140 bin]# tar -zxvf kubepug_linux_amd64.tar.gz
LICENSE
README.md
kubepug

[root@k140 bin]# mv ./kubepug  /usr/local/bin/

[root@k140 bin]# kubepug version
 __  ___  __    __  .______    _______ .______    __    __    _______
|  |/  / |  |  |  | |   _  \  |   ____||   _  \  |  |  |  |  /  _____|
|  '  /  |  |  |  | |  |_)  | |  |__   |  |_)  | |  |  |  | |  |  __
|    <   |  |  |  | |   _  <  |   __|  |   ___/  |  |  |  | |  | |_ |
|  .  \  |  `--'  | |  |_)  | |  |____ |  |      |  `--'  | |  |__| |
|__|\__\  \______/  |______/  |_______|| _|       \______/   \______|
kubepug: Shows all the deprecated objects in a Kubernetes cluster allowing the operator to verify them before upgrading the cluster.
It uses the Kubernetes API source code markers to define deprecated and deleted versions.

GitVersion:    v1.7.1
GitCommit:     unknown
GitTreeState:  unknown
BuildDate:     unknown
GoVersion:     go1.21.3
Compiler:      gc
Platform:      linux/amd64

配置为 kubectl 插件-可选

[root@k140 bin]# tar -zxvf kubepug_linux_amd64.tar.gz
LICENSE
README.md
kubepug

[root@k140 bin]# mv kubepug kubectl-kubepug

[root@k140 bin]# mv ./kubectl-kubepug  /usr/local/bin/

[root@k140 bin]# kubectl kubepug version
 __  ___  __    __  .______    _______   ______ .___________. __               __  ___  __    __  .______    _______ .______    __    __    _______
|  |/  / |  |  |  | |   _  \  |   ____| /      ||           ||  |             |  |/  / |  |  |  | |   _  \  |   ____||   _  \  |  |  |  |  /  _____|
|  '  /  |  |  |  | |  |_)  | |  |__   |  ,----'`---|  |----`|  |      ______ |  '  /  |  |  |  | |  |_)  | |  |__   |  |_)  | |  |  |  | |  |  __
|    <   |  |  |  | |   _  <  |   __|  |  |         |  |     |  |     |______||    <   |  |  |  | |   _  <  |   __|  |   ___/  |  |  |  | |  | |_ |
|  .  \  |  `--'  | |  |_)  | |  |____ |  `----.    |  |     |  `----.        |  .  \  |  `--'  | |  |_)  | |  |____ |  |      |  `--'  | |  |__| |
|__|\__\  \______/  |______/  |_______| \______|    |__|     |_______|        |__|\__\  \______/  |______/  |_______|| _|       \______/   \______|
kubectl-kubepug: Shows all the deprecated objects in a Kubernetes cluster allowing the operator to verify them before upgrading the cluster.
It uses the Kubernetes API source code markers to define deprecated and deleted versions.

GitVersion:    v1.7.1
GitCommit:     unknown
GitTreeState:  unknown
BuildDate:     unknown
GoVersion:     go1.21.3
Compiler:      gc
Platform:      linux/amd64

验证

[root@k140 bin]# kubepug
Error: Get "https://kubepug.xyz/data/data.json": dial tcp 185.199.111.153:443: i/o timeout
time="2024-06-17T14:51:33+08:00" level=error msg="An error has occurred: Get \"https://kubepug.xyz/data/data.json\": dial tcp 185.199.111.153:443: i/o timeout"

因为离线环境，会报上面的错误，需要把对应的 data.json 文件下载下来上传。然后在检查的时候指定文件目录。

data.json下载地址：https://kubepug.xyz/data/data.json

[root@k140 bin]# kubepug --database=/tmp/data.json

假如当前k8s版本为1.20，我们希望升级到 v1.24.2

[root@k8s-m01 tmp]# /usr/local/bin/kubepug --k8s-version=v1.24.2  --database ./data.json
RESULTS:
Deprecated APIs:
CronJob found in batch/v1beta1
         ├─ Deprecated at: 1.21
         ├─ Replacement: batch/v1/CronJob
         ├─ CronJob represents the configuration of a single cron job.
                -> OBJECT: delete-es-index-job namespace: logging

PodDisruptionBudget found in policy/v1beta1
         ├─ Deprecated at: 1.21
         ├─ Replacement: policy/v1/PodDisruptionBudget
         ├─ PodDisruptionBudget is an object to define the max disruption that can be caused to a collection of pods
                -> OBJECT: calico-kube-controllers namespace: kube-system
                -> OBJECT: alertmanager-main namespace: monitoring
                -> OBJECT: prometheus-adapter namespace: monitoring
                -> OBJECT: prometheus-k8s namespace: monitoring

EndpointSlice found in discovery.k8s.io/v1beta1
         ├─ Deprecated at: 1.21
         ├─ Replacement: discovery.k8s.io/v1/EndpointSlice
         ├─ EndpointSlice represents a subset of the endpoints that implement a service.For a given service there may be multiple EndpointSlice objects, selected bylabels, which must be joined to produce the full set of endpoints.
                -> OBJECT: kubernetes namespace: default
                -> OBJECT: default-http-backend-bkbp8 namespace: ingress-nginx
                -> OBJECT: ingress-nginx-controller-admission-2v97t namespace: ingress-nginx
                -> OBJECT: monitoring-ingress-nginx-g8t8z namespace: ingress-nginx
                -> OBJECT: sccgygsjh-rabbitmq-single-svc-74jbt namespace: tools
                -> OBJECT: weave-scope-app-2l42b namespace: weave
                -> OBJECT: zabbix-server-svc-2k2jl namespace: zabbix
                -> OBJECT: zabbix-web-svc-6455r namespace: zabbix



Kubepug validates the APIs using Kubernetes markers. To know what are the deprecated and deleted APIS it checks, please go to https://kubepug.xyz/status/

通过检查可以看到 ，1.24 之后的废弃和已经删除的 API 资源。如果要升级到 1.24 需要把删除的 API 处理掉。

以下为CronJobAPI更新前后的差异，与检查结果一致：

参考

1.https://kubepug.xyz/

2.https://github.com/kubepug/kubepug