Kubernetes1.21搭建harbor

对你无可奈何

背景:

一直使用的腾讯云的个人仓库做镜像仓库。早些时候腾讯云有了tcr容器镜像服务:
image.png
瞄了一眼感觉略贵。个人也就50个之内的image。就想用一下镜像安全,漏洞扫描。也没有那么强硬的需求。600多块一个月还是感觉略贵!还是老老实实搭建一下harbor吧!

kubernetes1.21搭建harbor

注:开始在tke上面尝试的是kubernetes1.20.6两个版本差距不大就忽略吧。最后是在我的腾讯云自建的kubernetes1.21集群上。参考了早些时候个人写的博客:https://duiniwukenaihe.github.io/2019/10/29/k8s-helm-install-hrbor/

1. 下载harbor-helm仓库

git clone方式

4301a4b7774a0237204a44c0e3fbcca.png

git clone https://github.com/goharbor/harbor-helm

helm必备

当然了这里已经安装了helm3,helm环境是必备的
041c26c6d632bbd5561c360a75c1cc5.png

wget https://get.helm.sh/helm-v3.6.3-linux-amd64.tar.gz
tar zxvf helm-v3.6.3-linux-amd64.tar.g
cd linux-amd64
cp helm /usr/local/bin/

helm fetch

这里也可以直接helm添加仓库的方式,算是复习一下helm命令吧。我是直接用了git clone的方式

[root@k8s-master-01 harbor-helm]# helm repo add harbor https://helm.goharbor.io
"harbor" has been added to your repositories
[root@k8s-master-01 harbor-helm]# cd /data/
[root@k8s-master-01 data]# helm search repo harbor
NAME             CHART VERSION    APP VERSION    DESCRIPTION                                       
harbor/harbor    1.7.2            2.3.2          An open source trusted cloud native registry th...
[root@k8s-master-01 data]# helm fetch harbor/harbor --version 1.7.2

image.png

修改配置文件

修改value.yaml配置文件:
集群使用traefik代理外部访问。expose type设置了clusterIP.设置了externalURL,storageclass。如下:

type:

image.png

externalURL:

image.png

storageclass:

image.png
注:由于cbs最小单位允许为10g切步长为10g.故除了registry外其他的存储都使用了10G.当然了使用其他存储可个人合理设置!

helm install安装

helm install harbor -f values.yaml . --namespace kube-ops
kubectl get pods -n kube-ops -w  

2bd310ee496179a0725a49a03725877.png
image.png
注:此图后补的

helm upgrade

如后面更改了values.yaml 更新应用可以使用一下命令升级应用:

helm upgrade harbor -f values.yaml . --namespace kube-ops

如删除harbor应用,则:

helm uninstall harbor -n kube-ops

traefik代理harbor对外暴露应用:

ingressroute:

cat ingress-harbor.yml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-http
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`harbor.xxx.com`) && PathPrefix(`/`)
      kind: Rule
      services:
        - name: harbor-portal
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-api
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`harbor.xxx.com`) && PathPrefix(`/api`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-service
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`harbor.xxx.com`) && PathPrefix(`/service`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-v2
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`harbor.xxx.com`) && PathPrefix(`/v2`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-chartrepo
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`harbor.xxx.com`) && PathPrefix(`/chartrepo`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-c
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`harbor.xxx.com`) && PathPrefix(`/c`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
kubectl apply -f ingress-harbor.yaml

image.png
默认登陆密码Harbor12345。当然也可以在value.yaml中进行提前替换修改!
image.png
image.png

traefik ingress

试一下ingress的方式
image.png
image.png

helm upgrade harbor -f values.yaml . --namespace kube-ops

image.png
注意:这里绑定了 另外一个域名!
web访问也是正常的!
-----------------------------分隔符------------------------------------------------------------------------------

出现的其他问题:

web访问正常 docker login登陆也正常 但是docker push 出现unkonwn blob?

0b6974bf2cea0087d864a1e2c588d2f.png
这样的原因估计是我的slb上面做了http自动跳转https。docker push的时候就出现了异常。网上看了很多解决的方法无从下手。基本上是说这样的?
1bbcf3787fec914dd5858b33af7cba2.png
最后偷懒用了一个简单的方法:
新建一个slb 。将主要的slb上面摘下个server放在新的slb上面。直接tcp代理。不做http强跳https。
image.png

 kubectl create secret tls all-xxxx-com --key=2_xxxx.com.key --cert=1_xxxx.com_bundle.crt -n kube-ops

image.png
ingress.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-http
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-xxxx-com
  routes:
    - match: Host(`harbor.xxxx.com`) && PathPrefix(`/`)
      kind: Rule
      services:
        - name: harbor-portal
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-api
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-xxxx-com
  routes:
    - match: Host(`harbor.xxxx.com`) && PathPrefix(`/api/`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-service
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-xxxx-com
  routes:
    - match: Host(`harbor.xxxx.com`) && PathPrefix(`/service/`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-v2
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-xxxx-com
  routes:
    - match: Host(`harbor.xxxx.com`) && PathPrefix(`/v2`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-chartrepo
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-xxxx-com
  routes:
    - match: Host(`harbor.xxxx.com`) && PathPrefix(`/chartrepo/`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-ops
  name: harbor-c
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-xxxx-com
  routes:
    - match: Host(`harbor.xxxx.com`) && PathPrefix(`/c/`)
      kind: Rule
      services:
        - name: harbor-core
          port: 80
kubectl apply -f ingress.yaml

image.png

image.png

还发现一个好玩的;
我最终是在我的自建集群搭建的harbor。然后呢存储是cbs!参见:Kuberentes集群添加腾讯云CBS为默认存储。但是我的work节点有ap-shanghai2还有ap-shanghai-3区的主机。虽然3区的节点我设置了不可调度。但是还有有快存储建在了三区然后这样的结果就是pod不能正常running调度。毕竟云硬盘是不能跨区挂载的。解决方式就是新建一个storageclass ap-shanghai-2,更改了harbor中的存储类!

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: cbs-shanghai-2
provisioner: com.tencent.cloud.csi.cbs
parameters:
  diskZone: ap-shanghai-2

当然了 最终我还是换成了nfs......因为我不想给redis database分配10g的硬盘啊 浪费资源。nfs的存储这里更要注意一下selfLink 了,可以参照Kubernetes 1.19.12升级到1.20.9(强调一下selfLink)中selfLink的配置。

体验一下审查服务

image.png
image.png
嗯呢要更新一下依赖了....

后记:

其实就是想体验一下harbor的审查服务.但是这页面感觉还是不太成熟。扫描完成能不能给我出一个漏洞分布图呢?高危漏洞比重?每个images的漏洞比重?同一个镜像不同tag的漏洞趋势?
image.png

阅读 1.6k

19 声望
5 粉丝
0 条评论
19 声望
5 粉丝
文章目录
宣传栏