StarRocks之K8S-4层代理TCP

发表于 更新于 阅读次数： 阅读次数：

背景

内网k8s集群需求：StarRocks的9030端口或mysql的3306端口需要暴露出去，而他们TCP协议，是L4层服务，而ingress是http协议，是L7层服务，不能使用ingress暴露出去

  • k8s-Starrocks情况
    • services: starrocks/starrockscluster-fe-service
      eaVxzN

相关配置

  • deployment: ingress-nginx-controller配置

    • 增加 hostNetwork: true，pod中运行的应用程序可以直接看到宿主主机的网络接口，宿主机所在的局域网上所有网络接口都可以访问到该应用程序及端口
    • 增加 - '--tcp-services-configmap=$(POD_NAMESPACE)/tcp-services'
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      spec:
        hostNetwork: true   // 增加
        containers:
          - name: controller
            image: dyrnq/ingress-nginx-controller:v1.6.4
            args:
              - /nginx-ingress-controller
              - '--election-id=ingress-nginx-leader'
              - '--controller-class=k8s.io/ingress-nginx'
              - '--ingress-class=nginx'
              - '--configmap=$(POD_NAMESPACE)/ingress-nginx-controller'
              - '--validating-webhook=:8443'
              - '--validating-webhook-certificate=/usr/local/certificates/cert'
              - '--validating-webhook-key=/usr/local/certificates/key'
              - '--tcp-services-configmap=$(POD_NAMESPACE)/tcp-services'  // 增加
              - '--udp-services-configmap=$(POD_NAMESPACE)/udp-services'

  • 编写TCP/UDP端口转发规则实现L4层服务暴露
    kubectl create -f tcp-services-configmap.yaml -n ingress-nginx

    1
    2
    3
    4
    5
    6
    7
    8
    9
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: tcp-services
      namespace: ingress-nginx
    data:
      '8030': starrocks/starrockscluster-fe-service:8030
      '8040': starrocks/starrockscluster-cn-service:8040
      '9030': starrocks/starrockscluster-fe-service:9030

  • 验证TCP 端口的L4服务暴露，查看pod nginx-ingress-controller的ip

    1
    2
    3
    4
    5
    > kubectl get pod -n ingress-nginx -owide
    NAME                                        READY   STATUS      RESTARTS   AGE   IP                NODE          NOMINATED NODE   READINESS GATES
    ingress-nginx-admission-create-cpjcl        0/1     Completed   0          70d   10.244.3.20       k8s-node3     <none>           <none>
    ingress-nginx-admission-patch-r6ql7         0/1     Completed   0          70d   10.244.2.12       k8s-node1     <none>           <none>
    ingress-nginx-controller-58bcff6c76-xdmzq   1/1     Running     0          14m   192.168.103.202   k8s-master1   <none>           <none>

  • navicat连接
    I7GLoM

相关连接

Nginx+Ingress-controller解决服务暴露和负载均衡