暂无图片
暂无图片
暂无图片
暂无图片
暂无图片
首页 / 使用nacos做为istio的服务发现

使用nacos做为istio的服务发现

codeImport 2020-03-29
2814

nacos是阿里巴巴开源的一个服务发现组件，同时也可以做为配置中心使用，可以理解为Eureka跟Apollo的组合。目前在github上有11k颗星，社区也是非常活跃，笔者也是其中一个代码贡献者。 

istio是由Google跟IBM的一些大佬开源出来的一套构建在k8s之上的网络服务，作为ServiceMesh的落地方案，主要作用是能够让使用者在不需要在项目中引入任何微服务组件的情况下，却能够拥有微服务的能力，这也是下一代微服务的方向。 

nacos从1.1.4版本开始，就支持作为istio的服务发现组件。官方文档为： nacos 1.1.4[1] ，从官方文档上看，使用nacos作为istio的服务发现组件，能够让istio托管的consumer直接调用nacos的provider，效果非常诱人，可惜网上几乎没有这方面的资料可以参考。本文将参考这篇官方文档使用nacos作为istio的服务发现组件，并尝试使用istio中的envoy调用nacos的provider。

本文使用的各组件版本为： 

nacos-1.1.4 ;

istio-1.4.4 ;

k8s-1.16

首先创建托管在istio中的consumer，注意nacos会把下发到istio的服务名字改成{serviceName}.sn-nacos的形式，所以此处使用的服务名为istio-provider.sn.nacos

@RequestMapping("/consumer")
@RestController
public class ConsumerController {


    @GetMapping("/nacos")
    public String nacos(){
        RestTemplate restTemplate = new RestTemplate();
        String version = restTemplate.getForObject("http://istio-provider.sn.nacos/provider/version",String.class);
        return version;
    }


}

在k8s的node节点将consumer打包成docker

[root@k8s-node1 consumer]# ls
build  consumer-0.0.1-SNAPSHOT.jar  dockerfile
[root@k8s-node1 consumer]# cat dockerfile 
FROM ascdc/jdk8
ENV HOME_PATH /spring
RUN ["mkdir", "-p", "/spring"]
ADD consumer-0.0.1-SNAPSHOT.jar $HOME_PATH/app.jar
WORKDIR $HOME_PATH
ENTRYPOINT ["java", "-jar", "app.jar"]
[root@k8s-node1 consumer]# cat build 
docker build -t istio-consumer:v1 .
[root@k8s-node1 consumer]#

在k8s的默认namespace上启用sidecar，只有开启了sidecar，istio才会给consumer生成envoy

kubectl label namespace default istio-injection=enabled

修改istio的配置，使用nacos做为服务发现

kubectl -n istio-system edit configmap istio


// 找到configSources配置，修改成nacos的MCP服务地址
    configSources:
    - address: istio-galley.istio-system.svc:9901
    - address: 192.168.5.131:18848

重启pilot

[root@k8s-master conf]# kubectl get pod -n istio-system
NAME                                      READY   STATUS      RESTARTS   AGE
istio-citadel-5565ccb5db-ds9n9            1/1     Running     0          93m
istio-galley-6764869b8d-rhsdv             1/1     Running     1          4d21h
istio-ingressgateway-fb88bfd88-ngcpj      1/1     Running     0          93m
istio-init-crd-10-1.4.4-pcjjg             0/1     Completed   0          4d21h
istio-init-crd-11-1.4.4-hjbxf             0/1     Completed   0          4d21h
istio-init-crd-14-1.4.4-km88h             0/1     Completed   0          4d21h
istio-pilot-7569fd9ff8-jzl8g              2/2     Running     0          38m
istio-policy-6496d968b-rnc2h              2/2     Running     7          4d21h
istio-security-post-install-1.4.4-6skm4   0/1     Completed   0          4d21h
istio-sidecar-injector-845b9499c7-nlltx   1/1     Running     1          4d21h
istio-telemetry-5b799647f5-cmn6l          2/2     Running     6          4d21h
prometheus-c8fdbd64f-nbvld                1/1     Running     0          93m
[root@k8s-master conf]# kubectl delete pod istio-pilot-7569fd9ff8-jzl8g -n istio-system
pod "istio-pilot-7569fd9ff8-jzl8g" deleted
[root@k8s-master conf]# kubectl get pod -n istio-system
NAME                                      READY   STATUS      RESTARTS   AGE
istio-citadel-5565ccb5db-ds9n9            1/1     Running     0          95m
istio-galley-6764869b8d-rhsdv             1/1     Running     1          4d21h
istio-ingressgateway-fb88bfd88-ngcpj      1/1     Running     0          95m
istio-init-crd-10-1.4.4-pcjjg             0/1     Completed   0          4d21h
istio-init-crd-11-1.4.4-hjbxf             0/1     Completed   0          4d21h
istio-init-crd-14-1.4.4-km88h             0/1     Completed   0          4d21h
istio-pilot-7569fd9ff8-htrpz              2/2     Running     0          59s
istio-policy-6496d968b-rnc2h              2/2     Running     7          4d21h
istio-security-post-install-1.4.4-6skm4   0/1     Completed   0          4d21h
istio-sidecar-injector-845b9499c7-nlltx   1/1     Running     1          4d21h
istio-telemetry-5b799647f5-cmn6l          2/2     Running     6          4d21h
prometheus-c8fdbd64f-nbvld                1/1     Running     0          95m


在k8s中启动istio-consumer

kubectl run istio-consumer --image=istio-consumer:v1 --port=9009

查看istio-consumer是否启动成功

[root@k8s-master conf]# kubectl get pod 
NAME                              READY   STATUS    RESTARTS   AGE
istio-consumer-7cf646d476-2clgx   2/2     Running   2          4d21h
[root@k8s-master conf]#


创建一个nacos的provider服务

@RestController
@RequestMapping("/provider")
public class ProviderController {
    @GetMapping("/version")
    public String getVersion() {
        String addr = "UnKnow";
        String hostName = "UnKnow";
        try {
            addr = InetAddress.getLocalHost().getHostAddress();
            hostName = InetAddress.getLocalHost().getHostName();
        } catch (Exception ex) {
            ex.printStackTrace();
        }
        return "Provider nacos provider,addr:" + addr + ",hostName:" + hostName;
    }
}

再创建一个心跳服务，向nacos注册该provider，并保持provider与nacos的心跳

@Component
@Slf4j
public class NacosBeatService {


    private String nacosRegisterUrl = "http://192.168.5.131:8848/nacos/v1/ns/instance?ip=192.168.5.131&port=9020&serviceName=istio-provider";


    private String beanJson = "";


    private String nacosBeatUrl = "http://192.168.5.131:8848/nacos/v1/ns/instance/beat?serviceName={serviceName}&beat={beat}";


    private RestTemplate restTemplate = new RestTemplate();


    @Scheduled(cron = "*/5 * * * * ?")
    public void beat() {
        Map<String, Object> map = new HashMap<>();
        map.put("serviceName", "istio-provider");
        map.put("beat", beanJson);
        restTemplate.put(nacosBeatUrl, Object.class, map);
        log.info("执行心跳");
    }


    @PostConstruct
    public void register() {


        JsonObject jsonObject = new JsonObject();
        jsonObject.addProperty("instanceId", "192.168.5.131#9020#DEFAULT#DEFAULT_GROUP@@istio-provider");
        jsonObject.addProperty("ip", "192.168.5.131");
        jsonObject.addProperty("port", 9020);
        jsonObject.addProperty("weight", 1.0);
        jsonObject.addProperty("healthy", true);
        jsonObject.addProperty("enabled", true);
        jsonObject.addProperty("ephemeral", true);
        jsonObject.addProperty("clusterName", "DEFAULT");
        jsonObject.addProperty("serviceName", "DEFAULT_GROUP@@istio-provider");
        jsonObject.addProperty("instanceHeartBeatInterval", 5000);
        jsonObject.addProperty("instanceIdGenerator", "simple");
        jsonObject.addProperty("ipDeleteTimeout", 30000);
        jsonObject.addProperty("instanceHeartBeatTimeOut", 15000);
        this.beanJson = jsonObject.toString();
        log.info("beanJaon:{}", beanJson);


        RestTemplate restTemplate = new RestTemplate();
        Map<String, String> request = new HashMap<>();
        request.put("ip", "192.168.5.131");
        request.put("port", "9020");
        request.put("serviceName", "istio-provider");
        String result = restTemplate.postForObject(nacosRegisterUrl, null, String.class);
        log.info(">>>>>>>>>registerResult:{}", result);


    }
}

修改nacos的配置，开启对istio的支持，并以standalone模式启动nacos

// 修改nacos配置 /usr/local/nacos/conf/application.properties


# nacos.naming.distro.taskDispatchPeriod=200
# nacos.naming.distro.batchSyncKeyCount=1000
# nacos.naming.distro.syncRetryDelay=5000
# nacos.naming.data.warmup=true
# nacos.naming.expireInstance=true
// 下面这句原来是false，改成true
nacos.istio.mcp.server.enabled=true


// 以standalone模式启动nacos
./startup.sh -m standalone

启动provider

java -jar provider-0.0.1-SNATSHOT

查看provider是否注册成功：curl -X GET 127.0.0.1:8848/nacos/v1/ns/instance/list?serviceName=istio-provider

[root@k8s-master bin]# curl -X GET 127.0.0.1:8848/nacos/v1/ns/instance/list?serviceName=istio-provider
{"metadata":{},"dom":"istio-provider","cacheMillis":3000,"useSpecifiedURL":false,"hosts":[{"valid":true,"marked":false,"metadata":{},"instanceId":"192.168.5.131#9020#DEFAULT#DEFAULT_GROUP@@istio-provider","port":9020,"healthy":true,"ip":"192.168.5.131","clusterName":"DEFAULT","weight":1.0,"ephemeral":true,"serviceName":"istio-provider","enabled":true}],"name":"DEFAULT_GROUP@@istio-provider","checksum":"f886c5b9e5d0a01909e743ae3102ee95","lastRefTime":1585483785467,"env":"","clusters":""}


查看nacso是否将该服务上报到istio，如果打印如下log，说明上报成功

cat /usr/local/nacos/nacos/logs/istio-main.log


2020-03-29 19:53:49,104 INFO MCP push, resource count is: 1


2020-03-29 19:53:49,104 DEBUG MCP push, sending resources: collection: "istio/networking/v1alpha3/serviceentries"
resources {
  metadata {
    name: "nacos/istio-provider.sn"
    annotations {
      key: "virtual"
      value: "1"
    }
  }
  body {
    type_url: "type.googleapis.com/istio.networking.v1alpha3.ServiceEntry"
    value: "\n\027istio-provider.sn.nacos\032\017\b\220E\022\004HTTP\032\004http \001(\0012\034\n\r192.168.5.131\022\t\n\004http\020\274F0\001"
  }
}
nonce: "1585482829104"




2020-03-29 19:53:49,117 INFO receiving request, sink: , type: istio/networking/v1alpha3/serviceentries


2020-03-29 19:53:49,117 INFO ACK nonce: 1585482829104, type: istio/networking/v1alpha3/serviceentries




查看envoy是否接收到该服务，可以看到envoy已经接收到了nacos上报的服务名istio-provider.sn.nacos:8848

[root@k8s-master logs]# istioctl proxy-config clusters istio-consumer-7cf646d476-2clgx
SERVICE FQDN                                                         PORT      SUBSET         DIRECTION     TYPE                                              -         -              -             ORIGINAL_DST


istio-policy.istio-system.svc.cluster.local                          15004     -              outbound      EDS
istio-policy.istio-system.svc.cluster.local                          15014     -              outbound      EDS
istio-provider.sn.nacos                                              8848      -              outbound      EDS
istio-sidecar-injector.istio-system.svc.cluster.local                443       -              outbound      EDS
istio-sidecar-injector.istio-system.svc.cluster.local                15014     -              outbound      EDS                                                        -         -              -             STRICT_DNS

调用istio中的consumer服务

[root@k8s-master logs]# kubectl get pod -o wide
NAME                              READY   STATUS    RESTARTS   AGE     IP            NODE        NOMINATED NODE   READINESS GATES
istio-consumer-7cf646d476-2clgx   2/2     Running   2          4d21h   10.244.1.41   k8s-node1   <none>           <none>
[root@k8s-master logs]# curl http://10.244.1.41:9009/consumer/nacos
{"timestamp":"2020-03-29T12:53:39.816+0000","status":500,"error":"Internal Server Error","message":"I/O error on GET request for \"http://istio-provider.sn.nacos/provider/version\": istio-provider.sn.nacos; nested exception is java.net.UnknownHostException: istio-provider.sn.nacos","path":"/consumer/nacos"}[root@k8s-master logs]#

发现consumer并没有调用成功，没有找到istio-provider.sn.nacos服务，查看k8s中的服务列表，发现确实没有这个服务

[root@k8s-master logs]# kubectl get service
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   11d
[root@k8s-master logs]# kubectl get virtualservice
No resources found in default namespace.
[root@k8s-master logs]# kubectl get serviceentries
No resources found in default namespace.
[root@k8s-master logs]#

这次尝试使用nacos做为istio的服务发现，服务是注册上去了，但是没有调用成功，咨询了阿里的一些大佬，大佬说这套组合阿里内部已经使用在了生产环境，说明这个方案是可行的。但是这方面的资料比较少，期待社区的继续推动，如果有同学对这个感兴趣的，欢迎加我微信交流

References

[1]
 nacos 1.1.4: https://nacos.io/en-us/blog/nacos%201.1.4.html


近期热文

定位线上问题不求人

同一个事务操作多个数据库

聊一聊熟悉又陌生的autoCommit

Dubbo线程池配置详解


数据库
文章转载自codeImport，如果涉嫌侵权，请发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。

评论