作者:Tatsuya Naganawa 譯者:TF編譯組
成都創(chuàng)新互聯(lián)公司主要從事網(wǎng)站制作、網(wǎng)站建設(shè)、網(wǎng)頁(yè)設(shè)計(jì)、企業(yè)做網(wǎng)站、公司建網(wǎng)站等業(yè)務(wù)。立足成都服務(wù)楊浦,十年網(wǎng)站建設(shè)經(jīng)驗(yàn),價(jià)格優(yōu)惠、服務(wù)專業(yè),歡迎來(lái)電咨詢建站服務(wù):18980820575
編者按:
這里有幾個(gè)溝通的渠道,如果需要幫助,請(qǐng)嘗試使用它們:
官網(wǎng)社區(qū): https://tungsten.io/community/
中文官網(wǎng): https://tungstenfabric.org.cn/
微信公眾號(hào):TF中文社區(qū)
兩年來(lái),我在Tungsten Fabric的旅程中學(xué)到了不少知識(shí),接下來(lái),我會(huì)用幾篇文章來(lái)簡(jiǎn)要介紹一下。
首先,讓我們來(lái)看一個(gè)重要的問(wèn)題,SDN/Neutron/CNI都有很多不錯(cuò)的實(shí)現(xiàn)案例,為什么還要再嘗試另一個(gè)呢?據(jù)我所知,Tungsten Fabric具有兩個(gè)關(guān)鍵的差異化特征,這使它變得如此與眾不同。
盡管有很多技術(shù)使Linux軟件成為生產(chǎn)路由器/交換機(jī)的理想選擇,但ASIC仍然是這一行業(yè)的重要組成部分。為了與它們進(jìn)行互操作,SDN平臺(tái)需要使用路由協(xié)議,例如BGP或OVSDB。
許多服務(wù)提供商和云服務(wù)商使用VRF來(lái)終結(jié)和分離每個(gè)客戶的網(wǎng)絡(luò)連接,這使得路由器和SDN之間的連接變得很復(fù)雜。
通常,它們之間可以使用VLAN,但是SDN平臺(tái)上的終結(jié)點(diǎn)可能會(huì)成為瓶頸
此外,每個(gè)SDN終結(jié)點(diǎn)(類似于OpenStack中的網(wǎng)絡(luò)節(jié)點(diǎn))需要為每個(gè)客戶進(jìn)行單獨(dú)的配置,這使配置更加復(fù)雜
借助MP-BGP這個(gè)已經(jīng)成熟實(shí)現(xiàn)的協(xié)議,Tungsten Fabric解決了上述問(wèn)題,該協(xié)議允許路由器上的每個(gè)VRF將報(bào)文直接發(fā)送到vRouter,這些vRouter服務(wù)于每個(gè)客戶的應(yīng)用程序。此功能允許基于控制平面(而不是數(shù)據(jù)平面),對(duì)具有每個(gè)客戶獨(dú)立網(wǎng)絡(luò)的計(jì)算節(jié)點(diǎn)進(jìn)行水平擴(kuò)展,并使其更加直觀。
由于報(bào)文是從路由器直接發(fā)送到vRouter的,因此不需要網(wǎng)絡(luò)節(jié)點(diǎn),這使得Tungsten Fabric在數(shù)據(jù)平面上具有更大的可擴(kuò)展性。
另外,從控制平面的角度來(lái)看,Tungsten Fabric具有一個(gè)名為“路由目標(biāo)過(guò)濾器(route target filtering)”( https://tools.ietf.org/html/rfc4684 )的有意思的功能。
此功能在MP-BGP中很常見(jiàn),其他路由器也具有這個(gè)功能
這個(gè)功能意味著,如果vRouter沒(méi)有該路由目標(biāo)(route target)的前綴,那么控制平面接收到它時(shí)會(huì)丟棄前綴
在云服務(wù)中,客戶只使用云服務(wù)商的數(shù)據(jù)中心的有限功能,并且不同客戶使用不同的路由目標(biāo),因此vRouter和控制器不需要知道所有前綴。路由目標(biāo)過(guò)濾功能使這種行為成為可能,并大大減少了每個(gè)vRouter以及每個(gè)控制器(如果在它們之間使用RR的話)需要考慮的前綴數(shù)量,這使得該控制平面更具可擴(kuò)展性。
對(duì)于私有云或托管云來(lái)說(shuō),將其與安全策略、網(wǎng)絡(luò)策略/邏輯路由器(與AWS中的VPC peerling或transit-gateway類似)等其它功能結(jié)合在一起,這將是VPC基礎(chǔ)架構(gòu)(類似于AWS/Azure/GCP VPC/vnet)的一個(gè)不錯(cuò)的選擇,并且使其成為一個(gè)非常值得一試的有趣平臺(tái)。
在第一次嘗試使用Tungsten Fabric時(shí),即使你已經(jīng)熟悉其他CNI的部署,還是建議使用ansible-deployer( https://github.com/Juniper/contrail-ansible-deployer ),因?yàn)門(mén)ungsten Fabric使用了很多種沒(méi)有集成在Vanilla Linux中的工具。因此,建議首先嘗試可以良好運(yùn)行的設(shè)置,以查看新功能,然后再集成其它系統(tǒng)。
不幸的是,許多Tungsten Fabric的repos都與rawhide類似,并且在某些情況下已經(jīng)失去了依賴性。
因此,我選擇了一種組合,是我認(rèn)為通常都能正常工作并且足夠穩(wěn)定,可以用來(lái)嘗試大多數(shù)功能。
要嘗試此操作,你需要兩臺(tái)服務(wù)器,一臺(tái)用于K8s主服務(wù)器,另一臺(tái)用于K8s節(jié)點(diǎn)。K8s主服務(wù)器需要至少有2個(gè)vCPU、8GB內(nèi)存和8GB磁盤(pán)空間。K8s節(jié)點(diǎn)服務(wù)器需要1個(gè)vCPU、4GB內(nèi)存和8GB磁盤(pán)空間。
我個(gè)人通常在AWS的ap-northeast-1區(qū)域中使用ami-3185744e(CentOS7.5, login-id: centos),t2.large的規(guī)格
由于在我看來(lái),與Tungsten Fabric進(jìn)行OpenStack和vCenter集成比使用Kubernetes進(jìn)行集成要復(fù)雜得多,因此即使你不需要容器支持,我還是建議首先嘗試此設(shè)置
安裝過(guò)程,需要有互聯(lián)網(wǎng)連接
##all the commands are typed at k8s master nodesudo yum -y remove PyYAML python-requests sudo yum -y install git sudo easy_install pip sudo pip install PyYAML requests ansible==2.7.15 ssh-keygen cd .ssh/ cat id_rsa.pub >> authorized_keys ssh-copy-id root@(k8s node's ip) ## or manually register id_rsa.pub to authorized_keyscd git clone -b R5.1 http://github.com/Juniper/contrail-ansible-deployer cd contrail-ansible-deployer vi config/instances.yaml (replace contents with this)provider_config: bms: ssh_user: root ssh_public_key: /root/.ssh/id_rsa.pub ssh_private_key: /root/.ssh/id_rsa domainsuffix: local ntpserver: 0.centos.pool.ntp.orginstances: bms1: provider: bms roles: config_database: config: control: analytics: analytics_database: webui: k8s_master: kubemanager: ip: 172.31.14.47 ## k8s master's ip bms2: provider: bms roles: vrouter: k8s_node: ip: 172.31.41.236 ## k8s node's ipcontrail_configuration: CONTRAIL_CONTAINER_TAG: r5.1 KUBERNETES_CLUSTER_PROJECT: {} JVM_EXTRA_OPTS: "-Xms128m -Xmx1g"global_configuration: CONTAINER_REGISTRY: tungstenfabric ansible-playbook -e orchestrator=kubernetes -i inventory/ playbooks/configure_instances.yml - it takes about 10 minutes ansible-playbook -e orchestrator=kubernetes -i inventory/ playbooks/install_k8s.yml - it takes about 5 minutes ansible-playbook -e orchestrator=kubernetes -i inventory/ playbooks/install_contrail.yml - it takes about 20 minutes
需要注意的一點(diǎn)是,使用受支持的內(nèi)核版本是一項(xiàng)相當(dāng)嚴(yán)格的要求,因?yàn)門(mén)ungsten Fabric將自己的內(nèi)核模塊(vrouter.ko)用于其數(shù)據(jù)平面。我嘗試了CentOS7.5、7.6,Ubuntu Xenial,并發(fā)現(xiàn)它也運(yùn)行良好(對(duì)于Ubuntu Bionic,需要進(jìn)行一些修改),但如果是第一次嘗試,我建議使用特定的AMI ID,因?yàn)橛捎诓荒苷9ぷ鞫M(jìn)行調(diào)試和排錯(cuò)不是一件容易的事。
如果所有“劇本”(playbooks)都運(yùn)行良好,可以先輸入內(nèi)容,
contrail-status
它負(fù)責(zé)檢查一切是否正常。
[root@ip-172-31-14-47 contrail-ansible-deployer]# contrail-status Pod Service Original Name State Status redis contrail-external-redis running Up 5 minutes analytics alarm-gen contrail-analytics-alarm-gen running Up 2 minutes analytics api contrail-analytics-api running Up 2 minutes analytics collector contrail-analytics-collector running Up 2 minutes analytics nodemgr contrail-nodemgr running Up 2 minutes analytics query-engine contrail-analytics-query-engine running Up 2 minutes analytics snmp-collector contrail-analytics-snmp-collector running Up 2 minutes analytics topology contrail-analytics-topology running Up 2 minutes config api contrail-controller-config-api running Up 4 minutes config device-manager contrail-controller-config-devicemgr running Up 3 minutes config nodemgr contrail-nodemgr running Up 4 minutes config schema contrail-controller-config-schema running Up 4 minutes config svc-monitor contrail-controller-config-svcmonitor running Up 4 minutes config-database cassandra contrail-external-cassandra running Up 4 minutes config-database nodemgr contrail-nodemgr running Up 4 minutes config-database rabbitmq contrail-external-rabbitmq running Up 4 minutes config-database zookeeper contrail-external-zookeeper running Up 4 minutes control control contrail-controller-control-control running Up 3 minutes control DNS contrail-controller-control-dns running Up 3 minutes control named contrail-controller-control-named running Up 3 minutes control nodemgr contrail-nodemgr running Up 3 minutes database cassandra contrail-external-cassandra running Up 2 minutes database kafka contrail-external-kafka running Up 2 minutes database nodemgr contrail-nodemgr running Up 2 minutes database zookeeper contrail-external-zookeeper running Up 2 minutes kubernetes kube-manager contrail-kubernetes-kube-manager running Up About a minute webui job contrail-controller-webui-job running Up 3 minutes webui web contrail-controller-webui-web running Up 3 minutes WARNING: container with original name 'contrail-external-redis' have Pod or Service empty. Pod: '' / Service: 'redis'. Please pass NODE_TYPE with pod name to container's env== Contrail control ==control: activenodemgr: activenamed: activedns: active== Contrail config-database ==nodemgr: initializing (Disk for DB is too low. )zookeeper: activerabbitmq: activecassandra: active== Contrail kubernetes ==kube-manager: active== Contrail database ==kafka: activenodemgr: initializing (Disk for DB is too low. )zookeeper: activecassandra: active== Contrail analytics ==snmp-collector: activequery-engine: activeapi: activealarm-gen: activenodemgr: activecollector: activetopology: active== Contrail webui ==web: activejob: active== Contrail config ==svc-monitor: activenodemgr: activedevice-manager: activeapi: activeschema: active[root@ip-172-31-14-47 contrail-ansible-deployer]# [root@ip-172-31-41-236 ~]# contrail-status Pod Service Original Name State Status vrouter agent contrail-vrouter-agent running Up 52 seconds vrouter nodemgr contrail-nodemgr running Up 52 seconds vrouter kernel module is PRESENT == Contrail vrouter ==nodemgr: activeagent: active[root@ip-172-31-41-236 ~]#
這應(yīng)該表明大多數(shù)組件都處于“active”狀態(tài),除了以下的:
nodemgr: initializing (Disk for DB is too low.)
這是可以在demo設(shè)置中安全地忽略的問(wèn)題。
注意:這基本上表明使用率超過(guò)50%,對(duì)于Cassandra來(lái)說(shuō)這是一個(gè)很重要的問(wèn)題。
如果一切正常,則可以嘗試使用此命令查看Tungsten Fabric路由表的狀態(tài)。
pip install lxml prettytable git clone https://github.com/vcheny/contrail-introspect-cli.git ##or curl -O https://raw.githubusercontent.com/vcheny/contrail-introspect-cli/master/ist.py ./contrail-introspect-cli/ist.py ctr status./contrail-introspect-cli/ist.py ctr nei ## similar to 'show bgp summary'./contrail-introspect-cli/ist.py ctr route summary ## similar to 'show route summary'./contrail-introspect-cli/ist.py ctr route tables ## show routing-tables./contrail-introspect-cli/ist.py ctr route show ## similar to 'show route'[root@ip-172-31-14-47 contrail-ansible-deployer]# ./contrail-introspect-cli/ist.py ctr statusmodule_id: contrail-control state: Functional description +-----------+-----------+---------------------+--------+----------------------------------+| type | name | server_addrs | status | description | +-----------+-----------+---------------------+--------+----------------------------------+| Collector | n/a | 172.31.14.47:8086 | Up | Established | | Database | Cassandra | 172.31.14.47:9041 | Up | Established Cassandra connection | | Database | RabbitMQ | 172.31.14.47:5673 | Up | RabbitMQ connection established | +-----------+-----------+---------------------+--------+----------------------------------+[root@ip-172-31-14-47 contrail-ansible-deployer]# ./contrail-introspect-cli/ist.py ctr nei +--------------------------------------+---------------+----------+----------+-----------+-------------+------------+------------+-----------+| peer | peer_address | peer_asn | encoding | peer_type | state | send_state | flap_count | flap_time | +--------------------------------------+---------------+----------+----------+-----------+-------------+------------+------------+-----------+| ip-172-31-41-236.ap- | 172.31.41.236 | 0 | XMPP | internal | Established | in sync | 0 | n/a | | northeast-1.compute.internal | | | | | | | | | +--------------------------------------+---------------+----------+----------+-----------+-------------+------------+------------+-----------+[root@ip-172-31-14-47 contrail-ansible-deployer]# ./contrail-introspect-cli/ist.py ctr route summary +----------------------------------------------------+----------+-------+---------------+-----------------+------------------+| name | prefixes | paths | primary_paths | secondary_paths | infeasible_paths | +----------------------------------------------------+----------+-------+---------------+-----------------+------------------+| default-domain:default- | 0 | 0 | 0 | 0 | 0 | | project:__link_local__:__link_local__.inet.0 | | | | | | | default-domain:default-project:default-virtual- | 0 | 0 | 0 | 0 | 0 | | network:default-virtual-network.inet.0 | | | | | | | inet.0 | 0 | 0 | 0 | 0 | 0 | | default-domain:default-project:ip-fabric:ip- | 3 | 3 | 1 | 2 | 0 | | fabric.inet.0 | | | | | | | default-domain:k8s-default:k8s-default-pod-network | 3 | 3 | 1 | 2 | 0 | | :k8s-default-pod-network.inet.0 | | | | | | | default-domain:k8s-default:k8s-default-service- | 3 | 3 | 1 | 2 | 0 | | network:k8s-default-service-network.inet.0 | | | | | | +----------------------------------------------------+----------+-------+---------------+-----------------+------------------+[root@ip-172-31-14-47 contrail-ansible-deployer]# ./contrail-introspect-cli/ist.py ctr route tablesname: default-domain:default-project:__link_local__:__link_local__.inet.0name: default-domain:default-project:default-virtual-network:default-virtual-network.inet.0name: inet.0name: default-domain:default-project:ip-fabric:ip-fabric.inet.0name: default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network.inet.0name: default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network.inet.0[root@ip-172-31-14-47 contrail-ansible-deployer]# ./contrail-introspect-cli/ist.py ctr route showbgp.ermvpn.0: 6 destinations, 6 routes (0 primary, 6 secondary, 0 infeasible)1-172.31.41.236:1-172.31.14.47,255.255.255.255,0.0.0.0, age: 0:02:26.545449, last_modified: 2019-Apr-13 01:41:18.023211 [Local|None] age: 0:02:26.548569, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None1-172.31.41.236:2-172.31.14.47,255.255.255.255,0.0.0.0, age: 0:01:09.096721, last_modified: 2019-Apr-13 01:42:35.471939 [Local|None] age: 0:01:09.100272, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None1-172.31.41.236:3-172.31.14.47,255.255.255.255,0.0.0.0, age: 0:00:41.812247, last_modified: 2019-Apr-13 01:43:02.756413 [Local|None] age: 0:00:41.816037, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None2-172.31.41.236:1-172.31.14.47,255.255.255.255,0.0.0.0, age: 0:02:26.544851, last_modified: 2019-Apr-13 01:41:18.023809 [Local|None] age: 0:02:26.548875, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None2-172.31.41.236:2-172.31.14.47,255.255.255.255,0.0.0.0, age: 0:01:09.096567, last_modified: 2019-Apr-13 01:42:35.472093 [Local|None] age: 0:01:09.100828, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None2-172.31.41.236:3-172.31.14.47,255.255.255.255,0.0.0.0, age: 0:00:41.812032, last_modified: 2019-Apr-13 01:43:02.756628 [Local|None] age: 0:00:41.816542, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: Nonebgp.evpn.0: 3 destinations, 3 routes (0 primary, 3 secondary, 0 infeasible)2-172.31.41.236:1-0-0e:92:cc:bd:aa:08,0.0.0.0, age: 0:02:26.545224, last_modified: 2019-Apr-13 01:41:18.023436 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.550028, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'mpls-o-gre', 'udp'], label: 20, AS path: None2-172.31.41.236:1-0-0e:92:cc:bd:aa:08,172.31.41.236, age: 0:02:26.545271, last_modified: 2019-Apr-13 01:41:18.023389 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.550313, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'mpls-o-gre', 'udp'], label: 20, AS path: None3-172.31.41.236:1-2-172.31.41.236, age: 0:02:26.545365, last_modified: 2019-Apr-13 01:41:18.023295 [Local|None] age: 0:02:26.550656, localpref: 100, nh: 172.31.41.236, encap: ['vxlan'], label: 2, AS path: Nonebgp.l3vpn.0: 3 destinations, 3 routes (0 primary, 3 secondary, 0 infeasible)172.31.41.236:1:172.31.41.236/32, age: 0:02:26.545019, last_modified: 2019-Apr-13 01:41:18.023641 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.550608, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp', 'native'], label: 16, AS path: None172.31.41.236:2:10.47.255.252/32, age: 0:00:41.733374, last_modified: 2019-Apr-13 01:43:02.835286 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:00:41.739187, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 25, AS path: None172.31.41.236:3:10.96.0.10/32, age: 0:00:41.732905, last_modified: 2019-Apr-13 01:43:02.835755 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:00:41.738945, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 25, AS path: Nonebgp.rtarget.0: 7 destinations, 7 routes (7 primary, 0 secondary, 0 infeasible)64512:target:64512:8000001, age: 0:02:26.592101, last_modified: 2019-Apr-13 01:41:17.976559 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.598445, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None64512:target:64512:8000002, age: 0:02:26.592073, last_modified: 2019-Apr-13 01:41:17.976587 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.598626, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None64512:target:64512:8000003, age: 0:02:26.592051, last_modified: 2019-Apr-13 01:41:17.976609 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.598800, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None64512:target:172.31.14.47:0, age: 0:05:09.194543, last_modified: 2019-Apr-13 01:38:35.374117 [Local|None] age: 0:05:09.201488, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None64512:target:172.31.14.47:1, age: 0:02:26.592028, last_modified: 2019-Apr-13 01:41:17.976632 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.599168, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None64512:target:172.31.14.47:4, age: 0:01:09.099898, last_modified: 2019-Apr-13 01:42:35.468762 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:01:09.107253, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None64512:target:172.31.14.47:5, age: 0:00:41.824049, last_modified: 2019-Apr-13 01:43:02.744611 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:00:41.831612, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: Nonedefault-domain:default-project:ip-fabric:ip-fabric.ermvpn.0: 3 destinations, 3 routes (3 primary, 0 secondary, 0 infeasible)0-172.31.41.236:1-0.0.0.0,255.255.255.255,0.0.0.0, age: 0:02:26.544896, last_modified: 2019-Apr-13 01:41:18.023764 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.552710, localpref: 100, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 0, AS path: None1-0:0-172.31.14.47,255.255.255.255,0.0.0.0, age: 0:02:26.545544, last_modified: 2019-Apr-13 01:41:18.023116 [Local|None] age: 0:02:26.553571, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None2-0:0-172.31.14.47,255.255.255.255,0.0.0.0, age: 0:02:26.544992, last_modified: 2019-Apr-13 01:41:18.023668 [Local|None] age: 0:02:26.553215, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: Nonedefault-domain:default-project:ip-fabric:ip-fabric.evpn.0: 4 destinations, 4 routes (4 primary, 0 secondary, 0 infeasible)2-0:0-0-0e:92:cc:bd:aa:08,0.0.0.0, age: 0:02:26.545298, last_modified: 2019-Apr-13 01:41:18.023362 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.553810, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'mpls-o-gre', 'udp'], label: 20, AS path: None2-0:0-0-0e:92:cc:bd:aa:08,172.31.41.236, age: 0:02:26.545318, last_modified: 2019-Apr-13 01:41:18.023342 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.554076, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'mpls-o-gre', 'udp'], label: 20, AS path: None2-172.31.41.236:1-2-ff:ff:ff:ff:ff:ff,0.0.0.0, age: 0:02:26.545486, last_modified: 2019-Apr-13 01:41:18.023174 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.554476, localpref: 100, nh: 172.31.41.236, encap: ['vxlan'], label: 2, AS path: None3-172.31.41.236:1-2-172.31.41.236, age: 0:02:26.545411, last_modified: 2019-Apr-13 01:41:18.023249 [Local|None] age: 0:02:26.554614, localpref: 100, nh: 172.31.41.236, encap: ['vxlan'], label: 2, AS path: Nonedefault-domain:default-project:ip-fabric:ip-fabric.inet.0: 3 destinations, 3 routes (1 primary, 2 secondary, 0 infeasible)10.47.255.252/32, age: 0:00:41.733312, last_modified: 2019-Apr-13 01:43:02.835348 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:00:41.742801, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 25, AS path: None10.96.0.10/32, age: 0:00:41.732847, last_modified: 2019-Apr-13 01:43:02.835813 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:00:41.742561, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 25, AS path: None172.31.41.236/32, age: 0:02:26.545051, last_modified: 2019-Apr-13 01:41:18.023609 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.554985, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp', 'native'], label: 16, AS path: Nonedefault-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network.ermvpn.0: 3 destinations, 3 routes (3 primary, 0 secondary, 0 infeasible)0-172.31.41.236:2-0.0.0.0,255.255.255.255,0.0.0.0, age: 0:01:09.096823, last_modified: 2019-Apr-13 01:42:35.471837 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:01:09.107020, localpref: 100, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 0, AS path: None1-0:0-172.31.14.47,255.255.255.255,0.0.0.0, age: 0:01:09.096765, last_modified: 2019-Apr-13 01:42:35.471895 [Local|None] age: 0:01:09.107383, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None2-0:0-172.31.14.47,255.255.255.255,0.0.0.0, age: 0:01:09.096621, last_modified: 2019-Apr-13 01:42:35.472039 [Local|None] age: 0:01:09.107473, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: Nonedefault-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network.inet.0: 3 destinations, 3 routes (1 primary, 2 secondary, 0 infeasible)10.47.255.252/32, age: 0:00:41.733411, last_modified: 2019-Apr-13 01:43:02.835249 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:00:41.744526, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 25, AS path: None10.96.0.10/32, age: 0:00:41.732872, last_modified: 2019-Apr-13 01:43:02.835788 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:00:41.744256, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 25, AS path: None172.31.41.236/32, age: 0:02:26.544986, last_modified: 2019-Apr-13 01:41:18.023674 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.556602, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp', 'native'], label: 16, AS path: Nonedefault-domain:k8s-default:k8s-default-service-network:k8s-default-service-network.ermvpn.0: 3 destinations, 3 routes (3 primary, 0 secondary, 0 infeasible)0-172.31.41.236:3-0.0.0.0,255.255.255.255,0.0.0.0, age: 0:00:41.812457, last_modified: 2019-Apr-13 01:43:02.756203 [XMPP|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:00:41.824352, localpref: 100, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 0, AS path: None1-0:0-172.31.14.47,255.255.255.255,0.0.0.0, age: 0:00:41.812393, last_modified: 2019-Apr-13 01:43:02.756267 [Local|None] age: 0:00:41.824504, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: None2-0:0-172.31.14.47,255.255.255.255,0.0.0.0, age: 0:00:41.812099, last_modified: 2019-Apr-13 01:43:02.756561 [Local|None] age: 0:00:41.824428, localpref: 100, nh: 172.31.14.47, encap: [], label: 0, AS path: Nonedefault-domain:k8s-default:k8s-default-service-network:k8s-default-service-network.inet.0: 3 destinations, 3 routes (1 primary, 2 secondary, 0 infeasible)10.47.255.252/32, age: 0:00:41.733337, last_modified: 2019-Apr-13 01:43:02.835323 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:00:41.745932, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 25, AS path: None10.96.0.10/32, age: 0:00:41.732935, last_modified: 2019-Apr-13 01:43:02.835725 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:00:41.745758, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 25, AS path: None172.31.41.236/32, age: 0:02:26.544959, last_modified: 2019-Apr-13 01:41:18.023701 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:02:26.558031, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp', 'native'], label: 16, AS path: None[root@ip-172-31-14-47 contrail-ansible-deployer]#
如果顯示類似信息,則說(shuō)明一切正常,你可以基于K8s yaml創(chuàng)建容器了。
vi first-containers.yamlapiVersion: apps/v1beta1kind: Deploymentmetadata: name: cirros-deployment labels: app: cirros-deploymentspec: replicas: 2 selector: matchLabels: app: cirros-deployment template: metadata: labels: app: cirros-deployment spec: containers: - name: cirros image: cirros ports: - containerPort: 22 kubectl create -f first-containers.yaml kubectl get pod -o wide ## check pod name and ipkubectl exec -it cirros-deployment-xxxx sh ping (another pod's ip) [root@ip-172-31-14-47 ~]# kubectl create -f first-containers.yamldeployment "cirros-deployment" created [root@ip-172-31-14-47 ~]# [root@ip-172-31-14-47 ~]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE cirros-deployment-54b65ccf48-cr9dd 1/1 Running 0 34s 10.47.255.250 ip-172-31-41-236.ap-northeast-1.compute.internal cirros-deployment-54b65ccf48-z9dds 1/1 Running 0 34s 10.47.255.251 ip-172-31-41-236.ap-northeast-1.compute.internal [root@ip-172-31-14-47 ~]#[root@ip-172-31-14-47 ~]# kubectl exec -it cirros-deployment-54b65ccf48-cr9dd sh/ # / # / # ping 10.47.255.251PING 10.47.255.251 (10.47.255.251): 56 data bytes 64 bytes from 10.47.255.251: seq=0 ttl=63 time=0.572 ms 64 bytes from 10.47.255.251: seq=1 ttl=63 time=0.086 ms ^C --- 10.47.255.251 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 0.086/0.329/0.572 ms / #
太棒了!這是通過(guò)Tungsten Fabric vRouter傳輸?shù)牡谝粋€(gè)報(bào)文。
如果運(yùn)行有問(wèn)題,也請(qǐng)不要介意。Tungten Fabric的Slack站點(diǎn)可以為你提供幫助。在以下的地址提交你的日志,并嘗試尋求幫助來(lái)解決該問(wèn)題。地址: https://tungstenfabric.slack.com
( 編者按:國(guó)內(nèi)用戶請(qǐng)關(guān)注微信TF中文社區(qū),中文官網(wǎng):https://tungstenfabric.org.cn/ )
再次輸入“ist.py ctr route show”,你將看到k8s-pod-network包含了來(lái)自兩個(gè)pod以及每個(gè)pod的下一跳,與K8s節(jié)點(diǎn)的IP相同。
./contrail-introspect-cli/ist.py ctr route show (pod ip) ## similar to 'show route (some ip)'[root@ip-172-31-14-47 contrail-ansible-deployer]# ./contrail-introspect-cli/ist.py ctr route show 10.47.255.250default-domain:default-project:ip-fabric:ip-fabric.inet.0: 5 destinations, 5 routes (1 primary, 4 secondary, 0 infeasible)10.47.255.250/32, age: 0:03:10.553628, last_modified: 2019-Apr-13 01:46:13.217388 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:03:10.556716, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 37, AS path: Nonedefault-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network.inet.0: 5 destinations, 5 routes (3 primary, 2 secondary, 0 infeasible)10.47.255.250/32, age: 0:03:10.553734, last_modified: 2019-Apr-13 01:46:13.217282 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:03:10.557251, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 37, AS path: Nonedefault-domain:k8s-default:k8s-default-service-network:k8s-default-service-network.inet.0: 5 destinations, 5 routes (1 primary, 4 secondary, 0 infeasible)10.47.255.250/32, age: 0:03:10.553654, last_modified: 2019-Apr-13 01:46:13.217362 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:03:10.557453, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 37, AS path: None[root@ip-172-31-14-47 contrail-ansible-deployer]#
請(qǐng)注意,ip-fabric VN和k8s-default-service-network也具有該前綴,因?yàn)閗8s-pod-network的路由已“泄漏”到那些網(wǎng)絡(luò)。想要具有某個(gè)特定路由表的路由,可以使用-t選項(xiàng)。
[root@ip-172-31-14-47 contrail-ansible-deployer]# ./contrail-introspect-cli/ist.py ctr route show -t default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network.inet.0 10.47.255.251default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network.inet.0: 5 destinations, 5 routes (3 primary, 2 secondary, 0 infeasible)10.47.255.251/32, age: 0:05:44.533377, last_modified: 2019-Apr-13 01:46:09.193202 [XMPP (interface)|ip-172-31-41-236.ap-northeast-1.compute.internal] age: 0:05:44.536291, localpref: 200, nh: 172.31.41.236, encap: ['gre', 'udp'], label: 32, AS path: None [root@ip-172-31-14-47 contrail-ansible-deployer]#
附錄:外部訪問(wèn)
有人說(shuō)Tungsten Fabric總是需要好的路由器才能實(shí)現(xiàn)外部訪問(wèn),我認(rèn)為這里有些誤解。
實(shí)際上這是不對(duì)的,因?yàn)閺膙4.1開(kāi)始,Tungsten Fabric開(kāi)始支持一項(xiàng)稱為無(wú)網(wǎng)關(guān)(gatewayless)的功能,該功能允許容器直接與外界通信(對(duì)于帶有Calico的類似用例也很有用)。
要啟用此功能,你可以登錄Tungsten Fabric WebUI(https://(k8s masters's ip):8143,admin:contrail123)并進(jìn)入Configure > Networks > k8s-default-pod-network,以切換Advanced Options > IP Fabric Forwarding。
你還需要在該VN和default-domain:default-project:ip-fabric之間設(shè)置網(wǎng)絡(luò)策略,因?yàn)槿绻麤](méi)有此策略,RPF check將丟棄該報(bào)文
如果ping一下從容器到K8s master的IP,你會(huì)發(fā)現(xiàn)K8s master從容器接收到一個(gè)報(bào)文,并向K8s master添加靜態(tài)路由,ping的運(yùn)行效果良好。
請(qǐng)注意,如果你使用的是AWS,則需要進(jìn)行K8s節(jié)點(diǎn)的接口設(shè)置(EC2 > Network Interfaces > Change Source/Dest Check > Disabled)
因此,它允許對(duì)基于外部訪問(wèn)的網(wǎng)絡(luò)節(jié)點(diǎn)進(jìn)行類似的設(shè)置,該訪問(wèn)基于路由器上的靜態(tài)路由。
你還可以選擇將IPv4 BGP與無(wú)網(wǎng)關(guān)(gatewayless)結(jié)合使用,因?yàn)樗梢詣?dòng)態(tài)更新每個(gè)容器的下一跳,并將報(bào)文直接發(fā)送到正確的vRouter,從而消除了瓶頸。
注意:此虛擬網(wǎng)絡(luò)(virtual-network)也可以用作浮動(dòng)IP(floating-ip)的來(lái)源。
1.此虛擬網(wǎng)絡(luò)上設(shè)置“Advanced Options”>“External”( 浮動(dòng)IP池將使用“default”作為名稱來(lái)創(chuàng)建)
2.從Kubernetes或OpenStack分配浮動(dòng)IP(floating-ip)
對(duì)于Kubernetes,它將是外部IP的來(lái)源,并且需要將此參數(shù)指定給kube-manager: KUBERNETES_PUBLIC_FIP_POOL。例如:KUBERNETES_PUBLIC_FIP_POOL={'domain': 'default-domain', 'project': 'default', 'network': 'public-network1', 'name': 'default' }
對(duì)于OpenStack,horizon或cli可用于為虛擬機(jī)分配浮動(dòng)IP
這可能是你第一次接觸Tungsten Fabric,關(guān)于看完這篇文章后要怎么做,這里有一些建議。其實(shí),有很多事情可以做,例如高可用性、監(jiān)視,與其它編排器或路由器/交換機(jī)的集成等。
網(wǎng)絡(luò)上有很多資源,但是你需要做好選擇,我將首先推薦Contrail軟件包和教育材料中的一些資源,即便你只會(huì)使用開(kāi)源版本。
https://www.juniper.net/documentation/product/en_US/contrail-networking
https://www.juniper.net/uk/en/training/certification/certification-tracks/cloud-track?tab=jncia-cloud
Tungsten Fabric是一個(gè)功能強(qiáng)大的平臺(tái),具有很多功能,例如安全策略、分析、l3dsr負(fù)載平衡器、服務(wù)鏈、BGPaaS等,其中許多功能對(duì)于解決現(xiàn)實(shí)世界中的問(wèn)題都是非常棒的。這些鏈接將包含很多內(nèi)容以及到其他資源的鏈接。
編者按:
這里有幾個(gè)溝通的渠道,如果需要幫助,請(qǐng)嘗試使用它們:
官網(wǎng)社區(qū): https://tungsten.io/community/
中文官網(wǎng): https://tungstenfabric.org.cn/
微信公眾號(hào):TF中文社區(qū)
Tungsten Fabric 架構(gòu)解析 系列文章——
第一篇: TF主要特點(diǎn)和用例
第二篇: TF怎么運(yùn)作
第三篇:詳解vRouter體系結(jié)構(gòu)
第四篇: TF的服務(wù)鏈
第五篇: vRouter的部署選項(xiàng)
第六篇: TF如何收集、分析、部署?
第七篇: TF如何編排
第八篇: TF支持API一覽
第九篇: TF如何連接到物理網(wǎng)絡(luò)
第十篇: TF基于應(yīng)用程序的安全策略