環(huán)境說明
openpstack-Pike對(duì)接cephRBD單集群，配置簡單，可參考o(jì)penstack官網(wǎng)或者ceph官網(wǎng)；
1.Openstack官網(wǎng)參考配置：
https://docs.openstack.org/cinder/train/configuration/block-storage/drivers/ceph-rbd-volume-driver.html
2.Ceph官網(wǎng)參考配置：
https://docs.ceph.com/docs/master/install/install-ceph-deploy/
由于物理環(huán)境和業(yè)務(wù)需求變更，當(dāng)前配置云計(jì)算環(huán)境要求一套o(hù)penstack對(duì)接后臺(tái)兩套不同版本的cephRBD存儲(chǔ)集群；
此處以現(xiàn)有以下正常運(yùn)行環(huán)境展開配置；
1）openstack-Pike
2）Ceph Luminous 12.2.5
3）Ceph Nautilus 14.2.7
其中，openstack對(duì)接ceph Luminous配置完成，且正常運(yùn)行?，F(xiàn)在此套o(hù)penstack+ceph環(huán)境基礎(chǔ)上，新增一套ceph Nautilus存儲(chǔ)集群，使openstack能夠同時(shí)調(diào)用兩套存儲(chǔ)資源。

創(chuàng)新互聯(lián)2013年開創(chuàng)至今，是專業(yè)互聯(lián)網(wǎng)技術(shù)服務(wù)公司，擁有項(xiàng)目成都網(wǎng)站制作、成都網(wǎng)站建設(shè)網(wǎng)站策劃，項(xiàng)目實(shí)施與項(xiàng)目整合能力。我們以讓每一個(gè)夢(mèng)想脫穎而出為使命，1280元惠農(nóng)做網(wǎng)站,已為上家服務(wù),為惠農(nóng)各地企業(yè)和個(gè)人服務(wù),聯(lián)系電話:13518219792

配置步驟
1.拷貝配置文件
#拷貝配置文件、cinder賬戶key到openstack的cinder節(jié)點(diǎn)
/etc/ceph/ceph3.conf
/etc/ceph/ceph.client.cinder2.keyring
#此處使用cinder賬戶，僅拷貝cinder2賬戶的key即可

2.創(chuàng)建存儲(chǔ)池
#OSD添加完成后，創(chuàng)建存儲(chǔ)池，指定存儲(chǔ)池pg/pgp數(shù)，配置其對(duì)應(yīng)功能模式
ceph osd pool create volumes 512 512
ceph osd pool create backups 128 128
ceph osd pool create vms 512 512
ceph osd pool create images 128 128

ceph osd pool application enable volumes rbd
ceph osd pool application enable backups rbd
ceph osd pool application enable vms rbd
ceph osd pool application enable images rbd

3.創(chuàng)建集群訪問賬戶
ceph auth get-or-create client.cinder2 mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rx pool=images'
ceph auth get-or-create client.cinder2-backup mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=backups'
ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images'

4.查看進(jìn)程信息
#查看當(dāng)前openstack的cinder組件服務(wù)進(jìn)程
source /root/keystonerc.admin
cinder service-list

5.修改配置文件
#修改cinder配置文件
[DEFAULT]
enabled_backends = ceph2,ceph3

[ceph2]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph2
rbd_pool = volumes1
rbd_ceph_conf = /etc/ceph2/ceph2.conf
rbd_flatten_volume_from_snapshot = false
rbd_max_clone_depth = 5
rados_connect_timeout = -1
glance_api_version = 2
rbd_user = cinder1
rbd_secret_uuid = **

[ceph3]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph3
rbd_pool = volumes2
rbd_ceph_conf = /etc/ceph/ceph3/ceph3.conf
rbd_flatten_volume_from_snapshot = false
rbd_max_clone_depth = 5
rados_connect_timeout = -1
glance_api_version = 2
rbd_user = cinder2
rbd_secret_uuid = **

6.重啟服務(wù)
#重啟cinder-volume服務(wù)
service openstack-cinder-volume restart Redirecting to /bin/systemctl restart openstack-cinder-volume.service
service openstack-cinder-scheduler restart Redirecting to /bin/systemctl restart openstack-cinder-scheduler.service

7.查看進(jìn)程
cinder service-list

8.創(chuàng)建卷測(cè)試
#卷類型綁定
cinder type-create ceph2
cinder type-key ceph2 set volume_backend_name=ceph2
cinder type-create ceph3
cinder type-key ceph3 set volume_backend_name=ceph3

9.查看綁定結(jié)果
cinder create --volume-type ceph2 --display_name {volume-name}{volume-size}
cinder create --volume-type ceph3 --display_name {volume-name}{volume-size}

配置libvirt
1.將第二套ceph的密鑰添加到nova-compute節(jié)點(diǎn)的libvirt
#為了使VM可以訪問到第二套cephRBD云盤，需要在nova-compute節(jié)點(diǎn)上將第二套ceph的cinder用戶的密鑰添加到libvirt
ceph -c /etc/ceph3/ceph3/ceph3.conf -k /etc/ceph3/ceph.client.cinder2.keyring auth get-key client.cinder2 |tee client.cinder2.key

#綁定之前cinder.conf中第二個(gè)ceph集群的uuid
cat > secret2.xml <
***

client.cinder2 secret


#以上整段拷貝執(zhí)行即可，替換uuid值

sudo virsh secret-define --file secret2.xml

sudo virsh secret-set-value --secret *****--base64 $(cat client.cinder2.key) rm client.cinder2.key secret2.xml
#刪除提示信息，輸入Y即可

2.驗(yàn)證配置是否生效
#通過之前創(chuàng)建的兩個(gè)類型的云盤掛載到openstack的VM驗(yàn)證配置
nova volume-attach {instance-id}{volume1-id}
nova volume-attach {instance-id}{volume2-id}

參考資料：
《ceph設(shè)計(jì)原理與實(shí)現(xiàn)》---謝型果
紅帽官網(wǎng)
https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3/html/ceph_block_device_to_openstack_guide/installing_and_configuring_ceph_clients
ceph官網(wǎng)
https://docs.ceph.com/docs/master/install/install-ceph-deploy/