<style>.ui-infobar, #doc.markdown-body { max-width: 1200px; }</style>
# Deploy OpenStack using Kolla-Ansible
> last update: 2022-12-09
###### tags: `openstack`, `kolla-ansible`
Use Kolla-Ansible to deploy a OpenStack.
Run NFS server on controller host as Cinder backend and for cinder-backup.
#### Lab Environment
Hosts
```
Hostname | Role and Spec.
--------------------------------------------------------------
nuc10 | deploy (Ubuntu 22.04)
nuc111 | controller (Ubuntu 20.04, 4CPU, 32G RAM, 192.168.1.23)
nuc71 | compute (Ubuntu 20.04, 2CPU, 16G RAM, 192.168.1.24)
nuc72 | compute (Ubuntu 20.04, 2CPU, 16G RAM, 192.168.1.25)
nuc73 | compute (Ubuntu 20.04, 2CPU, 16G RAM, 192.168.1.26)
```
NICs:
```
eno1: internal (192.168.1.0/24)
eno2: external
```
**_NOTE_:** The eno2 interface is a "USB" NIC, rename it to eno2 base on mac address on each host.
Account username: "ubuntu"
#### Prerequest
Add hosts to /etc/hosts in all hosts
```
192.168.1.15 NUC10i5FNH nuc10
192.168.1.22 NUC11ATKC42 nuc112
192.168.1.23 NUC11ATKC41 nuc111
192.168.1.24 NUC7CJYHN1 nuc71
192.168.1.25 NUC7CJYHN2 nuc72
192.168.1.26 NUC7CJYHN3 nuc73
```
Set key-based authentication over ssh
```
ycheng@NUC10:~$ ssh-keygen
ycheng@NUC10:~$ ssh-copy-id nuc111
ycheng@NUC10:~$ ssh-copy-id nuc71
ycheng@NUC10:~$ ssh-copy-id nuc72
ycheng@NUC10:~$ ssh-copy-id nuc73
```
Configure host network interface. For example on nuc111
```
ubuntu@nuc111:~$ cat /etc/netplan/00-installer-config.yaml
# This is the network config written by 'subiquity'
network:
ethernets:
eno1:
dhcp4: false
link-local: []
addresses:
- 192.168.1.23/24
gateway4: 192.168.1.1
nameservers:
addresses:
- 8.8.8.8
- 8.8.4.4
eno2:
dhcp4: false
link-local: []
match:
macaddress: f8:e4:3b:87:00:1f
set-name: eno2
version: 2
```
```
ubuntu@nuc111:~$ sudo netplan try
ubuntu@nuc111:~$ sudo netplan apply
```
**_NOTE_:** Do not configure IP address on eno2 interface.
Add user to /etc/sudoers for no password sudo
```
ubuntu ALL=(ALL) NOPASSWD:ALL
```
**__NOTE__**: Refer "Use local docker registry to deploy" section for using local registry.
#### 1. Setup kolla ansible on deploy node
Install dependency packages on
```
ycheng@NUC10:~$ sudo apt update -y
ycheng@NUC10:~$ sudo apt install python3-dev libffi-dev gcc libssl-dev -y
```
Install python virtual environment for kolla-ansible
```
ycheng@NUC10:~$ sudo apt install python3-venv -y
```
```
ycheng@NUC10:~$ python3 -m venv ~/Data/kolla-ansible-venv
```
```
ycheng@NUC10:~$ source ~/Data/kolla-ansible-venv/bin/activate
(kolla-ansible-venv) ycheng@NUC10:~$
```
```
(kolla-ansible-venv) ycheng@NUC10:~$ pip install -U pip
```
```
(kolla-ansible-venv) ycheng@NUC10:~$ pip install 'ansible>=4,<6'
```
Install Kolla-ansible
```
(kolla-ansible-venv) ycheng@NUC10:~$ pip install git+https://opendev.org/openstack/kolla-ansible@master
```
```
(kolla-ansible-venv) ycheng@NUC10:~$ sudo mkdir -p /etc/kolla
(kolla-ansible-venv) ycheng@NUC10:~$ sudo chown $USER:$USER /etc/kolla/
(kolla-ansible-venv) ycheng@NUC10:~$ ls -ld /etc/kolla/
drwxr-xr-x 2 ycheng ycheng 4096 Nov 5 16:01 /etc/kolla/
```
```
(kolla-ansible-venv) ycheng@NUC10:~$ cp -r ~/Data/kolla-ansible-venv/share/kolla-ansible/etc_examples//kolla/* /etc/kolla/
(kolla-ansible-venv) ycheng@NUC10:~$ cp ~/Data/kolla-ansible-venv/share/kolla-ansible/ansible/inventory/multinode /etc/kolla/
```
**_NOTE:_** for all-in-one deploy, use ~/Data/kolla-ansible-venv/share/kolla-ansible/ansible/inventory/all-in-one
Install Ansible Galaxy requirements
```
(kolla-ansible-venv) ycheng@NUC10:~$ kolla-ansible install-deps
Installing Ansible Galaxy dependencies
Starting galaxy collection install process
Process install dependency map
Cloning into '/home/ycheng/.ansible/tmp/ansible-local-111805_n2pn0om/tmp5syatw1g/ansible-collection-kolla9pniyzg2'...
remote: Enumerating objects: 150, done.
remote: Counting objects: 100% (150/150), done.
remote: Compressing objects: 100% (97/97), done.
remote: Total 409 (delta 120), reused 53 (delta 53), pack-reused 259
Receiving objects: 100% (409/409), 79.20 KiB | 477.00 KiB/s, done.
Resolving deltas: 100% (159/159), done.
Already on 'master'
Your branch is up to date with 'origin/master'.
Starting collection install process
Installing 'openstack.kolla:1.0.0' to '/home/ycheng/.ansible/collections/ansible_collections/openstack/kolla'
Created collection for openstack.kolla:1.0.0 at /home/ycheng/.ansible/collections/ansible_collections/openstack/kolla
openstack.kolla:1.0.0 was installed successfully
(kolla-ansible-venv) ycheng@NUC10:~$
```
Configure Ansible
```
(kolla-ansible-venv) ycheng@NUC10:~$ cat /etc/ansible/ansible.cfg
[default]
deprecation_warnings=False
host_key_checking=False
pipelining=True
forks=100
```
Prepare initial configuration
```
(kolla-ansible-venv) ycheng@NUC10:~$ cd /etc/kolla/
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ ls -l
total 52
-rw-rw-r-- 1 ycheng ycheng 32632 Nov 5 16:07 globals.yml
-rw-rw-r-- 1 ycheng ycheng 9490 Nov 5 16:25 multinode
-rw-rw-r-- 1 ycheng ycheng 5163 Nov 5 16:07 passwords.yml
```
Edit multinode inventory file
```
1 # These initial groups are the only groups required to be modified. The
2 # additional groups are for more control of the environment.
3 [control]
4 # These hostname must be resolvable from your deployment host
5 nuc111
6
7 # The above can also be specified as follows:
8 #control[01:03] ansible_user=kolla
9
10 # The network nodes are where your l3-agent and loadbalancers will run
11 # This can be the same as a host in the control group
12 [network:children]
13 control
14
15 [compute]
16 nuc7[1:3]
17
18 [monitoring:children]
19 control
20
21 # When compute nodes and control nodes use different interfaces,
22 # you need to comment out "api_interface" and other interfaces from the globals.yml
23 # and specify like below:
24 #compute01 neutron_external_interface=eth0 api_interface=em1 tunnel_interface=em1
25
26 [storage:children]
27 control
```
**__NOTE__**: I use "network:children", "storage:children" and "monitoring:children" to add "control" as children group. Use hostname if you do not want to use children groups.
Check inventory
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ ansible -i multinode all -m ping
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details
localhost | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
nuc111 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
nuc72 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
nuc73 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
nuc71 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
```
Set Kolla passwords
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ kolla-genpwd
```
Edit global.yml file as below
```
48 # This should be a VIP, an unused IP on your network that will float between
49 # the hosts running keepalived for high-availability. If you want to run an
50 # All-In-One without haproxy and keepalived, you can set enable_haproxy to no
51 # in "OpenStack options" section, and set this value to the IP of your
52 # 'network_interface' as set in the Networking section below.
53 #kolla_internal_vip_address: "10.10.10.254"
54 kolla_internal_vip_address: "192.168.1.250"
111 ##############################
112 # Neutron - Networking Options
113 ##############################
114 # This interface is what all your api services will be bound to by default.
115 # Additionally, all vxlan/tunnel and storage network traffic will go over this
116 # interface by default. This interface must contain an IP address.
117 # It is possible for hosts to have non-matching names of interfaces - these can
118 # be set in an inventory file per host or per group or stored separately, see
119 # http://docs.ansible.com/ansible/intro_inventory.html
120 # Yet another way to workaround the naming problem is to create a bond for the
121 # interface on all hosts and give the bond name here. Similar strategy can be
122 # followed for other types of interfaces.
123 #network_interface: "eth0"
124 network_interface: "eno1"
149 # This is the raw interface given to neutron as its external network port. Even
150 # though an IP address can exist on this interface, it will be unusable in most
151 # configurations. It is recommended this interface not be configured with any IP
152 # addresses for that reason.
153 #neutron_external_interface: "eth1"
154 neutron_external_interface: "eno2"
302 # OpenStack services can be enabled or disabled with these options
303 #enable_aodh: "no"
304 #enable_barbican: "no"
305 #enable_blazar: "no"
306 #enable_ceilometer: "no"
307 #enable_ceilometer_ipmi: "no"
308 #enable_cells: "no"
309 #enable_central_logging: "no"
310 #enable_ceph_rgw: "no"
311 #enable_ceph_rgw_loadbalancer: "{{ enable_ceph_rgw | bool }}"
312 #enable_cinder: "no"
313 enable_cinder: "yes"
314 #enable_cinder_backup: "yes"
315 enable_cinder_backup: "yes"
316 #enable_cinder_backend_hnas_nfs: "no"
317 #enable_cinder_backend_iscsi: "{{ enable_cinder_backend_lvm | bool }}"
318 #enable_cinder_backend_lvm: "no"
319 #enable_cinder_backend_nfs: "no"
320 enable_cinder_backend_nfs: "yes"
544 # Valid options are [ nfs, swift, ceph ]
545 #cinder_backup_driver: "ceph"
546 #cinder_backup_share: ""
547 #cinder_backup_mount_options_nfs: ""
548 cinder_backup_driver: "nfs"
549 cinder_backup_share: "nuc111:/kolla_nfs"
550 cinder_backup_mount_options_nfs: ""
```
Create "nfs_shares" file with nfs configuration
```
ycheng@NUC10:/etc/kolla$ mkdir config
ycheng@NUC10:/etc/kolla$ cat config/nfs_shares
nuc111:/kolla_nfs
```
#### 2. Setup NFS server on controller for cinder backend
```
root@nuc111:~# apt install nfs-kernel-server -y
root@nuc111:~# cat /etc/exports
/kolla_nfs 192.168.1.0/24(rw,sync,no_root_squash)
root@nuc111:~# mkdir /kolla_nfs
root@nuc111:~# systemctl start nfs-kernel-server
```
```
root@nuc111:~# systemctl status nfs-kernel-server
● nfs-server.service - NFS server and services
Loaded: loaded (/lib/systemd/system/nfs-server.service; enabled; vendor preset: enabled)
Drop-In: /run/systemd/generator/nfs-server.service.d
└─order-with-mounts.conf
Active: active (exited) since Sat 2022-11-05 11:51:21 UTC; 24min ago
Process: 67275 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS)
Process: 67276 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=0/SUCCESS)
Main PID: 67276 (code=exited, status=0/SUCCESS)
Nov 05 11:51:19 nuc111 systemd[1]: Starting NFS server and services...
Nov 05 11:51:19 nuc111 exportfs[67275]: exportfs: /etc/exports [1]: Neither 'subtree_check' or 'no_subtree_check' specified for export "192.168.1.0/24:/kolla_nfs".
Nov 05 11:51:19 nuc111 exportfs[67275]: Assuming default behaviour ('no_subtree_check').
```
#### 3. Deployment
Bootstrap servers with kolla deploy dependencies
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ kolla-ansible -i ./multinode bootstrap-servers
... output omit
PLAY RECAP *********************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
nuc111 : ok=31 changed=15 unreachable=0 failed=0 skipped=21 rescued=0 ignored=0
nuc71 : ok=30 changed=14 unreachable=0 failed=0 skipped=22 rescued=0 ignored=0
nuc72 : ok=30 changed=14 unreachable=0 failed=0 skipped=22 rescued=0 ignored=0
nuc73 : ok=30 changed=14 unreachable=0 failed=0 skipped=22 rescued=0 ignored=0
```
*_NOTE_*: if bootstrap-servers playbook failed with error. `Failed to update apt cache: W:Updating from such a repository can't be done securely, and is therefore disabled by default`, re-run the playbook.
Do pre-deployment checks for hosts
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ kolla-ansible -i ./multinode prechecks
... output omit
PLAY RECAP ******************************************************************************************************************************
localhost : ok=11 changed=0 unreachable=0 failed=0 skipped=14 rescued=0 ignored=0
nuc111 : ok=93 changed=0 unreachable=0 failed=0 skipped=159 rescued=0 ignored=0
nuc71 : ok=29 changed=0 unreachable=0 failed=0 skipped=30 rescued=0 ignored=0
nuc72 : ok=28 changed=0 unreachable=0 failed=0 skipped=27 rescued=0 ignored=0
nuc73 : ok=28 changed=0 unreachable=0 failed=0 skipped=27 rescued=0 ignored=0
```
Run actual deployment
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ kolla-ansible -i ./multinode deploy
... output omit
PLAY RECAP ******************************************************************************************************************************
localhost : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
nuc111 : ok=287 changed=122 unreachable=0 failed=0 skipped=220 rescued=0 ignored=0
nuc71 : ok=59 changed=33 unreachable=0 failed=0 skipped=45 rescued=0 ignored=0
nuc72 : ok=54 changed=33 unreachable=0 failed=0 skipped=45 rescued=0 ignored=0
nuc73 : ok=54 changed=33 unreachable=0 failed=0 skipped=46 rescued=0 ignored=0
```
#### 4. Access OpenStack
Install openstack client
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ pip install python-openstackclient -c https://releases.openstack.org/constraints/upper/master
```
Generate admin openrc and clouds.yaml
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ kolla-ansible post-deploy
```
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ ls -l
total 96
-rw------- 1 ycheng ycheng 565 Nov 5 23:38 admin-openrc.sh
-rw------- 1 ycheng ycheng 573 Nov 5 23:38 clouds.yaml
drwxrwxr-x 2 ycheng ycheng 4096 Nov 5 22:54 config
-rw-rw-r-- 1 ycheng ycheng 32846 Nov 5 22:27 globals.yml
-rw-rw-r-- 1 ycheng ycheng 9475 Nov 5 16:59 multinode
-rw-rw-r-- 1 ycheng ycheng 34047 Nov 5 17:10 passwords.yml
```
Access method 1: source admin-openrc.sh
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ source admin-openrc.sh
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ openstack catalog list
+-----------+----------------+---------------------------------------------------------------------------+
| Name | Type | Endpoints |
+-----------+----------------+---------------------------------------------------------------------------+
| nova | compute | RegionOne |
| | | public: http://192.168.1.250:8774/v2.1 |
| | | RegionOne |
| | | internal: http://192.168.1.250:8774/v2.1 |
| | | |
| cinderv3 | volumev3 | RegionOne |
| | | public: http://192.168.1.250:8776/v3/496d5b465e684f20a8217fa408728ac2 |
| | | RegionOne |
| | | internal: http://192.168.1.250:8776/v3/496d5b465e684f20a8217fa408728ac2 |
| | | |
| placement | placement | RegionOne |
| | | internal: http://192.168.1.250:8780 |
| | | RegionOne |
| | | public: http://192.168.1.250:8780 |
| | | |
| glance | image | RegionOne |
| | | internal: http://192.168.1.250:9292 |
| | | RegionOne |
| | | public: http://192.168.1.250:9292 |
| | | |
| keystone | identity | RegionOne |
| | | public: http://192.168.1.250:5000 |
| | | RegionOne |
| | | internal: http://192.168.1.250:5000 |
| | | |
| neutron | network | RegionOne |
| | | public: http://192.168.1.250:9696 |
| | | RegionOne |
| | | internal: http://192.168.1.250:9696 |
| | | |
| heat-cfn | cloudformation | RegionOne |
| | | internal: http://192.168.1.250:8000/v1 |
| | | RegionOne |
| | | public: http://192.168.1.250:8000/v1 |
| | | |
| heat | orchestration | RegionOne |
| | | public: http://192.168.1.250:8004/v1/496d5b465e684f20a8217fa408728ac2 |
| | | RegionOne |
| | | internal: http://192.168.1.250:8004/v1/496d5b465e684f20a8217fa408728ac2 |
| | | |
+-----------+----------------+---------------------------------------------------------------------------+
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ openstack compute service list
+--------------------------------------+----------------+--------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+--------------------------------------+----------------+--------+----------+---------+-------+----------------------------+
| 9cb2e662-0446-417d-9ecb-d0ef5679459e | nova-scheduler | nuc111 | internal | enabled | up | 2022-11-05T12:41:51.000000 |
| 19dc19a4-4d78-4909-95a4-60fda0e62839 | nova-conductor | nuc111 | internal | enabled | up | 2022-11-05T12:41:48.000000 |
| 795eefbd-9461-4462-8a68-125e77027a5b | nova-compute | nuc71 | nova | enabled | up | 2022-11-05T12:41:51.000000 |
| b1734c47-918c-4aee-9fc0-78326b4dde8e | nova-compute | nuc73 | nova | enabled | up | 2022-11-05T12:41:51.000000 |
| feb93682-0dab-4dc0-95cc-872d4196339d | nova-compute | nuc72 | nova | enabled | up | 2022-11-05T12:41:51.000000 |
+--------------------------------------+----------------+--------+----------+---------+-------+----------------------------+
```
Access method 2: use clouds.yaml
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ cat clouds.yaml
clouds:
kolla-admin:
auth:
auth_url: http://192.168.1.250:5000
project_domain_name: Default
user_domain_name: Default
project_name: admin
username: admin
password: Lo6CMreQQAwHmp3sCjR0NqaqDm0fCoX7I0OBVY1c
region_name: RegionOne
kolla-admin-internal:
auth:
auth_url: http://192.168.1.250:5000
project_domain_name: Default
user_domain_name: Default
project_name: admin
username: admin
password: Lo6CMreQQAwHmp3sCjR0NqaqDm0fCoX7I0OBVY1c
interface: internal
region_name: RegionOne
```
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ cp ./clouds.yaml /etc/openstack/
OR
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ mkdir ~/.config/openstack
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ cp ./clouds.yaml ~/.config/openstack
OR
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ export OS_CLIENT_CONFIG_FILE=/etc/kolla/clouds.yaml
```
openstack command with "--os-cloud" option to choose cloud
```
(kolla-ansible-venv) ycheng@NUC10:~$ openstack --os-cloud kolla-admin compute service list
+--------------------------------------+----------------+--------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+--------------------------------------+----------------+--------+----------+---------+-------+----------------------------+
| 9cb2e662-0446-417d-9ecb-d0ef5679459e | nova-scheduler | nuc111 | internal | enabled | up | 2022-11-05T13:05:22.000000 |
| 19dc19a4-4d78-4909-95a4-60fda0e62839 | nova-conductor | nuc111 | internal | enabled | up | 2022-11-05T13:05:18.000000 |
| 795eefbd-9461-4462-8a68-125e77027a5b | nova-compute | nuc71 | nova | enabled | up | 2022-11-05T13:05:21.000000 |
| b1734c47-918c-4aee-9fc0-78326b4dde8e | nova-compute | nuc73 | nova | enabled | up | 2022-11-05T13:05:21.000000 |
| feb93682-0dab-4dc0-95cc-872d4196339d | nova-compute | nuc72 | nova | enabled | up | 2022-11-05T13:05:21.000000 |
+--------------------------------------+----------------+--------+----------+---------+-------+----------------------------+
```
#### 5. Access Web GUI (Horizon)
Get admin password from admin-open.rc or clouds.yaml
#### Extra: create examples for demo
**__NOTE__**: copy "clouds.yaml" in controller node to /etc/kolla/ first.
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ ~/Data/kolla-ansible-venv/share/kolla-ansible/init-runonce
Checking for locally available cirros image.
None found, downloading cirros image.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 20.1M 100 20.1M 0 0 2928k 0 0:00:07 0:00:07 --:--:-- 4371k
Creating glance image.
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
| container_format | bare |
| created_at | 2022-11-06T01:03:54Z |
| disk_format | qcow2 |
| file | /v2/images/1fa43af3-ac22-4dba-85e9-04b21807c5d4/file |
| id | 1fa43af3-ac22-4dba-85e9-04b21807c5d4 |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros |
| owner | 496d5b465e684f20a8217fa408728ac2 |
| properties | os_hidden='False', os_type='linux', owner_specified.openstack.md5='', owner_specified.openstack.object='images/cirros', owner_specified.openstack.sha256='' |
| protected | False |
| schema | /v2/schemas/image |
| status | queued |
| tags | |
| updated_at | 2022-11-06T01:03:54Z |
| visibility | public |
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
Configuring neutron.
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2022-11-06T01:03:56Z |
| description | |
| distributed | False |
| enable_ndp_proxy | None |
| external_gateway_info | null |
| flavor_id | None |
| ha | False |
| id | d8adf379-d03a-4c58-80cb-9cf7003524a6 |
| name | demo-router |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| revision_number | 1 |
| routes | |
| status | ACTIVE |
| tags | |
| tenant_id | 496d5b465e684f20a8217fa408728ac2 |
| updated_at | 2022-11-06T01:03:56Z |
+-------------------------+--------------------------------------+
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2022-11-06T01:03:57Z |
| description | |
| dns_domain | None |
| id | 67e89302-e5c7-4a17-ad1c-675b664874c1 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | False |
| is_vlan_transparent | None |
| mtu | 1450 |
| name | demo-net |
| port_security_enabled | True |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| provider:network_type | vxlan |
| provider:physical_network | None |
| provider:segmentation_id | 472 |
| qos_policy_id | None |
| revision_number | 1 |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| tenant_id | 496d5b465e684f20a8217fa408728ac2 |
| updated_at | 2022-11-06T01:03:57Z |
+---------------------------+--------------------------------------+
+----------------------+--------------------------------------+
| Field | Value |
+----------------------+--------------------------------------+
| allocation_pools | 10.0.0.2-10.0.0.254 |
| cidr | 10.0.0.0/24 |
| created_at | 2022-11-06T01:03:59Z |
| description | |
| dns_nameservers | 8.8.8.8 |
| dns_publish_fixed_ip | None |
| enable_dhcp | True |
| gateway_ip | 10.0.0.1 |
| host_routes | |
| id | 312eff74-d80c-4281-a51a-9b4bed5132c4 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | demo-subnet |
| network_id | 67e89302-e5c7-4a17-ad1c-675b664874c1 |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2022-11-06T01:03:59Z |
+----------------------+--------------------------------------+
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2022-11-06T01:04:03Z |
| description | |
| dns_domain | None |
| id | 17cb537a-88ac-42ed-9eb2-29177cb7490e |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | False |
| is_vlan_transparent | None |
| mtu | 1500 |
| name | public1 |
| port_security_enabled | True |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| provider:network_type | flat |
| provider:physical_network | physnet1 |
| provider:segmentation_id | None |
| qos_policy_id | None |
| revision_number | 1 |
| router:external | External |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| tenant_id | 496d5b465e684f20a8217fa408728ac2 |
| updated_at | 2022-11-06T01:04:03Z |
+---------------------------+--------------------------------------+
+----------------------+--------------------------------------+
| Field | Value |
+----------------------+--------------------------------------+
| allocation_pools | 10.0.2.150-10.0.2.199 |
| cidr | 10.0.2.0/24 |
| created_at | 2022-11-06T01:04:04Z |
| description | |
| dns_nameservers | |
| dns_publish_fixed_ip | None |
| enable_dhcp | False |
| gateway_ip | 10.0.2.1 |
| host_routes | |
| id | 142f17ca-c06d-4c07-abdc-dc935b1b83a3 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | public1-subnet |
| network_id | 17cb537a-88ac-42ed-9eb2-29177cb7490e |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2022-11-06T01:04:04Z |
+----------------------+--------------------------------------+
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| created_at | 2022-11-06T01:04:11Z |
| description | |
| direction | ingress |
| ether_type | IPv4 |
| id | b485970a-d7ee-43d2-a1ea-1e440be2dd12 |
| name | None |
| normalized_cidr | 0.0.0.0/0 |
| port_range_max | None |
| port_range_min | None |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| protocol | icmp |
| remote_address_group_id | None |
| remote_group_id | None |
| remote_ip_prefix | 0.0.0.0/0 |
| revision_number | 0 |
| security_group_id | ab6be57e-a748-4b92-bbe0-3d26bab3d60b |
| tags | [] |
| updated_at | 2022-11-06T01:04:11Z |
+-------------------------+--------------------------------------+
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| created_at | 2022-11-06T01:04:12Z |
| description | |
| direction | ingress |
| ether_type | IPv4 |
| id | caec3c73-02cd-4f31-8c6c-c27370599559 |
| name | None |
| normalized_cidr | 0.0.0.0/0 |
| port_range_max | 22 |
| port_range_min | 22 |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| protocol | tcp |
| remote_address_group_id | None |
| remote_group_id | None |
| remote_ip_prefix | 0.0.0.0/0 |
| revision_number | 0 |
| security_group_id | ab6be57e-a748-4b92-bbe0-3d26bab3d60b |
| tags | [] |
| updated_at | 2022-11-06T01:04:12Z |
+-------------------------+--------------------------------------+
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| created_at | 2022-11-06T01:04:13Z |
| description | |
| direction | ingress |
| ether_type | IPv4 |
| id | 1b7a08bc-cdf3-4ced-a402-113302e5bd6a |
| name | None |
| normalized_cidr | 0.0.0.0/0 |
| port_range_max | 8000 |
| port_range_min | 8000 |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| protocol | tcp |
| remote_address_group_id | None |
| remote_group_id | None |
| remote_ip_prefix | 0.0.0.0/0 |
| revision_number | 0 |
| security_group_id | ab6be57e-a748-4b92-bbe0-3d26bab3d60b |
| tags | [] |
| updated_at | 2022-11-06T01:04:13Z |
+-------------------------+--------------------------------------+
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| created_at | 2022-11-06T01:04:14Z |
| description | |
| direction | ingress |
| ether_type | IPv4 |
| id | 05ed928f-07b9-4af1-abf5-589a0493e206 |
| name | None |
| normalized_cidr | 0.0.0.0/0 |
| port_range_max | 8080 |
| port_range_min | 8080 |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| protocol | tcp |
| remote_address_group_id | None |
| remote_group_id | None |
| remote_ip_prefix | 0.0.0.0/0 |
| revision_number | 0 |
| security_group_id | ab6be57e-a748-4b92-bbe0-3d26bab3d60b |
| tags | [] |
| updated_at | 2022-11-06T01:04:14Z |
+-------------------------+--------------------------------------+
Generating ssh key.
Generating public/private ecdsa key pair.
Your identification has been saved in /home/ycheng/.ssh/id_ecdsa
Your public key has been saved in /home/ycheng/.ssh/id_ecdsa.pub
The key fingerprint is:
SHA256:61Ij/vwAIbuaC7nT6W1LwCGCfJwNqhoJhq4+dia4Dao ycheng@NUC10
The key's randomart image is:
+---[ECDSA 256]---+
| . |
|+ o + |
|*+.+... |
|*+.. o . |
|+.o . . S |
|oo . ...o. |
|B. .o. oo. |
|=OoB. oo . |
|E=@oo. o+.. |
+----[SHA256]-----+
Configuring nova public key and quotas.
+-------------+-------------------------------------------------+
| Field | Value |
+-------------+-------------------------------------------------+
| created_at | None |
| fingerprint | 4e:2d:29:67:6d:2e:af:24:b1:40:22:0a:79:d0:6e:35 |
| id | mykey |
| is_deleted | None |
| name | mykey |
| type | ssh |
| user_id | 6714a667d1e3406fab0c7add7d71d412 |
+-------------+-------------------------------------------------+
+----------------------------+---------+
| Field | Value |
+----------------------------+---------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| description | None |
| disk | 1 |
| id | 1 |
| name | m1.tiny |
| os-flavor-access:is_public | True |
| properties | |
| ram | 512 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 1 |
+----------------------------+---------+
+----------------------------+----------+
| Field | Value |
+----------------------------+----------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| description | None |
| disk | 20 |
| id | 2 |
| name | m1.small |
| os-flavor-access:is_public | True |
| properties | |
| ram | 2048 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 1 |
+----------------------------+----------+
+----------------------------+-----------+
| Field | Value |
+----------------------------+-----------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| description | None |
| disk | 40 |
| id | 3 |
| name | m1.medium |
| os-flavor-access:is_public | True |
| properties | |
| ram | 4096 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 2 |
+----------------------------+-----------+
+----------------------------+----------+
| Field | Value |
+----------------------------+----------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| description | None |
| disk | 80 |
| id | 4 |
| name | m1.large |
| os-flavor-access:is_public | True |
| properties | |
| ram | 8192 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 4 |
+----------------------------+----------+
+----------------------------+-----------+
| Field | Value |
+----------------------------+-----------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| description | None |
| disk | 160 |
| id | 5 |
| name | m1.xlarge |
| os-flavor-access:is_public | True |
| properties | |
| ram | 16384 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 8 |
+----------------------------+-----------+
Done.
To deploy a demo instance, run:
openstack --os-cloud=kolla-admin server create \
--image cirros \
--flavor m1.tiny \
--key-name mykey \
--network demo-net \
demo1
(kolla-ansible-venv) ycheng@NUC10:~/Data/kolla-ansible-venv/share/kolla-ansible$
```
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ openstack --os-cloud=kolla-admin server create \
--image cirros \
--flavor m1.tiny \
--key-name mykey \
--network demo-net \
demo1
+-------------------------------------+-----------------------------------------------+
| Field | Value |
+-------------------------------------+-----------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | 6Fa52DZ6AXEy |
| config_drive | |
| created | 2022-11-06T01:08:01Z |
| flavor | m1.tiny (1) |
| hostId | |
| id | b942a0c6-7541-4380-91a4-19a16d875860 |
| image | cirros (1fa43af3-ac22-4dba-85e9-04b21807c5d4) |
| key_name | mykey |
| name | demo1 |
| progress | 0 |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2022-11-06T01:08:02Z |
| user_id | 6714a667d1e3406fab0c7add7d71d412 |
| volumes_attached | |
+-------------------------------------+-----------------------------------------------+
```
**_NOTE_**: configuration are located in /etc/kolla in all cluster nodes.
```
root@nuc111:/etc/kolla# ls -l
total 140
drwxrwx--- 2 root root 4096 Nov 5 11:44 cinder-api
drwxrwx--- 2 root root 4096 Nov 7 13:11 cinder-backup
drwxrwx--- 2 root root 4096 Nov 5 11:44 cinder-scheduler
drwxrwx--- 2 root root 4096 Nov 5 12:03 cinder-volume
drwxrwx--- 2 root root 4096 Nov 5 11:29 cron
drwxrwx--- 2 root root 4096 Nov 5 11:29 fluentd
drwxrwx--- 2 root root 4096 Nov 5 11:42 glance-api
drwxrwx--- 3 root root 4096 Nov 5 11:36 haproxy
drwxrwx--- 2 root root 4096 Nov 5 12:24 heat-api
drwxrwx--- 2 root root 4096 Nov 5 12:24 heat-api-cfn
drwxrwx--- 2 root root 4096 Nov 5 12:24 heat-engine
drwxrwx--- 2 root root 4096 Nov 5 12:25 horizon
drwxrwx--- 3 root root 4096 Nov 5 11:36 keepalived
drwxrwx--- 2 root root 4096 Nov 5 11:39 keystone
drwxrwx--- 2 root root 4096 Nov 5 11:39 keystone-fernet
drwxrwx--- 2 root root 4096 Nov 5 11:39 keystone-ssh
drwxrwx--- 2 root root 4096 Nov 5 11:29 kolla-toolbox
drwxrwx--- 2 root root 4096 Nov 5 11:37 mariadb
drwxrwx--- 2 root root 4096 Nov 5 11:37 mariadb-clustercheck
drwxrwx--- 2 root root 4096 Nov 5 11:38 memcached
drwxrwx--- 2 root root 4096 Nov 5 12:20 neutron-dhcp-agent
drwxrwx--- 2 root root 4096 Nov 5 12:21 neutron-l3-agent
drwxrwx--- 2 root root 4096 Nov 5 12:21 neutron-metadata-agent
drwxrwx--- 2 root root 4096 Nov 5 12:20 neutron-openvswitch-agent
drwxrwx--- 2 root root 4096 Nov 5 12:20 neutron-server
drwxrwx--- 2 root root 4096 Nov 5 12:07 nova-api
drwxrwx--- 2 root root 4096 Nov 5 12:05 nova-api-bootstrap
drwxrwx--- 2 root root 4096 Nov 5 12:06 nova-cell-bootstrap
drwxrwx--- 2 root root 4096 Nov 5 12:07 nova-conductor
drwxrwx--- 2 root root 4096 Nov 5 12:07 nova-novncproxy
drwxrwx--- 2 root root 4096 Nov 5 12:07 nova-scheduler
drwxrwx--- 2 root root 4096 Nov 5 12:18 openvswitch-db-server
drwxrwx--- 2 root root 4096 Nov 5 12:18 openvswitch-vswitchd
drwxrwx--- 2 root root 4096 Nov 5 12:05 placement-api
drwxrwx--- 2 root root 4096 Nov 5 11:39 rabbitmq
```
```
ubuntu@nuc71:~$ ls -al /etc/kolla/
total 44
drwxr-xr-x 11 root root 4096 Nov 5 12:20 .
drwxr-xr-x 107 root root 4096 Nov 5 12:07 ..
drwxrwx--- 2 root root 4096 Nov 5 11:29 cron
drwxrwx--- 2 root root 4096 Nov 5 11:29 fluentd
drwxrwx--- 2 root root 4096 Nov 5 11:29 kolla-toolbox
drwxrwx--- 2 root root 4096 Nov 5 12:20 neutron-openvswitch-agent
drwxrwx--- 2 root root 4096 Nov 5 12:08 nova-compute
drwxrwx--- 2 root root 4096 Nov 5 12:08 nova-libvirt
drwxrwx--- 2 root root 4096 Nov 5 12:08 nova-ssh
drwxrwx--- 2 root root 4096 Nov 5 12:18 openvswitch-db-server
drwxrwx--- 2 root root 4096 Nov 5 12:18 openvswitch-vswitchd
```
#### Extra: Add Ceph storage as an Cinder backend
**__NOTE__**: Refer to https://hackmd.io/8tkrVJP7QU-MjO-0_W13jQ for setup a Ceph storage.
This section provides steps to add a Ceph storage into Cinder volume as backend; and set Nova to use the Ceph storage for running instance.
Reference:
> https://docs.ceph.com/en/latest/rbd/rbd-openstack/
> https://docs.openstack.org/kolla-ansible/latest/reference/storage/external-ceph-guide.html
Create pool for volume
```
root@nuc111:~# ceph osd pool create volumes
pool 'volumes' created
root@nuc111:~# ceph osd pool create vms
pool 'vms' created
root@nuc111:~# ceph osd pool create backups
pool 'backups' created
```
```
root@nuc111:~# rbd pool init volumes
root@nuc111:~# rbd pool init vms
root@nuc111:~# rbd pool init backups
```
**__Note__**: we still create "backups" pool because "Kolla-Ansible" playbook still add Ceph keyring for cinder-backup no matter "nfs" is configured to cinder_backup_driver. I think its kinda of bug. Fail message if no keyring for "cinder-backup" configured.
```
TASK [cinder : Copy over Ceph keyring files for cinder-backup] ***************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: If you are using a module and expect the file to exist on the remote, see the remote_src option
failed: [nuc111] (item=cinder-backup/ceph.client.cinder.keyring) => {"ansible_loop_var": "item", "changed": false, "item": "cinder-backup/ceph.client.cinder.keyring", "msg": "Could not find or access '/etc/kolla/config/cinder/cinder-backup/ceph.client.cinder.keyring' on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"}
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: If you are using a module and expect the file to exist on the remote, see the remote_src option
failed: [nuc111] (item=cinder-backup/ceph.client.cinder-backup.keyring) => {"ansible_loop_var": "item", "changed": false, "item": "cinder-backup/ceph.client.cinder-backup.keyring", "msg": "Could not find or access '/etc/kolla/config/cinder/cinder-backup/ceph.client.cinder-backup.keyring' on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"}
```
Setup Ceph client authentication to create cinder user for accessing volumes and vms pool.
```
root@nuc111:~# ceph auth get-or-create client.cinder mon 'profile rbd' osd 'profile rbd pool=volumes, profile rbd pool=vms' mgr 'profile rbd pool=volumes, profile rbd pool=vms'
[client.cinder]
key = AQAC5nBjU4CrCRAAi3kUAheIpOz5dvWZKg9pNw==
```
Also for cinder-backup. (We still use NFS for cinder-backup anyway)
```
root@nuc111:/etc/kolla# ceph auth get-or-create client.cinder-backup mon 'profile rbd' osd 'profile rbd pool=backups' mgr 'profile rbd pool=backups'
[client.cinder-backup]
key = AQB++HBjbprHIBAAai68wc04/vfhYJGmNgTQoQ==
```
Copy the key to deploy node for cinder
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ mkdir -p ./config/cinder/cinder-volume/
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ cat ./config/cinder/cinder-volume/ceph.client.cinder.keyring
[client.cinder]
key = AQAC5nBjU4CrCRAAi3kUAheIpOz5dvWZKg9pNw==
```
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ mkdir -p ./config/cinder/cinder-backup/
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ cp ./config/cinder/cinder-volume/ceph.client.cinder.keyring ./config/cinder/cinder-backup/
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ cat ./config/cinder/cinder-backup/ceph.client.cinder-backup.keyring
[client.cinder-backup]
key = AQB++HBjbprHIBAAai68wc04/vfhYJGmNgTQoQ==
```
**__NOTE__**: cinder-backup needs cinder and cinder-backup keys.
Copy ceph.conf to deploy node
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ cat ./config/cinder/ceph.conf
# minimal ceph.conf for 16e21856-6030-11ed-8a24-db02e0e54b22
[global]
fsid = 16e21856-6030-11ed-8a24-db02e0e54b22
mon_host = [v2:192.168.1.23:3300/0,v1:192.168.1.23:6789/0]
```
**__NOTE__**: remove the "tab" in the ceph.conf. Otherwise kolla-ansible would fail with error.
```
TASK [cinder : Copying over ceph.conf for Cinder]
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: oslo_config.iniparser.ParseError: at line 3, Unexpected continuation line: '\tfsid = 16e21856-6030-11ed-8a24-db02e0e54b22'
fatal: [nuc111]: FAILED! => {"msg": "Unexpected failure during module execution.", "stdout": ""}
```
```
[global]
fsid =
mon_host =
```
Copy ceph.conf and cinder keyring for Nova
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ mkdir -p ./config/nova
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ cp ./config/cinder/ceph.conf ./config/nova/
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ cp ./config/cinder/cinder-volume/ceph.client.cinder.keyring ./config/nova/
```
Check Ceph config files for kolla
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ tree ./config/
./config/
├── cinder
│ ├── ceph.conf
│ ├── cinder-backup
│ │ ├── ceph.client.cinder-backup.keyring
│ │ └── ceph.client.cinder.keyring
│ └── cinder-volume
│ └── ceph.client.cinder.keyring
├── nfs_shares
└── nova
├── ceph.client.cinder.keyring
└── ceph.conf
```
Config globals.yml
```
cinder_backend_ceph: "yes"
nova_backend_ceph: "yes"
ceph_cinder_keyring: "ceph.client.cinder.keyring"
ceph_cinder_user: "cinder"
ceph_cinder_pool_name: "volumes"
ceph_cinder_backup_keyring: "ceph.client.cinder-backup.keyring"
ceph_cinder_backup_user: "cinder-backup"
ceph_cinder_backup_pool_name: "backups"
ceph_nova_keyring: "{{ ceph_cinder_keyring }}"
ceph_nova_user: "{{ ceph_cinder_user }}"
ceph_nova_pool_name: "vms"
```
Apply configuration
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ kolla-ansible -i multinode deploy
```
Check cinder volume service status
```
root@nuc111:~# openstack volume service list
+------------------+--------------+------+---------+-------+----------------------------+
| Binary | Host | Zone | Status | State | Updated At |
+------------------+--------------+------+---------+-------+----------------------------+
| cinder-scheduler | nuc111 | nova | enabled | up | 2022-11-13T14:36:43.000000 |
| cinder-volume | nuc111@nfs-1 | nova | enabled | up | 2022-11-13T14:36:44.000000 |
| cinder-backup | nuc111 | nova | enabled | up | 2022-11-13T14:36:48.000000 |
| cinder-volume | nuc111@rbd-1 | nova | enabled | up | 2022-11-13T14:36:47.000000 |
+------------------+--------------+------+---------+-------+----------------------------+
```
The cinder-backup use NFS backup driver
```
root@nuc111:/etc/kolla/cinder-backup# cat cinder.conf | grep backup
backup_driver = cinder.backup.drivers.nfs.NFSBackupDriver
backup_mount_options =
backup_mount_point_base = /var/lib/cinder/backup
backup_share = nuc111:/kolla_nfs
backup_file_size = 327680000
```
Cinder volume has two backends enabled
```
root@nuc111:/etc/kolla/cinder-volume# cat cinder.conf | grep enabled_backends
enabled_backends = rbd-1,nfs-1
```
```
[rbd-1]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = rbd-1
rbd_pool = volumes
rbd_ceph_conf = /etc/ceph/ceph.conf
rados_connect_timeout = 5
rbd_user = cinder
rbd_secret_uuid = a8c5fb21-a2da-46a3-9275-2c3365bc3a12
report_discard_supported = True
[nfs-1]
volume_driver = cinder.volume.drivers.nfs.NfsDriver
volume_backend_name = nfs-1
nfs_shares_config = /etc/cinder/nfs_shares
nfs_snapshot_support = True
nas_secure_file_permissions = False
nas_secure_file_operations = False
```
Create volume types
```
root@nuc111:/etc/kolla/cinder-volume# openstack volume type create ceph
+-------------+--------------------------------------+
| Field | Value |
+-------------+--------------------------------------+
| description | None |
| id | f2c1ce62-d33f-4045-b94b-fce501961679 |
| is_public | True |
| name | ceph |
+-------------+--------------------------------------+
root@nuc111:/etc/kolla/cinder-volume# openstack volume type set ceph --property volume_backend_name=rbd-1
```
```
root@nuc111:/etc/kolla/cinder-volume# openstack volume type list --long
+--------------------------------------+-------------+-----------+---------------------+-----------------------------+
| ID | Name | Is Public | Description | Properties |
+--------------------------------------+-------------+-----------+---------------------+-----------------------------+
| f2c1ce62-d33f-4045-b94b-fce501961679 | ceph | True | None | volume_backend_name='rbd-1' |
| f9782ab2-931d-430b-92dd-b6810677c0e0 | __DEFAULT__ | True | Default Volume Type | |
+--------------------------------------+-------------+-----------+---------------------+-----------------------------+
```
Test VM creation with ceph volume type
```
root@nuc111:/etc/kolla/cinder-volume# openstack flavor list
+--------------------------------------+-----------+-------+------+-----------+-------+-----------+
| ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+--------------------------------------+-----------+-------+------+-----------+-------+-----------+
| 1 | m1.tiny | 512 | 1 | 0 | 1 | True |
| 2 | m1.small | 2048 | 20 | 0 | 1 | True |
| 3 | m1.medium | 4096 | 40 | 0 | 2 | True |
| 4 | m1.large | 8192 | 80 | 0 | 4 | True |
| 5 | m1.xlarge | 16384 | 160 | 0 | 8 | True |
| 8f99ab4a-e2de-493d-9940-37de21dfd623 | 1c2r4d | 2048 | 4 | 0 | 1 | False |
+--------------------------------------+-----------+-------+------+-----------+-------+-----------+
root@nuc111:/etc/kolla/cinder-volume# openstack keypair list
+-----------+-------------------------------------------------+
| Name | Fingerprint |
+-----------+-------------------------------------------------+
| key-pair1 | 31:a1:03:89:15:fb:5b:46:e7:b8:5a:94:8d:39:82:12 |
+-----------+-------------------------------------------------+
root@nuc111:/etc/kolla/cinder-volume# openstack image list
+--------------------------------------+-------------+--------+
| ID | Name | Status |
+--------------------------------------+-------------+--------+
| 1fa43af3-ac22-4dba-85e9-04b21807c5d4 | cirros | active |
| 40cbae5f-79ab-4c4f-ba97-fc97c6cd65f4 | ubuntu22.04 | active |
+--------------------------------------+-------------+--------+
root@nuc111:/etc/kolla/cinder-volume# openstack network list
+--------------------------------------+----------+--------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+----------+--------------------------------------+
| 17cb537a-88ac-42ed-9eb2-29177cb7490e | public1 | 142f17ca-c06d-4c07-abdc-dc935b1b83a3 |
| 67e89302-e5c7-4a17-ad1c-675b664874c1 | demo-net | 312eff74-d80c-4281-a51a-9b4bed5132c4 |
+--------------------------------------+----------+--------------------------------------+
```
Create a VM with "ephemeral" disk.
```
root@nuc111:~# openstack server create --flavor 1c2r4d --image ubuntu22.04 --key-name key-pair1 --network demo-net ubuntu_vm
+-------------------------------------+----------------------------------------------------+
| Field | Value |
+-------------------------------------+----------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | s5vdJaDqsBh7 |
| config_drive | |
| created | 2022-11-14T11:58:01Z |
| flavor | 1c2r4d (8f99ab4a-e2de-493d-9940-37de21dfd623) |
| hostId | |
| id | 1adcbf1f-0dd9-4086-a67f-ccf73d8b94e6 |
| image | ubuntu22.04 (40cbae5f-79ab-4c4f-ba97-fc97c6cd65f4) |
| key_name | key-pair1 |
| name | ubuntu_vm |
| progress | 0 |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2022-11-14T11:58:01Z |
| user_id | 6714a667d1e3406fab0c7add7d71d412 |
| volumes_attached | |
+-------------------------------------+----------------------------------------------------+
root@nuc111:~# openstack server show ubuntu_vm
+-------------------------------------+----------------------------------------------------------+
| Field | Value |
+-------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | nuc72 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | NUC7CJYHN2 |
| OS-EXT-SRV-ATTR:instance_name | instance-00000007 |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2022-11-14T11:59:04.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | demo-net=10.0.0.142 |
| config_drive | |
| created | 2022-11-14T11:58:01Z |
| flavor | 1c2r4d (8f99ab4a-e2de-493d-9940-37de21dfd623) |
| hostId | 6a619bdab198165b54faafe85d858871dff1042234d57e47742b0720 |
| id | 1adcbf1f-0dd9-4086-a67f-ccf73d8b94e6 |
| image | ubuntu22.04 (40cbae5f-79ab-4c4f-ba97-fc97c6cd65f4) |
| key_name | key-pair1 |
| name | ubuntu_vm |
| progress | 0 |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| properties | |
| security_groups | name='default' |
| status | ACTIVE |
| updated | 2022-11-14T11:59:04Z |
| user_id | 6714a667d1e3406fab0c7add7d71d412 |
| volumes_attached | |
+-------------------------------------+----------------------------------------------------------+
root@nuc111:~# rbd ls -l vms
NAME SIZE PARENT FMT PROT LOCK
1adcbf1f-0dd9-4086-a67f-ccf73d8b94e6_disk 2.2 GiB 2 excl
```
Test live migration
```
root@nuc111:~# openstack server migrate --live-migration ubuntu_vm
root@nuc111:~# openstack server show ubuntu_vm
+-------------------------------------+----------------------------------------------------------+
| Field | Value |
+-------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | nuc71 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | NUC7CJYHN1 |
| OS-EXT-SRV-ATTR:instance_name | instance-00000007 |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2022-11-14T11:59:04.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | demo-net=10.0.0.142 |
| config_drive | |
| created | 2022-11-14T11:58:01Z |
| flavor | 1c2r4d (8f99ab4a-e2de-493d-9940-37de21dfd623) |
| hostId | c7db1b2553c921575d60036b5d792993c9a242db70405cb52bc0bfea |
| id | 1adcbf1f-0dd9-4086-a67f-ccf73d8b94e6 |
| image | ubuntu22.04 (40cbae5f-79ab-4c4f-ba97-fc97c6cd65f4) |
| key_name | key-pair1 |
| name | ubuntu_vm |
| progress | 0 |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| properties | |
| security_groups | name='default' |
| status | ACTIVE |
| updated | 2022-11-14T12:33:46Z |
| user_id | 6714a667d1e3406fab0c7add7d71d412 |
| volumes_attached | |
+-------------------------------------+----------------------------------------------------------+
```
Create cinder volume on ceph
```
root@nuc111:~# openstack volume create --bootable --image ubuntu22.04 --type ceph --size 3 ubuntu22.04_boot_image
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2022-11-14T12:37:31.146975 |
| description | None |
| encrypted | False |
| id | 6c4f5912-4480-46cd-a960-f0a59a3f5d98 |
| migration_status | None |
| multiattach | False |
| name | ubuntu22.04_boot_image |
| properties | |
| replication_status | None |
| size | 3 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| type | ceph |
| updated_at | None |
| user_id | 6714a667d1e3406fab0c7add7d71d412 |
+---------------------+--------------------------------------+
root@nuc111:~# openstack volume list --long
+--------------------------------------+------------------------+-----------+------+------+----------+-------------+------------+
| ID | Name | Status | Size | Type | Bootable | Attached to | Properties |
+--------------------------------------+------------------------+-----------+------+------+----------+-------------+------------+
| 6c4f5912-4480-46cd-a960-f0a59a3f5d98 | ubuntu22.04_boot_image | available | 3 | ceph | true | | |
+--------------------------------------+------------------------+-----------+------+------+----------+-------------+------------+
root@nuc111:~# rbd ls -l volumes
NAME SIZE PARENT FMT PROT LOCK
volume-6c4f5912-4480-46cd-a960-f0a59a3f5d98 3 GiB 2
```
Create VM with the volume
```
root@nuc111:~# openstack server create --volume ubuntu22.04_boot_image --flavor 1c2r4d --key-name key-pair1 --network demo-net ubuntu22.04_ceph_volume_vm
+-------------------------------------+-----------------------------------------------+
| Field | Value |
+-------------------------------------+-----------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | vTNp7WWrhuFi |
| config_drive | |
| created | 2022-11-14T12:41:40Z |
| flavor | 1c2r4d (8f99ab4a-e2de-493d-9940-37de21dfd623) |
| hostId | |
| id | b7143bbd-f23e-42f9-9125-0b61592602f0 |
| image | |
| key_name | key-pair1 |
| name | ubuntu22.04_ceph_volume_vm |
| progress | 0 |
| project_id | 496d5b465e684f20a8217fa408728ac2 |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2022-11-14T12:41:40Z |
| user_id | 6714a667d1e3406fab0c7add7d71d412 |
| volumes_attached | |
+-------------------------------------+-----------------------------------------------+
root@nuc111:~# openstack server list
+--------------------------------------+----------------------------+--------+---------------------+-------------+--------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+----------------------------+--------+---------------------+-------------+--------+
| b7143bbd-f23e-42f9-9125-0b61592602f0 | ubuntu22.04_ceph_volume_vm | ACTIVE | demo-net=10.0.0.182 | | 1c2r4d |
| 1adcbf1f-0dd9-4086-a67f-ccf73d8b94e6 | ubuntu_vm | ACTIVE | demo-net=10.0.0.142 | ubuntu22.04 | 1c2r4d |
+--------------------------------------+----------------------------+--------+---------------------+-------------+--------+
```
#### Extra: Use local docker registry to deploy (**Not Work**)
Create a local docker registry on host nuc10 for Kolla-Ansible
**__NOTE__**: This does not work because kolla-ansible use "quay.io" to pull images and under "openstack.kolla" organization. However the docker registry **https://registry-1.docker.io** does not have "openstack.kolla". This results image pull failure.
Error message exampes:
```
root@nuc111:~# docker pull nuc10:5000/openstack.kolla/fluentd:master-rocky-9
Error response from daemon: manifest for nuc10:5000/openstack.kolla/fluentd:master-rocky-9 not found: manifest unknown: manifest unknown
```
```
docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=master-rocky-9&fromImage=nuc10%3A5000%2Fopenstack.kolla%2Ffluentd: Not Found (\"manifest for nuc10:5000/openstack.kolla/fluentd:master-rocky-9 not found: manifest unknown: manifest unknown
```
> There is an way to make it work by build kolla images manually and push to local registry.
> https://blog.csdn.net/weixin_40579389/article/details/124595609#t5
Create **/etc/registry** directory
```
root@NUC10:~# mkdir -p /etc/docker/registry
root@NUC10:~# cd /etc/docker/registry
root@NUC10:/etc/docker/registry#
```
Create **docker-compose.yml** file
```
root@NUC10:/etc/docker/registry# cat docker-compose.yml
version: '3'
services:
registry:
container_name: registry
image: registry:2
ports:
- "5000:5000"
environment:
REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data
volumes:
- /mnt/docker-registry-data:/data
- /etc/docker/registry/config.yml:/etc/docker/registry/config.yml
```
Create **config.yml** file
```
root@NUC10:/etc/docker/registry# cat ./config.yml
version: 0.1
log:
fields:
service: registry
storage:
cache:
blobdescriptor: inmemory
filesystem:
rootdirectory: /var/lib/registry
http:
addr: :5000
headers:
X-Content-Type-Options: [nosniff]
health:
storagedriver:
enabled: true
interval: 10s
threshold: 3
proxy:
remoteurl: https://registry-1.docker.io
```
Run docker registry container
```
root@NUC10:/etc/docker/registry# docker-compose up -d
Creating network "registry_default" with the default driver
Pulling registry (registry:2)...
2: Pulling from library/registry
ca7dd9ec2225: Pull complete
c41ae7ad2b39: Pull complete
1ed0fc8a6161: Pull complete
21df229223d2: Pull complete
626897ccab21: Pull complete
Digest: sha256:ce14a6258f37702ff3cd92232a6f5b81ace542d9f1631966999e9f7c1ee6ddba
Status: Downloaded newer image for registry:2
Creating registry ... done
```
Configure insecure-registries and registry-mirrors
```
root@NUC10:/etc/docker/registry# cat /etc/docker/daemon.json
{
"insecure-registries": ["nuc10:5000"],
"registry-mirrors": ["http://nuc10:5000"]
}
root@NUC10:/etc/docker/registry# systemctl restart docker
```
Test docker registry
```
root@NUC10:/etc/docker/registry# docker pull fedora
Using default tag: latest
latest: Pulling from library/fedora
cb8b1ed77979: Pull complete
Digest: sha256:f99efcddc4dd6736d8a88cc1ab6722098ec1d77dbf7aed9a7a514fc997ca08e0
Status: Downloaded newer image for fedora:latest
docker.io/library/fedora:latest
root@NUC10:/etc/docker/registry# du -sh /mnt/docker-registry-data/docker/registry/v2/
64M /mnt/docker-registry-data/docker/registry/v2/
```
**__NOTE__**: docker will query configured registry mirror which trigger mirror download image. You should see image data usage increased in "/mnt/docker-registry-data/docker/registry/v2/blobs/" )
If test success, configure **/etc/docker/daemon.json** on all OpenStack nodes. For example on controller host:
```
ubuntu@nuc111:~$ cat /etc/docker/daemon.json
{
"bridge": "none",
"ip-forward": false,
"iptables": false,
"insecure-registries" : ["nuc10:5000"],
"registry-mirrors": ["http://nuc10:5000"]
}
ubuntu@nuc111:~$ sudo systemctl restart docker
```
Edit **globals.yml**
```
docker_registry: nuc10:5000
docker_registry_insecure: "yes"
```
Run kolla-ansible
```
(kolla-ansible-venv) ycheng@NUC10:/etc/kolla$ kolla-ansible -i multinode deploy
```
**__NOTE__**: Do not run kolla-ansible precheck, it removes "registry-mirrors" in docker daemon config.
**__NOTE__**: We have to set "insecure-registries" because we do not use https.
#### Troubleshoot
Error when use local docker registry.
```
RUNNING HANDLER [common : Restart fluentd container] ****************************************************************************************
fatal: [nuc111]: FAILED! => {"changed": true, "msg": "'Traceback (most recent call last):\\n File \"/usr/local/lib/python3.8/dist-packages/docker/api/client.py\", line 268, in _raise_for_status\\n response.raise_for_status()\\n File \"/usr/lib/python3/dist-packages/requests/models.py\", line 940, in raise_for_status\\n raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/images/create?tag=master-rocky-9&fromImage=nuc10%3A4000%2Fopenstack.kolla%2Ffluentd\\n\\nDuring handling of the above exception, another exception occurred:\\n\\nTraceback (most recent call last):\\n File \"/tmp/ansible_kolla_docker_payload_hl3f6zvw/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 381, in main\\n File \"/tmp/ansible_kolla_docker_payload_hl3f6zvw/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 660, in recreate_or_restart_container\\n self.pull_image()\\n File \"/tmp/ansible_kolla_docker_payload_hl3f6zvw/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 450, in pull_image\\n json.loads(line.strip().decode(\\'utf-8\\')) for line in self.dc.pull(\\n File \"/usr/local/lib/python3.8/dist-packages/docker/api/image.py\", line 430, in pull\\n self._raise_for_status(response)\\n File \"/usr/local/lib/python3.8/dist-packages/docker/api/client.py\", line 270, in _raise_for_status\\n raise create_api_error_from_http_exception(e)\\n File \"/usr/local/lib/python3.8/dist-packages/docker/errors.py\", line 31, in create_api_error_from_http_exception\\n raise cls(e, response=response, explanation=explanation)\\ndocker.errors.APIError: 500 Server Error for http+docker://localhost/v1.41/images/create?tag=master-rocky-9&fromImage=nuc10%3A4000%2Fopenstack.kolla%2Ffluentd: Internal Server Error (\"Get \"https://nuc10:4000/v2/\": http: server gave HTTP response to HTTPS client\")\\n'"}
```
```
fatal: [nuc72]: FAILED! => {"changed": true, "msg": "'Traceback (most recent call last):\\n File \"/usr/local/lib/python3.8/dist-packages/docker/api/client.py\", line 268, in _raise_for_status\\n response.raise_for_status()\\n File \"/usr/lib/python3/dist-packages/requests/models.py\", line 940, in raise_for_status\\n raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/images/create?tag=master-rocky-9&fromImage=nuc10%3A4000%2Fopenstack.kolla%2Ffluentd\\n\\nDuring handling of the above exception, another exception occurred:\\n\\nTraceback (most recent call last):\\n File \"/tmp/ansible_kolla_docker_payload_h4_q1cuz/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 381, in main\\n File \"/tmp/ansible_kolla_docker_payload_h4_q1cuz/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 660, in recreate_or_restart_container\\n self.pull_image()\\n File \"/tmp/ansible_kolla_docker_payload_h4_q1cuz/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 450, in pull_image\\n json.loads(line.strip().decode(\\'utf-8\\')) for line in self.dc.pull(\\n File \"/usr/local/lib/python3.8/dist-packages/docker/api/image.py\", line 430, in pull\\n self._raise_for_status(response)\\n File \"/usr/local/lib/python3.8/dist-packages/docker/api/client.py\", line 270, in _raise_for_status\\n raise create_api_error_from_http_exception(e)\\n File \"/usr/local/lib/python3.8/dist-packages/docker/errors.py\", line 31, in create_api_error_from_http_exception\\n raise cls(e, response=response, explanation=explanation)\\ndocker.errors.APIError: 500 Server Error for http+docker://localhost/v1.41/images/create?tag=master-rocky-9&fromImage=nuc10%3A4000%2Fopenstack.kolla%2Ffluentd: Internal Server Error (\"Get \"https://nuc10:4000/v2/\": x509: certificate relies on legacy Common Name field, use SANs instead\")\\n'"}
```
```
fatal: [nuc111]: FAILED! => {"changed": true, "msg": "'Traceback (most recent call last):\\n File \"/usr/local/lib/python3.8/dist-packages/docker/api/client.py\", line 268, in _raise_for_status\\n response.raise_for_status()\\n File \"/usr/lib/python3/dist-packages/requests/models.py\", line 940, in raise_for_status\\n raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/create?tag=master-rocky-9&fromImage=nuc10%3A5000%2Fopenstack.kolla%2Ffluentd\\n\\nDuring handling of the above exception, another exception occurred:\\n\\nTraceback (most recent call last):\\n File \"/tmp/ansible_kolla_docker_payload_s36ljb33/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 381, in main\\n File \"/tmp/ansible_kolla_docker_payload_s36ljb33/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 651, in recreate_or_restart_container\\n self.start_container()\\n File \"/tmp/ansible_kolla_docker_payload_s36ljb33/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 669, in start_container\\n self.pull_image()\\n File \"/tmp/ansible_kolla_docker_payload_s36ljb33/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 450, in pull_image\\n json.loads(line.strip().decode(\\'utf-8\\')) for line in self.dc.pull(\\n File \"/usr/local/lib/python3.8/dist-packages/docker/api/image.py\", line 430, in pull\\n self._raise_for_status(response)\\n File \"/usr/local/lib/python3.8/dist-packages/docker/api/client.py\", line 270, in _raise_for_status\\n raise create_api_error_from_http_exception(e)\\n File \"/usr/local/lib/python3.8/dist-packages/docker/errors.py\", line 31, in create_api_error_from_http_exception\\n raise cls(e, response=response, explanation=explanation)\\ndocker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=master-rocky-9&fromImage=nuc10%3A5000%2Fopenstack.kolla%2Ffluentd: Not Found (\"manifest for nuc10:5000/openstack.kolla/fluentd:master-rocky-9 not found: manifest unknown: manifest unknown\")\\n'"}
```
#### kolla-ansible cli
```
kolla-ansible -i INVENTORY deploy is used to deploy and start all Kolla containers.
kolla-ansible -i INVENTORY destroy is used to clean up containers and volumes in the cluster.
kolla-ansible -i INVENTORY mariadb_recovery is used to recover a completely stopped mariadb cluster.
kolla-ansible -i INVENTORY prechecks is used to check if all requirements are meet before deploy for each of the OpenStack services.
kolla-ansible -i INVENTORY post-deploy is used to do post deploy on deploy node to get the admin openrc file.
kolla-ansible -i INVENTORY pull is used to pull all images for containers.
kolla-ansible -i INVENTORY reconfigure is used to reconfigure OpenStack service.
kolla-ansible -i INVENTORY upgrade is used to upgrades existing OpenStack Environment.
kolla-ansible -i INVENTORY stop is used to stop running containers.
kolla-ansible -i INVENTORY deploy-containers is used to check and if necessary update containers, without generating configuration.
kolla-ansible -i INVENTORY prune-images is used to prune orphaned Docker images on hosts.
kolla-ansible -i INVENTORY genconfig is used to generate configuration files for enabled OpenStack services, without then restarting the containers so it is not applied right away.
kolla-ansible -i INVENTORY1 -i INVENTORY2 ... Multiple inventories can be specified by passing the --inventory or -i command line option multiple times. This can be useful to share configuration between multiple environments. Any common configuration can be set in INVENTORY1 and INVENTORY2 can be used to set environment specific details.
kolla-ansible -i INVENTORY gather-facts is used to gather Ansible facts, for example to populate a fact cache.
```
#### Todo
- Swift storage
#### Reference
```
- https://docs.openstack.org/kolla-ansible/latest/user/multinode.html
- https://docs.openstack.org/project-deploy-guide/kolla-ansible/queens/multinode.html
- https://www.packetcoders.io/openstack-installation-via-kolla-anisble/
- https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html
- https://zhuanlan.zhihu.com/p/143327139
- https://cdn.igene.tw/kolla-ansible-deploy
- https://docs.openstack.org/kolla-ansible/queens/reference/cinder-guide.html
- https://docs.openstack.org/python-openstackclient/pike/configuration/index.html
- https://docs.openstack.org/kolla-ansible/latest/user/multinode.html
- https://abayard.com/openstack-kolla-deploy-external-ceph-ansible/
- https://docs.openstack.org/cinder/latest/admin/default-volume-types.html
- https://www.jianshu.com/p/56717aca4ab2
- https://www.linuxjournal.com/content/build-versatile-openstack-lab-kolla
- https://ithelp.ithome.com.tw/articles/10271837
- https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html
- https://techglimpse.com/why-does-openstack-report-hypervisor-type-as-qemu-when-libvirt_type-is-kvm/
```
Sign in with Wallet
Connect another wallet