Skip to content

Add upgrade documentation for XOSTOR #319

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add upgrade documentation for XOSTOR #319

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 120 additions & 0 deletions docs/xostor/xostor.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -323,6 +323,126 @@ Only XCP-ng 8.2.1 is currently supported and with a maximum of 7 machines per po

See this documentation: [RPU](/management/updates/#rolling-pool-update-rpu).

## Upgrade

If you are reading this documentation, we assume that you want to upgrade a pool on which XOSTOR is deployed, i.e. change the version of XCP-ng, for example from 8.2 to 8.3.
For updates that don't change the version numbers (bugfixes, security fixes), see [the update section](#update).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"updates that don't change the version numbers" is a strange concept, this might need rewording

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"updates that don't change the version number of XCP-ng" ?


### 1. Prerequisites

- All hosts must be up to date on the version of XCP-ng you are currently using. For this refer to [the update section](#update).
- HA must be disabled on your pool.
- Ensure all nodes are reachable and resources are in "OK" state via XO's XOSTOR view. Alternatively, you can use the CLI:
```
linstor n list
linstor r list
linstor adv r
```

### 2. Information to retrieve before upgrading
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to keep this section once we have an ISO that makes the process simpler?


1. Note the UUID of the XOSTOR SR you are using on your pool.

2. Before starting the upgrade procedure, you must also retrieve the `group-name` used by the XOSTOR SR of your pool via this command (replace `<UUID>` with the correct one):
```
xe pbd-list sr-uuid=<UUID>
```

Example output where the group-name is `linstor_group/thin_device`:
```
uuid ( RO) : 06d10e9e-c7ad-2ed6-a901-53ac1c2c7486
host-uuid ( RO): 4bac16be-b25b-4d0b-a159-8f5bda930640
sr-uuid ( RO): d5f990f6-abca-0ebf-8582-b7e55901fb50
device-config (MRO): group-name: linstor_group/thin_device; redundancy: 2; provisioning: thin
currently-attached ( RO): true


uuid ( RO) : 06b5e263-8ec1-74e9-3162-d39785be6ba7
host-uuid ( RO): f7737f79-ad49-491c-a303-95ac37fb6a13
sr-uuid ( RO): d5f990f6-abca-0ebf-8582-b7e55901fb50
device-config (MRO): group-name: linstor_group/thin_device; redundancy: 2; provisioning: thin
currently-attached ( RO): true


uuid ( RO) : 1d872d5b-fb60-dbd7-58fc-555a211f18fa
host-uuid ( RO): ef942670-e37d-49e6-81d0-d2a484b0cd10
sr-uuid ( RO): d5f990f6-abca-0ebf-8582-b7e55901fb50
device-config (MRO): group-name: linstor_group/thin_device; redundancy: 2; provisioning: thin
currently-attached ( RO): true
```

### 3. Upgrade

From this point we can proceed to upgrade your XOSTOR-enabled pool.

An upgrade can take quite a long time so we recommend disabling the auto-evict mechanism during this procedure to avoid bad behavior.
On the host where the controller is running:
```
linstor controller set-property DrbdOptions/AutoEvictAllowEviction False
```

For each host of the pool (starting with the master), we break down the different steps as follows:

1. Update the host following the instructions in [this guide](../installation/upgrade/#-upgrade-via-installation-iso-recommended), but do not proceed to the next host yet.

2. After rebooting the host, you will need to run specific commands to have a working LINSTOR node.

Install the LINSTOR RPMs:
```
yum install -y xcp-ng-release-linstor
yum install -y xcp-ng-linstor
```

Restart the toolstack to detect the LINSTOR driver:
```
xe-toolstack-restart
```

Then finally open the ports and enable the LINSTOR services using the `<GROUP_NAME>` and `<HOST_UUID>` obtained above:
```
xe host-call-plugin host-uuid=<HOST_UUID> plugin=linstor-manager fn=addHost args:groupName=<GROUP_NAME>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it wise to wait a little after the toolstack restart, or maybe to grep the logs for the line which indicates that startup has completed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say that waiting for the previous command to return is enough to use the plugins API.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't that the job of xapi-wait-init-complete?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first sight it would look like a better idea to get the installer to install the extra packages with the rest of the system, rather than doing that after installation, so:

  • the rpm can be obtained from the same repo as the rest of the system
  • no need to mess with a XAPI restart

Or is there any problem with this? Making sure we have a properly-compatible linstor version?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a problem with that. In this case, we can wait for XAPI to start, and then calling the plugin is enough.
We just need to be sure to have installed in 8.3 the same RPMs of LINSTOR controller/satellite as the hosts in 8.2.

```

Find where the controller is running on your pool and check the status of the nodes:
```
linstor n list
```

The node can be marked `OFFLINE` or `EVICTED` (if the auto-evict mechanism is still active):
```
╭───────────────────────────────────────────────────────────────────────────────────────────╮
┊ Node ┊ NodeType ┊ Addresses ┊ State ┊
╞═══════════════════════════════════════════════════════════════════════════════════════════╡
┊ node01 ┊ COMBINED ┊ 10.1.0.60:3366 (PLAIN) ┊ EVICTED ┊
┊ node02 ┊ COMBINED ┊ 10.1.0.61:3366 (PLAIN) ┊ Online ┊
┊ node03 ┊ COMBINED ┊ 10.1.0.62:3366 (PLAIN) ┊ Online ┊
╰───────────────────────────────────────────────────────────────────────────────────────────╯
```

In this situation, you can restore the node:
```
linstor node restore node01
```

Wait a little while and check if there is an issue with the resources:
```
linstor r list
linstor advise r # Give possible fix commands in case of problems.
```

Finally check in XOA that the PBD of the SR of this host is connected. If not, do so.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can it be not connected?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't plug the PBD without the LINSTOR packages (before this point it was not the case), so I guess most of the time you have to manually connect the PBD unless the XAPI tries again on its own, which to my knowledge is not the case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this won't be necessary if the installer takes care of the packages before we reboot, right?
What will be left then?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You must be sure to execute addHost before the initial PBD plug. Otherwise, the master will not be able to plug in properly (as a reminder, each SR during the master PBD attach calls runs a scan).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You must be sure to execute addHost before the initial PBD plug

Oh, that's somewhat "persisting a linstor state", right?


:::warning
Very important, if you don't want to break the quorum or your production environment. You must execute the commands given above after upgrading a host and do not reboot/upgrade the others until the host's satellite is operational and its PBD is plugged.
:::

### 4. After pool upgrade

If you have deactivated auto eviction as recommended, it's necessary to reactivate it. On the host where the controller resides, execute this command:
```
linstor controller set-property DrbdOptions/AutoEvictAllowEviction True
```

## Global questions

### The linstor command does not work!?
Expand Down