Zadara Storage Clouds offer customers our on premise OPaaS "zStorage" platform. This comprises of several storage nodes each containing multiple drives. Customers can request drive types and capacities specific to their use-case and our Solutions teams work with customers to deliver the most optimal Storage Solution. The zStorage Platform allows customers to create VPSA and VPSA Object Instances (ZOS). VPSA being our Virtual Private Storage Array aimed at delivering NAS/SAN storage and our ZOS providing S3/Swift compatible object storage.
From time to time we release a new version of our zStorage software and this document covers how we update a Zadara Storage Cloud (zStorage).
Note: we will not cover VPSA upgrades here; our VPSA upgrade process explained article covers that topic, but a general rule is that the Storage Cloud is first upgraded prior to applying VPSA updates.
Storage Cloud Overview
A typical OPaaS storage cloud comprises 2 or more server class systems - each with multiple drive slots and potentially a SAS cable connection to a supplementary JBOD(drive array). High speed networking devices and a firewall with integral management switch or dedicated firewall and management switches can be provided. Another option is that customers use their own high speed networking devices if they meet required specifications.
Solutions are available for smaller installations through to larger enterprise customers across multiple sites "Multizone". Customers can locate this cloud hardware in their own premises and/or a data centre of their choice.
Upgrade Steps
The above described "cloud storage" will have first been installed with the current release at the time it was shipped. In time though, as enhancements are added to the product or problems are fixed there is a need to upgrade the "cloud" to a new release.
Briefly an upgrade involves the below (however please read the whole article to ensure you are aware of all the pre-requisites);
- download of the latest software release from our online repositories
(for "offline" clouds an upload of the packages is necessary) - registration of the new release.
- triggering an automated cloud upgrade
The upgrade process can be performed by customers ie:Storage Admins, but typically Zadara Operations will be tasked with applying upgrades at a suitable date and time, Operations will perform the above tasks to ensure the upgrade can proceed. It's worthwhile asking Zadara Operations/Support that an upgrade can be applied and requirements are met, also ensure that Zadara Support are forewarned if you are scheduling a Cloud Upgrade so that they are aware of the "planned work".
Also, your account must have the "Admin" role within the cloud to be permitted to trigger a cloud upgrade.
Diagram 1 - Cloud Packages Management option
Diagram 2 - software package download options
Diagram 3 - register a downloaded software package
Diagram 4 - start of the cloud upgrade procedure
During the upgrade process each storage node is upgraded individually with the new release. The 2 Cluster Nodes (1 x active and 1 x standby) are upgraded first followed by Storage Nodes that do not provide clustered services. The standby cluster node is upgraded first and then we promote this to active, the ex-active cluster node is then upgraded. The remaining storage Nodes are then upgraded.
Diagram 5 - upgrade order for this cloud will be as shown, CCslave, CCMaster(after failover), then remaining Storage Nodes
Throughout this upgrade process, the physical nodes themselves remain active and the underlying VPSAs/ZOS "Storage Instances" remain online and delivering IO. Each node can be hosting multiple drives that any VPSA or VPSA Object Instances requires use of so this resource must remain accessible throughout.
If an automated cloud upgrade has been chosen then the upgrade order is Command Centre nodes, storage nodes and then the Command Centre Dashboard itself; there is a brief 5 minute period where the Command Centre dashboard ( due to the active Cluster Node failover ) is temporarily unavailable, again this has no impact on the physical Nodes and the underlying Storage Instances.
Items that we target for a "Cloud Upgrade" are;
- Storage Node Core
- Storage Node utilities
- Command Centre
Once the Storage Node and Command Centre show the new upgraded release version and the Clouds Central log confirms the upgrade successfull the upgrade is complete!
Manual Upgrade
It's preferred to use the automated Cloud Upgrade option; usually if a manual upgrade is required then there is a reason for this and extra steps may be required so Zadara Support and Operations are best to apply a manual upgrade. However, the same basic process mentioned above is followed.
Caveats
| Online cloud | an "online cloud" will have internet capabilities, this makes it much quicker to upload the latest software updates from our online repositories. |
| Offline cloud | an offline cloud will not have internet access. Access to the online Zadara software update repositories tends to be complicated and new packages will need to be manually transferred into the cloud, possibly in several hops. This can delay upgrade so it should be planned beforehand and sufficient time allowed. Many of the download packages are several GiB in size and with slow access links it can take several hours to complete one full package upload. |
| Multizone cloud | Some customers elect to split the nodes between 2 locations (low latency is mandated). This provides an extra safety aspect, but does require additional networking and complexity. Upgrades to these clouds are performed in the same way as a single zone cloud. |
| Upgrading more than 2 major releases | If upgrading from an older release that is more than 2 major releases behind, then we must upgrade in steps to ensure critical changes are applied. |
| Post upgrade requirements | Some upgrades will trigger an extra update requirement in the Command Centre Dashboard UI. These tend to be in the form of a comment/tag on the Node and request a reset/reboot; to allow a new driver or firmware to be applied and then cut-over to. It's possible to work this additional task into a scheduled upgrade along with VPSA/ZOS upgrade works to mitigate the number of failovers required. Not all cloud upgrades have this conditional need. |
| Update Packages | in order to upgrade, the target package must be loaded to the cloud and registered. |
Problems
If an upgrade stalls or errors then we recommend contacting the Zadara Support team. In some cases the issue may relate to pending actions which must complete first.
A cloud upgrade should only be started when all Storage Instances are in a Normal state, ZOS and VPSAs in Degraded states or non-Normal status will block the upgrade.
Storage Nodes must have VC and Drive scheduling resource services Enabled and running to allow the upgrade process to progress.
Final Word
We strongly recommend that customers remain within 2 major releases of our latest software release to ensure that they stay on a fully supported release.
Details of the latest release and our Release Notes for each release are available online, see zStorage Latest Release for more details.