From 58299e2e06640457e6d3939fe7797e865be19ace Mon Sep 17 00:00:00 2001 From: Jan Orel Date: Wed, 29 Jul 2026 16:08:21 +0200 Subject: [PATCH] B #7899: Document the fixed remote scripts directory The remotes are copied to /var/lib/one-remotes on the hosts, the monitoring DB lives in /var/lib/one-remotes_db and the tproxy command moved to /usr/lib/one/sbin. Cover the upgrade implications. --- .../resource_monitoring/forecast.md | 4 ++-- .../resource_monitoring/monitoring_system.md | 4 ++-- .../cluster_configuration/hosts_and_clusters/hosts.md | 4 ++-- content/product/cluster_configuration/lvm/filemode.md | 2 +- content/product/cluster_configuration/lvm/lvm.md | 2 +- .../product/cluster_configuration/san_storage/everpure.md | 2 +- .../product/cluster_configuration/san_storage/netapp.md | 2 +- .../infrastructure_drivers_development/devel-nm.md | 2 +- .../infrastructure_drivers_development/sd.md | 2 +- .../system_interfaces/hook_driver.md | 2 +- .../opennebula_services_configuration/oned.md | 3 --- .../opennebula_services_configuration/troubleshooting.md | 2 +- .../virtual_machines_networking/tproxy.md | 8 ++++---- .../upgrade_process/upgrade_guide/upgrading_single.md | 6 ++++++ 14 files changed, 24 insertions(+), 21 deletions(-) diff --git a/content/product/cloud_system_administration/resource_monitoring/forecast.md b/content/product/cloud_system_administration/resource_monitoring/forecast.md index 65f5ac54c..28341d11b 100644 --- a/content/product/cloud_system_administration/resource_monitoring/forecast.md +++ b/content/product/cloud_system_administration/resource_monitoring/forecast.md @@ -63,7 +63,7 @@ By default, short-term forecasts cover the next five minutes. ## Forecast Generation -Forecasts rely on OpenNebula’s built-in monitoring system ([The OpenNebula Monitoring System]({{% relref "product/cloud_system_administration/scheduler/configuration/#monitor-alert-configuration" %}})). Periodically, monitoring probes analyze real-time CPU, memory, disk, and network metrics to generate predictions. Each Host maintains a local time-series database (`/var/tmp/one_db/host.db`) storing historical data for forecasting. Similarly, every VM has a dedicated database (`/var/tmp/one_db/.db`) that tracks its individual metrics. +Forecasts rely on OpenNebula’s built-in monitoring system ([The OpenNebula Monitoring System]({{% relref "product/cloud_system_administration/scheduler/configuration/#monitor-alert-configuration" %}})). Periodically, monitoring probes analyze real-time CPU, memory, disk, and network metrics to generate predictions. Each Host maintains a local time-series database (`/var/lib/one-remotes_db/host.db`) storing historical data for forecasting. Similarly, every VM has a dedicated database (`/var/lib/one-remotes_db/.db`) that tracks its individual metrics. {{< alert title="Note" type="info" >}} When a VM migrates, a new database is created on the destination Host, temporarily reducing forecast accuracy until sufficient data accumulates.{{< /alert >}} @@ -122,7 +122,7 @@ Forecast database size depends on retention and monitoring frequency: To optimize storage: - Adjust retention based on available capacity. -- Monitor `/var/tmp/one_db/` usage. +- Monitor `/var/lib/one-remotes_db/` usage. - Tailor settings to workload cycles (daily, weekly, monthly). Changes take effect immediately, without requiring service restarts. diff --git a/content/product/cloud_system_administration/resource_monitoring/monitoring_system.md b/content/product/cloud_system_administration/resource_monitoring/monitoring_system.md index e493246e9..bf90d3af5 100644 --- a/content/product/cloud_system_administration/resource_monitoring/monitoring_system.md +++ b/content/product/cloud_system_administration/resource_monitoring/monitoring_system.md @@ -120,12 +120,12 @@ The monitoring data collected by OpenNebula probes is processed by the monitorin Each physical Host in an OpenNebula deployment maintains its own dedicated monitoring databases. These databases are updated through the regular Host and VM monitoring cycles: -- **Location**: `/var/tmp/one_db/host.db` +- **Location**: `/var/lib/one-remotes_db/host.db` - **Purpose**: Stores historical monitoring metrics for the Host Additionally, for each VM running on a Host, a dedicated database tracks its specific metrics: -- **Location**: `/var/tmp/one_db/.db` (stored on the Host where the VM is running) +- **Location**: `/var/lib/one-remotes_db/.db` (stored on the Host where the VM is running) - **Purpose**: Stores historical monitoring metrics for the specific VM {{< alert title="Note" type="info" >}} diff --git a/content/product/cluster_configuration/hosts_and_clusters/hosts.md b/content/product/cluster_configuration/hosts_and_clusters/hosts.md index 19baaba7b..4cfc9e0f2 100644 --- a/content/product/cluster_configuration/hosts_and_clusters/hosts.md +++ b/content/product/cluster_configuration/hosts_and_clusters/hosts.md @@ -258,7 +258,7 @@ This tag can be used at a later time for scheduling purposes, [see more details ### Updating Host Files -When OpenNebula monitors a Host it copies driver files to `/var/tmp/one`. When these files are updated they need to be copied again to the Hosts with the `sync` command. To keep track of the probes version there’s a file in `/var/lib/one/remotes/VERSION`. By default this holds the OpenNebula version (e.g., ‘7.0.0’). This version can be seen in the Hosts by using `onehost show `: +When OpenNebula monitors a Host it copies driver files to `/var/lib/one-remotes`. When these files are updated they need to be copied again to the Hosts with the `sync` command. To keep track of the probes version there’s a file in `/var/lib/one/remotes/VERSION`. By default this holds the OpenNebula version (e.g., ‘7.0.0’). This version can be seen in the Hosts by using `onehost show `: ```shell onehost show 0 @@ -321,7 +321,7 @@ Each PCI device reported by the monitor contains the following attributes: The `IFNAME` attribute is resolved using multiple methods, following this precedence (first match is used): 1. **udev rules** — reads `/etc/udev/rules.d/99-rename.rules`, looking for specially formatted comments. This file is automatically generated by [OneDeploy]({{% relref "../../../solutions/ai_factory_blueprints/deployment/cd_on-premises.md" %}}) and is not intended to be edited manually. It is used to assign persistent names to network interfaces, either for convenience (custom names) or to prevent losing the standard name after VFIO takes control of the device. -2. **File cache** — consults `/var/tmp/one_db/pci_net_names/`, where each file (named as the PCI address) contains the interface name. Created automatically on first resolution. +2. **File cache** — consults `/var/lib/one-remotes_db/pci_net_names/`, where each file (named as the PCI address) contains the interface name. Created automatically on first resolution. 3. **Direct sysfs lookup** — reads the directory `/sys/bus/pci/devices//net/` to find the kernel interface name. The cache is cleared when the monitoring daemon restarts. diff --git a/content/product/cluster_configuration/lvm/filemode.md b/content/product/cluster_configuration/lvm/filemode.md index eba52a4f6..b62c4dd68 100644 --- a/content/product/cluster_configuration/lvm/filemode.md +++ b/content/product/cluster_configuration/lvm/filemode.md @@ -36,7 +36,7 @@ The LVM Datastore does not need CLVM configured in your cluster. The drivers ref In case of rebooting the virtualization Host, the volumes need to be activated to have them available for the hypervisor again. There are two possibilities: * If the [node package]({{% relref "kvm_node_installation#kvm-node" %}}) is installed, they will be automatically activated by the `/etc/cron.d/opennebula-node` cron file. -* Otherwise, manual activation will be required. For each volume device of the Virtual Machines running on the Host before the reboot, run `lvchange -K -ay $DEVICE`. You can also run on the Host the activation script `/var/tmp/one/tm/fs_lvm_ssh/activate`, located in the remote scripts. +* Otherwise, manual activation will be required. For each volume device of the Virtual Machines running on the Host before the reboot, run `lvchange -K -ay $DEVICE`. You can also run on the Host the activation script `/var/lib/one-remotes/tm/fs_lvm_ssh/activate`, located in the remote scripts. Virtual Machine disks are symbolic links to the block devices. However, additional VM files like checkpoints or deployment files are stored under `/var/lib/one/datastores/`. Be sure that enough local space is present. diff --git a/content/product/cluster_configuration/lvm/lvm.md b/content/product/cluster_configuration/lvm/lvm.md index 0ec784320..58d968b73 100644 --- a/content/product/cluster_configuration/lvm/lvm.md +++ b/content/product/cluster_configuration/lvm/lvm.md @@ -39,7 +39,7 @@ The LVM Datastore does not need CLVM configured in your cluster. The drivers ref In case of rebooting the virtualization Host, the volumes need to be activated to have them available for the hypervisor again. There are two possibilities: * If the [node package]({{% relref "kvm_node_installation#kvm-node" %}}) is installed, they will be automatically activated by the `/etc/cron.d/opennebula-node` cron file. -* Otherwise, manual activation will be required. For each volume device of the Virtual Machines running on the Host before the reboot, run `lvchange -K -ay $DEVICE`. You can also run on the Host the activation script `/var/tmp/one/tm/lvm/activate`, located in the remote scripts. +* Otherwise, manual activation will be required. For each volume device of the Virtual Machines running on the Host before the reboot, run `lvchange -K -ay $DEVICE`. You can also run on the Host the activation script `/var/lib/one-remotes/tm/lvm/activate`, located in the remote scripts. Virtual Machine disks are symbolic links to the block devices. However, additional VM files like checkpoints or deployment files are stored under `/var/lib/one/datastores/`. To prevent filling local disks, allocate plenty of space for these files. diff --git a/content/product/cluster_configuration/san_storage/everpure.md b/content/product/cluster_configuration/san_storage/everpure.md index 2f9747d06..9d03f4ddd 100644 --- a/content/product/cluster_configuration/san_storage/everpure.md +++ b/content/product/cluster_configuration/san_storage/everpure.md @@ -260,7 +260,7 @@ Both Full and Incremental backups are supported by Everpure FlashArray. For Full Incremental backups are created using the Volume Difference Feature of Everpure FlashArray. This returns a list of block offsets and lengths which have changed since a target snapshot. This list is then used to create a sparse QCOW2 format file which is uploaded to the backup datastore. {{< alert title="Note" type="success" >}} -You can configure the block size ( Default and minimum 4096 B / 4 KB ) for incremental backups by modifying the file at `/var/tmp/one/etc/tm/san/backup.conf` +You can configure the block size ( Default and minimum 4096 B / 4 KB ) for incremental backups by modifying the file at `/var/lib/one-remotes/etc/tm/san/backup.conf` {{< /alert >}} {{< alert title="Warning" type="warning" >}} diff --git a/content/product/cluster_configuration/san_storage/netapp.md b/content/product/cluster_configuration/san_storage/netapp.md index a05000893..1d4e78db4 100644 --- a/content/product/cluster_configuration/san_storage/netapp.md +++ b/content/product/cluster_configuration/san_storage/netapp.md @@ -275,7 +275,7 @@ Both Full and Incremental backups are supported by NetApp. For Full Backups, a s Incremental backups are created by first creating the base full backup from the snapshot however this snapshot is then retained on the NetApp Volume rather than deleted after the backup is taken. When another incremental backup is taken, a new snapshot is taken and both the previous and current snapshots are cloned to new Volumes where they are attached to the host and compared for differences at the block level. These block changes are stored in a sparse QCOW2 file backed by the previous snapshot, which is then uploaded to the backup datastore. The old snapshot is then removed while the new one is retained. When incremental backups are restored, the backing chain is rebuilt before restoring the backup to the VM disk. {{< alert title="Note" type="info" >}} -You can configure the block size ( Defualt 2097152 B / 2 MB ) for incremental backups by modifing the file at `/var/tmp/one/etc/tm/san/backup.conf` +You can configure the block size ( Defualt 2097152 B / 2 MB ) for incremental backups by modifing the file at `/var/lib/one-remotes/etc/tm/san/backup.conf` {{< /alert >}} {{< alert title="Warning" type="warning" >}} diff --git a/content/product/integration_references/infrastructure_drivers_development/devel-nm.md b/content/product/integration_references/infrastructure_drivers_development/devel-nm.md index 60f57a38f..83861ac9f 100644 --- a/content/product/integration_references/infrastructure_drivers_development/devel-nm.md +++ b/content/product/integration_references/infrastructure_drivers_development/devel-nm.md @@ -72,7 +72,7 @@ Default driver actions support the execution of hooks after the main action is s For example, this is the directory tree of the bridge driver synced to a virtualization node with some custom scripts: ```default -root@ubuntu1804-local-6ee11-2:/var/tmp/one/vnm/bridge# tree ./ +root@ubuntu1804-local-6ee11-2:/var/lib/one-remotes/vnm/bridge# tree ./ ./ ├── clean ├── clean.d diff --git a/content/product/integration_references/infrastructure_drivers_development/sd.md b/content/product/integration_references/infrastructure_drivers_development/sd.md index 1f94fd434..2c7889d47 100644 --- a/content/product/integration_references/infrastructure_drivers_development/sd.md +++ b/content/product/integration_references/infrastructure_drivers_development/sd.md @@ -354,7 +354,7 @@ The plugins /monitor_ds + kvm-probes.d/monitor_ds.sh will report the inf ### The monitor_ds script. -The monitor_ds.sh probe from the IM, if the `.monitor` file is present (e.g., `/var/lib/one/datastores/100/.monitor`), will execute its contents in the form `/var/tmp/one/remotes/tm/$(cat .monitor)/monitor_ds /var/lib/one/datastores/100/`. Note that the argument is the datastore path and not the VM or VM disk. +The monitor_ds.sh probe from the IM, if the `.monitor` file is present (e.g., `/var/lib/one/datastores/100/.monitor`), will execute its contents in the form `/var/lib/one-remotes/remotes/tm/$(cat .monitor)/monitor_ds /var/lib/one/datastores/100/`. Note that the argument is the datastore path and not the VM or VM disk. The script is responsible for getting the information from all disks of all VMs in the datastore in that node. diff --git a/content/product/integration_references/system_interfaces/hook_driver.md b/content/product/integration_references/system_interfaces/hook_driver.md index 094d552cb..31a93a37e 100644 --- a/content/product/integration_references/system_interfaces/hook_driver.md +++ b/content/product/integration_references/system_interfaces/hook_driver.md @@ -44,7 +44,7 @@ Hook Execution Manager configuration is set in `/etc/one/onehem-server.conf`: |-----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------| | debug_level | Set the log debug level shown in `/var/log/one/onehem.log` | | hook_base_path | Base location to look for hook scripts when commands use a relative path (default value `/var/lib/one/remotes/hooks`) | -| remote_hook_base_path | Base location to look for hook scripts when commands use a relative path and `REMOTE="yes"` is specified (default value `'/var/tmp/one/hooks'`) | +| remote_hook_base_path | Base location to look for hook scripts when commands use a relative path and `REMOTE="yes"` is specified (default value `'/var/lib/one-remotes/hooks'`) | | subscriber_endpoint | To subscribe for OpenNebula events, must match those in `HM_MAD` section of `oned.conf`. | | replier_endpoint | To send hook execution results (reply to events) to oned, it must match those in `HM_MAD` section of `oned.conf`. | | concurrency | Number of hooks executed simultaneously. | diff --git a/content/product/operation_references/opennebula_services_configuration/oned.md b/content/product/operation_references/opennebula_services_configuration/oned.md index 2048ceea9..f8d44b838 100644 --- a/content/product/operation_references/opennebula_services_configuration/oned.md +++ b/content/product/operation_references/opennebula_services_configuration/oned.md @@ -29,7 +29,6 @@ For a quick view of any changes in configuration file options in maintenance rel - `MONITORING_INTERVAL_DATASTORE`: Time in seconds between each Datastore monitoring cycle. - `MONITORING_INTERVAL_MARKET`: Time in seconds between each Marketplace monitoring cycle. - `DS_MONITOR_VM_DISK`: Number of `MONITORING_INTERVAL_DATASTORE` intervals to monitor VM disks. `0` to disable. Only applies to `fs` and `fs_lvm` datastores. -- `SCRIPTS_REMOTE_DIR`: Remote path to store the monitoring and VM management script. - `PORT`: Port where `oned` will listen for XML-RPC calls. - `LISTEN_ADDRESS`: Host IP to listen for XML-RPC calls (default: all IPs). - `HOSTNAME`: Hostname to use instead of autodetect it. This hostname is used to connect to Front-end during driver operations. @@ -84,8 +83,6 @@ MONITORING_INTERVAL_MARKET = 600 #DS_MONITOR_VM_DISK = 10 -SCRIPTS_REMOTE_DIR=/var/tmp/one - PORT = 2633 LISTEN_ADDRESS = "0.0.0.0" diff --git a/content/product/operation_references/opennebula_services_configuration/troubleshooting.md b/content/product/operation_references/opennebula_services_configuration/troubleshooting.md index 0bc58454b..b674a80d9 100644 --- a/content/product/operation_references/opennebula_services_configuration/troubleshooting.md +++ b/content/product/operation_references/opennebula_services_configuration/troubleshooting.md @@ -257,7 +257,7 @@ The error message here (see `ERROR=[MESSAGE="Error monitoring host...`) shows an ```none Tue Jul 19 17:17:22 2011 [InM][I]: Monitoring host host01 (1) -Tue Jul 19 17:17:22 2011 [InM][I]: Command execution fail: scp -r /var/lib/one/remotes/. host01:/var/tmp/one +Tue Jul 19 17:17:22 2011 [InM][I]: Command execution fail: scp -r /var/lib/one/remotes/. host01:/var/lib/one-remotes Tue Jul 19 17:17:22 2011 [InM][I]: ssh: Could not resolve hostname host01: nodename nor servname provided, or not known Tue Jul 19 17:17:22 2011 [InM][I]: lost connection Tue Jul 19 17:17:22 2011 [InM][I]: ExitCode: 1 diff --git a/content/product/virtual_machines_operation/virtual_machines_networking/tproxy.md b/content/product/virtual_machines_operation/virtual_machines_networking/tproxy.md index 553f9c6c1..b40f7971c 100644 --- a/content/product/virtual_machines_operation/virtual_machines_networking/tproxy.md +++ b/content/product/virtual_machines_operation/virtual_machines_networking/tproxy.md @@ -108,7 +108,7 @@ The internal implementation of Transparent Proxies involves several networking p * `nft` (`nftables`) to store the service mapping and manage ARP resolutions * `ip netns` / `nsenter` family of commands to manage and use network namespaces * `ip link` / `ip address` / `ip route` commands -* `/var/tmp/one/vnm/tproxy` the actual implementation of the “String-Phone” daemon mesh +* `/usr/lib/one/sbin/tproxy` the actual implementation of the “String-Phone” daemon mesh Below are several example command invocations, to gain familiarity with the environment. @@ -165,7 +165,7 @@ default dev br0a scope link **Listing PIDs of running proxy processes:** ```default -$ /var/tmp/one/vnm/tproxy status +$ /usr/lib/one/sbin/tproxy status one_tproxy: 16803 one_tproxy_br0: 16809 ``` @@ -182,8 +182,8 @@ There is no PID file management implemented. For simplicity, all proxy processes **Restarting/reloading config of proxy daemons:** ```default -$ /var/tmp/one/vnm/tproxy restart -$ /var/tmp/one/vnm/tproxy reload +$ /usr/lib/one/sbin/tproxy restart +$ /usr/lib/one/sbin/tproxy reload ``` {{< alert title="Important" type="info" >}} diff --git a/content/software/upgrade_process/upgrade_guide/upgrading_single.md b/content/software/upgrade_process/upgrade_guide/upgrading_single.md index 1ea039b58..04f8f90e1 100644 --- a/content/software/upgrade_process/upgrade_guide/upgrading_single.md +++ b/content/software/upgrade_process/upgrade_guide/upgrading_single.md @@ -247,6 +247,12 @@ Note that the `` tag should be replaced by the name of the correspon {{< alert title="Important" type="info" >}} For KVM hypervisor it’s also necessary to restart the libvirt service{{< /alert >}} +{{< alert title="Important" type="warning" >}} +Do not sync the drivers (`onehost sync`) before the `opennebula-node` packages are upgraded on the Host: the drivers are now copied to `/var/lib/one-remotes` and execute helper commands from `/usr/lib/one/sbin` — both locations are created by the new node packages.{{< /alert >}} + +{{< alert title="Note" type="info" >}} +The remote scripts directory is now fixed to `/var/lib/one-remotes` and the `SCRIPTS_REMOTE_DIR` option is gone from oned.conf; a custom value is ignored after the upgrade. The monitoring metrics database is migrated automatically from `/var/tmp/one_db` to `/var/lib/one-remotes_db` when Host monitoring starts. The legacy `/var/tmp/one` directory on the Hosts is left behind: remove it once all Virtual Machines using virtioFS filesystems have been powered off/on or migrated, so their domains reference the new location.{{< /alert >}} + Then update the virtualization, storage, and networking drivers. As the `oneadmin` user, execute: ```bash