Advisory – Sophos for Virtual Environments Security Virtual Machines may run out of disk space

We are currently investigating an issue where the Security Virtual Machine (SVM) component of Sophos for Virtual Environments (SVE) can run out of disk space. This can cause additional issues on the SVM such as performance issues, communication issues to either Sophos Enterprise Console or Sophos Central and update failures; and has been reported by a number of customers.

This issue appears to be related to a current limitation on the SVM where log files are not rotated or purged effectively and therefore can use up additional unexpected space.

This is being investigated by Sophos Development in bug reference VIRTHV-3060.

Some customers continue to experience Sophos Central connectivity issues on their SVMs after following these steps. Recovery steps for SVMs in this condition are available in KBA 134494.

Applies to the following Sophos product(s) and version(s)

Sophos For Virtual Environments

SVMs may show as having no disk space in either vSphere or Hyper-V. Additionally scanning or update errors may be reported back to either Sophos Enterprise Console (SEC) or Sophos Central. Customers may also encounter that their SVMs do not report back to either SEC or Central.

A workaround is available below for affected customers. Sophos Development are also working on implementing a long term solution in a future release of SVE.

These steps can also be followed by customers concerned that the issue may occur at a future date even if it has not currently occurred.

The below steps can be followed to work around the issue – these steps guide you through removing old archived log files to clear additional space and then limiting the growth of new files:

  1. Access the command line interface of the SVM using VMware console, Hyper-V virtual machine remote connection or SSH
  2. Log into the SVM using the ‘sophos’ account and the credentials you specified during your installation of SVE
  3. Run the below command to purge the old archived logs on the SVM
    • sudo journalctl --vacuum-time=7d
    • Note. You will need to enter your password again after running this command
  4. Next add the below lines after the first open brace (“{“) in “/opt/sophox/conf/vm/tmpl/etc/logrotate.d/sophos-svms” to increase the rotation of Sophos log files
    • size 128M

      su syslog adm

      compress
    • Note. Installed on the SVM is the editor ‘VIM’ this is a standard editor for Linux based operating systems but does require some knowledge to use. If you encounter any difficulties using VIM then please contact Sophos Support for assistance.
  5. After adding in the above config lines please run the below commands to implement them:
    • atu
    • sudo logrotate /etc/logrotate.d/sophos-svms --force
    • Note. If you encounter any of the below errors these can be safely ignored as they do not indicate that the command was unsuccessful
      • error: opening /var/log/ssvm-install.log: Permission Denied
      • error: skipping “/var/log/ssvm-trace.log” because parent directory has insecure permissions…
      • error: skipping “/var/log/template-expand.log” because parent directory has insecure permissions…
  6. Next to configure the Operating System to only use a set amount of space for system log files please add the following line to “/etc/systemd/journald.conf”
    • SystemMaxUse=512M
    • Note. Installed on the SVM is the editor ‘VIM’ this is a standard editor for Linux based operating systems but does require some knowledge to use. If you encounter any difficulties using VIM then please contact Sophos Support for assistance.
  7. Then run the below command to restart the System logging daemon:
    • sudo systemctl restart systemd-journald

These steps should reduce the used disk space on the SVM and should also mitigate the risk of the issue occurring again moving forwards.

Some customers continue to experience Sophos Central connectivity issues on their SVMs after following these steps. Recovery steps for SVMs in this condition are available in KBA 134494.

If you are still encountering any other issues after running through these steps please contact Sophos Support for further assistance.

Updates will be provided to this article as they become available.

If you’ve spotted an error or would like to provide feedback on this article, please use the section below to rate and comment on the article.

This is invaluable to us to ensure that we continually strive to give our customers the best information possible.

Related:

  • No Related Posts

Leave a Reply