Skip to content

RFC: apps: Add ability to store app data on external disk

Currently, most FreedomBox apps can store data only on internal disk. But cheap single board computers have usually limited internal disk space. This can be problematic if an app has a large amount of data. To overcome this limitation, add ability for apps to use external disk by moving app data to an external disk and creating a symlink from /var/lib/--app_data_dir--.

Currently four apps (deluge, syncthing, mldonkey and transmission) can save data on external disks using Samba shares. Also the backup app can save backups on external disks.

This is continuation of "RFC: Using external storage with FreedomBox applications" https://discuss.freedombox.org/t/rfc-using-external-storage-with-freedombox-applications/228

# @sunilmohan comment from !1992 (comment 227737):

I am thinking that we could consider symlinking to these new data directories from the current location in /var/lib/.

  • This lifts the burden from app to implement a data storage location change. Otherwise, for some apps this will become hard to implement and in practise will never get implemented.
  • Potentially easier to implement and easier to test.
  • We can easily implement this for all the apps that need this feature without relying on upstream changes.
  • We should take care of ownership of folders and permissions.
  • Make sure dpkg does not get upset when trying to create/recreate/update these folders.
  • Relax sandboxed daemons to allow access external disk storage.
  • Implement moving existing data after a bit of usage.
  • Consider snapshots for external disks or at least implications and warnings.
  • Handle systemd dynamic users storage mechanism in /var/lib/private/ (should not be too hard).
  • Start with small implementation for a single app but with above as roadmap perhaps.
# @vexch thoughts:

  • Add ability to move app data between disks, move data between disks: internal->external, external->external, external->internal.

  • Only posix compliant filesystems are supported, chmod and chown commands should work, so filesystems not supported: NTFS, FAT32, exFAT, CIFS, SSHFS ... (Samba supports also NTFS, Backups app support any filesystem).

  • Use a special folder on external disk to store app data --mount_point--/FreedomBox/appdata/--app_data_dir--

  • If the disk with app data is unplugged:

    • Plinth service and essential apps must not be affected, hence for essential apps, allow only /var/lib/ for a data location.
    • The app configuration page in the Plinth UI should load and it should be possible to fix/re-install the app.
    • An app can be restored from a backup, at least after re-installing the app.

  • Stop the app before moving data.

  • Allow only one app move process at once, use a lock mechanism.

  • Check if there is enough free disk space on the destination filesystem before moving app data.

  • Show all disks in the disk selection form. If the filesystem is not supported, the option is disabled.

  • Create a storage component to support changing app data location.

  • Add tests.

  • Show progress while moving data. Moving data can take long time if the app has large amount of data. Avoid Apache timeouts.

  • Make it easier to replace an external disk - one command to move all app data from one disk to another.

  • Ability to set default disk to store data for apps that support movable data.

  • Storage: before unmounting a disk, disable apps that use the disk before unmounting or at least list apps that are using the disk.

  • Update developer documentation

# Alternative possible ways to have more disk space for apps:
Edited by Veiko Aasa