|
|
|
- [x] foo
|
|
|
|
- [ ] bar |
|
|
\ No newline at end of file |
|
|
|
This is a list of work for `live-build` and `live-manual` (upstream, this is just my work fork), covering:
|
|
|
|
- Stuff queued for submission
|
|
|
|
- Stuff that's WIP, both old stuff produced back in 2015 that's being reworked/assessed for submission, and new stuff.
|
|
|
|
- Stuff that's just been noted in an offline to-do list for consideration/investigation/whatever, or just not started yet.
|
|
|
|
|
|
|
|
Getting all of this work done is subject to team members having the time to review the submissions.
|
|
|
|
|
|
|
|
I'm not going to list here what's recently been accomplished so far, just the stuff that remains.
|
|
|
|
|
|
|
|
Things are listed in no particular order. Roughly organised by state of completion.
|
|
|
|
|
|
|
|
Queued
|
|
|
|
------
|
|
|
|
Queued for submission (or only in need of very tiny tweaks to finalise):
|
|
|
|
|
|
|
|
- [ ] Last adjustment to grub2 menu construction to completely remove dependence on perl.
|
|
|
|
- [ ] Fix for theme issues in grub2 submenus.
|
|
|
|
- [ ] Expansion of accessibility install menu entry set to match that of official Debian 10 install discs.
|
|
|
|
- [ ] Additional speech synth entries.
|
|
|
|
- [ ] Addition of dark contrast mode entries.
|
|
|
|
- [ ] Switching grub2 menus to use gfxpayload.
|
|
|
|
- [ ] Fix to add warning when auto file exists but is not executable so is being ignored.
|
|
|
|
- [ ] Optimisation for creating fresh config (no need to try to read existing files and re-run option processing).
|
|
|
|
- [ ] Work on helping solve the problem of users modifying an existing config by re-running `lb config` with some auto-determined options ending up with stale values (when not explicitly set):
|
|
|
|
- [ ] Patch adding `lb config --ignore-saved`.
|
|
|
|
- [ ] Patch adding `lb config --purge-saved`.
|
|
|
|
- [ ] Patch adding `lb clean --saved-config`.
|
|
|
|
- See more below.
|
|
|
|
- [ ] Work improving `lb clean`:
|
|
|
|
- [ ] Expanding debugging.
|
|
|
|
- [ ] Adding missing handling of options like `--debug`.
|
|
|
|
- [ ] Fix for junk config file loading code, wrong variable name, tweaks.
|
|
|
|
- [ ] Work resolving ability to rename config files without backwards compatibility concerns:
|
|
|
|
- [ ] Use of the new `config`/`clean` options above in the example auto files.
|
|
|
|
- See more below.
|
|
|
|
|
|
|
|
Work in progress (WIP) - old
|
|
|
|
----------------------------
|
|
|
|
Old stuff needing rework to some degree before resubmission:
|
|
|
|
|
|
|
|
- [ ] Solution for bug [#718225](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=718225) (secure authentication of downloaded d-i files).
|
|
|
|
This one is important for the security of anyone enabling inclusion of the installer, and using public mirrors (as opposed to a local one). I've only held back on resubmitting because it's a very big change (in the current form of the solution) and needs some tweaking and further review beforehand, and I'm a little reserved about whether it'll be accepted. This problem has existed for years so there's no particular rush I suppose, and I've wanted to focus on the mass of much smaller/easier stuff first.
|
|
|
|
- [ ] Renaming 'chroot' stage to 'livefs'. Use of the 'chroot' term can be confusing enough, but even worse is the fact that the `chroot_*` scripts can be divided into two groups, those for constructing the live OS filesystem, and those that are common helpers for applying then removing modifications to the chroot environment used by the build process (or in cases leaving things set up for live OS use).
|
|
|
|
- [x] Submission of patch to rename the `chroot_*` live-fs scripts to `livefs_*` ([upstream MR #125](https://salsa.debian.org/live-team/live-build/-/merge_requests/125)).
|
|
|
|
- [ ] Submission of patch to rename `--chroot-filesystem` (with backwards compatibility).
|
|
|
|
- [ ] Submission of patch to rename `LB_CHROOT_FILESYSTEM` (with backwards compatibility).
|
|
|
|
- [ ] Have patch for renaming the config files, but have reservations about backwards compatibility due to past poor design choice relating to direct deletion of config files in example `auto/clean` file, as mentioned below.
|
|
|
|
- [ ] Submission of assessment of other uses of term 'chroot' to avoid inconsistency and confusion. (I said I'd do a review of all use of 'chroot' so that we could consider things in the whole before acceptance is properly considered. I put together a comprehensive list, it just needs organising and posting to the discussion so decisions can be made).
|
|
|
|
- [ ] The `--firmware-binary` and `--firmware-chroot` documentation (manpage) seems odd. The former says it is a control for inclusion of firmware packages in d-i, and the latter for inclusion in the live-image, which does not seem to correspond with typical use of 'binary' and 'chroot' terms in the one case.
|
|
|
|
- [ ] Reorganisation of the scripts into directories, though I may just drop this.
|
|
|
|
- [ ] Modification to have substage/component scripts executed directly rather than running every single one of them through the frontend as a completely new invocation of the program. (I was also considering whether this could be built upon to also avoid re-loading the config in every single script execution).
|
|
|
|
- [ ] Review of the EFI support work I started but never completed in 2015, to compare with the solution that was implemented in case there's anything work keeping or whatever.
|
|
|
|
- [ ] My EFI solution did include syslinux-efi which is not offered in live-build currently. syslinux-efi does not support secure boot, but may be useful anyway. Someone else also submitted an MR for this, which needs to be considered.
|
|
|
|
|
|
|
|
Work in progress (WIP) - new stuff
|
|
|
|
----------------------------------
|
|
|
|
New stuff, some that I wanted to do previously but never started, some completely new.
|
|
|
|
|
|
|
|
- [ ] Finish investigating case of recent user having had difficulty changing a setting of an existing config where days later they discovered that it finally worked when they ran `lb config` as root. Why was there it seems no permissions error to help guide them? Should we add a warning if running `lb config` as root to help encourage not running as root and thus help avoid this in future?
|
|
|
|
- [ ] Fixes for d-i preseed handling and documentation. ([Was submitted](https://salsa.debian.org/live-team/live-build/-/merge_requests/188) but needs more work).
|
|
|
|
- [ ] Sorting out the derivative distro hacks stuff. Some junk we've removed already. Some additional stuff like `--mode` needs untangling, to clear out the hacks, but whilst providing sufficient means for derivative distros to provide the necessary customisations (e.g. strings), as partially covered by bug [#684891](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=684891).
|
|
|
|
- [ ] Expanding 'required stages' checks in scripts to cover necessary 'chroot prep'. (Several times during testing I've tried to directly run a component script only to find that I'd forgotten to apply the chroot prep stuff first, resulting in failure; the dependency checks should be used to block this).
|
|
|
|
- [ ] Handling cleaning up dpkg tmpfs mount on bad exit?
|
|
|
|
- [ ] Robustification of scripts, to help ensure that they work correctly on re-running, such as recovering from failure, and `--force` re-running such as during development testing.
|
|
|
|
- [x] Adding missing stagefile check to chroot prep scripts in remove mode to help avoid corruption and such.
|
|
|
|
- [ ] Going through each script carefully to add bullet proofing tweaks.
|
|
|
|
- [ ] Implementing use of (or rather checking of) stagefiles at the major build stage level, such that when recovering from failure, major stages that were completed can be skipped over, instead of every single substage of each being executed only for them to just exit with no real work done, except the 'chroot prep' scripts which spend effort applying and removing changes pointlessly.
|
|
|
|
- [ ] Fix some cases where scripts are missing stage dependencies.
|
|
|
|
- [ ] Fix for handling of multiple config environment files, whitespace, quoting.
|
|
|
|
- [ ] Getting the source stage to use caching for packages (will be a big help for testing/development).
|
|
|
|
- [ ] Proper quoting of printed command line arguments, such that things will actually work correctly if the user copies and pastes it. E.g. `lb config --bootloaders syslinux grub-efi` is not going to work correctly, it needs to be `lb config --bootloaders 'syslinux grub-efi`'.
|
|
|
|
- [ ] Output done in the frontend
|
|
|
|
- [ ] `chroot_apt`
|
|
|
|
- [ ] `Chroot()`
|
|
|
|
- [ ] Handling of `_EXTRA_OPTIONS` in `config`
|
|
|
|
- [ ] Ensuring proper validation of bootloader selections. (This is already covered somewhat by completed and merged work on basic config validation, I just want to go back over it to check that it sufficiently covers this).
|
|
|
|
- [ ] Transforming the manual (`live-manual`) from a "project manual" (encompassing both user manual stuff and a bunch of other stuff) to just a "user manual", trimming about ~20% of the content, content of no interest to those just wanting to know how to use `live-build`. (Submitted as [upstream MR #27](https://salsa.debian.org/live-team/live-manual/-/merge_requests/27)). This is pending final review decisions.
|
|
|
|
|
|
|
|
Not-started list
|
|
|
|
----------------
|
|
|
|
Stuff not yet or barely started, to investigate, to consider, whatever:
|
|
|
|
|
|
|
|
- [ ] Renaming `binary_loopback_cfg` script to something like `binary_grub2_cfg`, since it is responsible for creating the config for grub-pc, grub-efi, and a tiny 'loopback' file which redirects to it, so the name is rather misleading/confusing.
|
|
|
|
- [ ] Moving the helper scripts (`efi-image` and `grub-cpmodules`) to a subdirectory for clarity.
|
|
|
|
- [ ] Possibly moving the massive chunk of code in the EFI script being dynamically written into a script file for execution, into it's own helper file, like `efi-image` and `grub-cpmodules`, such that we get proper syntax highlighting, fewer possible mistakes with variable escaping, and such.
|
|
|
|
- [ ] Addressing the todo item in the loopback script of adding an option to control generation of the loopback support, if Raphaël feels this is worth having (otherwise the fixme can be removed).
|
|
|
|
- [ ] Get the loopback support adjusted to be more based within the grub config file, and working with install menu entries not just live entries, as per the design that inspired its implementation, following the discussion with author who put in into live-build.
|
|
|
|
- [ ] Consider improving the small bit of splash related theme logic in the grub config, as detailed somewhere else, which looked odd to me.
|
|
|
|
- [ ] Work on helping solve the problem of users modifying an existing config by re-running `lb config` with some auto-determined options ending up with stale values (when not explicitly set):
|
|
|
|
- Some work queued above.
|
|
|
|
- [ ] A further improvement might be if we stored options commented out in the initial save when not explicitly set, allowing auto-determination to still work on making changes (each run of a script runs a function which will fill in defaults for unset options so no problem with this). But how best to achieve this? Is is feasible?
|
|
|
|
- [ ] Work resolving ability to rename config files without backwards compatibility concerns:
|
|
|
|
- [ ] The change to use the new `config`/`clean` options above in the example auto files helps with this, though there's still existing user `auto/clean` files which explicitly delete the files. Perhaps if we just assume all such user files delete them and modify `clean` to do it internally unless instructed not to?
|
|
|
|
- This impacts for instance being able to rename the 'chroot' config file to 'livefs'.
|
|
|
|
- Note that the 'build' config file introduced by the partial conversion to a new config format long ago that we've now reverted was never added to the example, so presumably all this time user `auto/clean` files have never actually been cleaning it.
|
|
|
|
- [ ] Review of whether or not the binary stage can use the 'debianchroot' chroot prep, in which case it can be tidied up a little.
|
|
|
|
- [ ] `Chroot_has_package()` is unused.
|
|
|
|
- [ ] `Echo_breakage()` is only used by `bootstrap_debootstrap`, possibly worth moving to it.
|
|
|
|
- [ ] `Print_conffiles()` is only used by the `config` script, so inline it?
|
|
|
|
- [ ] Discover_package_architectures() only used by `chroot_package-lists` so inline/move it?
|
|
|
|
- [ ] `Check_architectures()` compares param values to `LB_ARCHITECTURE` in a very odd and inefficient way, possibly from a time when `LB_ARCHITECTURE` was called `LB_ARCHITECTURES` (only changed recently), and there was an idea of it holding multiple arches.
|
|
|
|
- [ ] Consider correctness of the order of the sysfs and selinuxfs 'chroot prep' application. The selinuxfs script is done first and mounts to `chroot/sys/fs/selinux`, the sysfs script is done second and mounts to `chroot/sys`. Does the latter not overwrite the selinux mount such that effectively it is as though selinux is not mounted?
|
|
|
|
- [ ] Check out the fact that `Chroot_bind_path()` is used by `bootstrap_archives` and `chroot_archives`, but `Chroot_unbind_path()` is only used by `chroot_archives`.
|
|
|
|
- [ ] Improve bootstrap cache handling; If the bootstrap exists in the cache, first it's copied, then the debootstrap script is skipped, then the cached copy is replaced with it again, which is madness.
|
|
|
|
- [ ] Consider, with the above mentioned change of having stage files checked in major build stages such that they are skipped in recovering and such (if re-running `lb build`), whether this is a sufficient improvement such that there is no longer value to the developer feature of caching stages (other than bootstrap which is always required).
|
|
|
|
- [ ] The binary stage directly removes the "chroot_archives" stagefile rather than running that 'chroot prep' script in a suitable mode, which is problematic for a bunch of reasons I've got jotted down in my offline todo list file, which I won't bother putting here.
|
|
|
|
- [ ] If failure occurs in a script that installed packages temporarily for accomplishing some action which it removes at the end, then if the user tries to re-run to recover, the list of installed packages which was held in memory is lost, thus those packages are stuck in place. The list should be recorded in a file such that recovery can be done to restore things correctly.
|
|
|
|
- [ ] dpkg-divert gives a warning about redirections which we are doing, which would be nice to suppress.
|
|
|
|
- [ ] Config validation is only performed when executing a build if you use `lb build` (typical), not when executing the major stages individually, which some may do. The change mentioned above regarding avoiding running every script through the frontend could help here.
|
|
|
|
- [ ] Consider fix for problem of user doing `--foo "foo\"bar"` which ends up as `LB_FOO="foo"bar"` in the saved config files, thus corrupting the file, only fixable by manually editing the file (running live-build loads it causing error). (Though the new `lb config --purge-saved` to be proposed shortly might be able to help here).
|
|
|
|
- [ ] Consider the issue discussed in the mailing list recently of the fact that Debian live images cannot be uniquely identified, such that from EFI (or perhaps rather secure boot) loading, the stub `grub.cfg` that searches for the main image filesystem can find that of a completely different image. This impacts both situations of having multiple images (CDs/DVS/pendrives) in the computer at the same time (uncommon) but also solutions like rEFInd that place multiple images onto a pendrive for selection from an EFI menu. I've been meaning to get back to the person I was discussing stuff with for several days.
|
|
|
|
- [ ] Updating the manual (`live-manual`, not manpages!).
|
|
|
|
Another contributor has privately already been working towards this, so I'm waiting upon the results of their work before contemplating any significant fixes and rewrites myself. I've just submitted the odd minor fix here and there.
|
|
|
|
- [ ] Moving the manual (`live-manual`) away from SiSu format to something much more modern and common.
|
|
|
|
- [ ] Possibly tweaking the manual design a little to make it look nicer.
|
|
|
|
- [ ] Consider removal of the ability to build without using the chroot (the option after all insists that users never try). Is it worthy of removal? Does it even work?
|
|
|
|
- [ ] Consider whether ability to run from local folder is useful enough to keep? Was it just for development purposes? (Keeping such features is a maintenance burden).
|
|
|
|
- [ ] Consider whether the bootloaders solution involving all of the stuff in `functions/bootloaders.sh` is really a good idea, whether there's something more simple that could be implemented in the config validation checks or something instead.
|
|
|
|
- [ ] Maybe consider meson support as a more modern and preferable alternative to autotools?
|
|
|
|
- [ ] I encountered a reference somewhere to livebuild 4.x not liking certain config files that start with logic, so a less helpful item in my offline todo list states. Is this still so? If so look into fixing.
|
|
|
|
- [ ] Check whether it is correct for the `live/filesystem.packages` file to exist in the final image, similarly the `live/filesystem.packages-remove` file.
|
|
|
|
- [ ] Check out xorriso warnings about too long string lengths.
|
|
|
|
- [ ] Check out the xorriso `-no-emul-boot` option being duplicated twice in the options given to it.
|
|
|
|
- [ ] In some scripts involving `_PASS` variables and similar, for an invalid value, `Usage()` is simply called, which is not necessarily clear/helpful. Consider adding error, possibly dropping `Usage()` output.
|
|
|
|
- [ ] Consider adding a warning if user config wants firmware, but has not added contrib/non-free archive areas to remind them that they will only be getting a subset if they do not correct this?
|
|
|
|
- [ ] When you execute a component script directly, there is no message output at the end to confirm that it completed successfully rather than failed without visible error. We would not want such a message normally. Consider how to accomplish printing such only when wanted, involving preferably an exit trap (which must not conflict with the one for removing the lock). It should also of course give a similar clear message on failure.
|
|
|
|
- [ ] Consider locking of `lb clean` and `lb config`. `lb config` does not use the lock at all, and so can be run in the middle of a build. `lb clean` is similar, it just deletes the lock early on (which may be useful for recovery purposes, but allows running during build).
|
|
|
|
|
|
|
|
Additionally, helping derivative distros somewhat:
|
|
|
|
- [ ] I've submitted some MRs previously for Kali to help tweak the booloaders menus to properly benefit from the bootloader improvements work. I think these were merged, but I should review things when the bootloader changes are completely finalised considering that Raphaël reorganised some things.
|
|
|
|
- [ ] I've noticed that Tails is stuck using live-build version 2.x, with custom hacks to get EFI support. They attempted a migration to v4.x some time ago, but after doing the work found that they experienced boot failure. I hope to try to assist them in migration to a newer version at some point if they need help.
|
|
|
|
|
|
|
|
Avoiding?
|
|
|
|
---------
|
|
|
|
Stuff I'm probabaly not going to try.
|
|
|
|
|
|
|
|
- Long ago there was a plan to convert live-build to python. This was actually started then reverted and never reattempted. Back in 2015 I thought it might be nice to try to get it accomplished, as an exercise to become more familiar with python (which I've not had much opportunity to work with), and considering how brittle shell code can be. I never got anywhere near starting that. Now, it still might be nice, however especially considering all the work above already on my list, and not having unlimited time and resources, I'm doubtful about whether I'd be able to do it. Most significantly though even if I could find the time, I suspect that the team might not have the appetite for this in terms of the effort to review, if they even agreed with it being a nice idea.
|
|
|
|
- The whole build is done with elevated privileges. Could it not be possible to drop privileges when not needed. I thought a little about this but it might not be at all easy in shell code, and also given directory permissions for instance, it might not be feasible at all. (There is also bug [#788764](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=788764) about this topic).
|
|
|
|
- Consideration of grub-legacy removal. It was a bit broken, with a few fixes now implemented, but I'm largely ignoring it, and is it worth keeping? There's no particular urgency to get rid, just no idea if anyone is using it any more... |
|
|
\ No newline at end of file |
