Skip to content
Snippets Groups Projects

Allow preseeding extra options for the local sources.list lines

Merged Allow preseeding extra options for the local sources.list lines
lolando/apt-setup:preseed-extra-options into master
Merged Roland Mas requested to merge lolando/apt-setup:preseed-extra-options into master

This patch allows a preseed to specify extra options for the sources.list lines. The concrete use case triggering this change was a local requirement to add check-valid-until=false on a local repository: setting the URL of the repo to "[check-valid-until=false] http://…" was a hack that only worked as long as it didn't require a specific key; when a key was specified, the line ended up like "deb [signed-with=…] [check-valid-until=false] https://…", which was invalid. This patch provides a cleaner way to fullfill that need, without the hackish solution.

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
generators/60local
22 22 if db_get "apt-setup/$repo_name/key"; then
23 23 key="$RET"
24 24 fi
25 options=
26 if db_get "apt-setup/$repo_name/options"; then
  • Alper Nebi Yasak
    Alper Nebi Yasak @alpernebbi ·
    Developer

    It may be better to have one template per option, to avoid parsing this template value in case we switch to generating deb822-style sources files.

  • Philip Hands
    Philip Hands @philh ·
    Owner

    @alpernebbi: Do you mean that we have e.g.: apt-setup/$repo_name/opt-check-valid-until which is set to true or false and then that generates the relevant option setting depending upon whether we generate it as a one-liner or deb822-style?

    If so, presumably we'd need to have one for every possible option, which seems like a bit of a maintenance burden.

    How about detecting that someone is using deb822 options (which would contain a ':') and if we're still generating one-line config, converting the options.

    I'm imagining that one might specify something like:

    apt-setup/$repo_name/option="Check-Valid-Until: false\nArchitecture: xyz"

    and get something like:

    deb [check-valid-until=false arch=xyz] ...

    Using \n as a delimiter seems pretty easy to deal with, and ought to work on the kernel command line (one would also want to handle \\ and replace it with \).

    We could then differentiate between options that include no colons (:), and are thus in the old style, and those that do.

    For those including colons, if we're still producing one-line output we could process such options back into one-line style.

    It seems like a good idea to use the deb822 version of the option names, because one can get most of the one-line versions of an option name by lower-casing the deb822 version, and we could have a lookup function for the few that are not (e.g. 'Architecture' -> 'arch').

    I'd say it's not worth making it possible to specify an old-style one-line option and have it generate a deb822 config, because people will quickly discover their options are not working and fix them - we could always throw an error if we notice an '=' once we're generating deb822 config.

    It seems to me that we could leave writing that code until we're actually generating deb822 config, and at present continue with simply putting whatever's set into the options bit of the one-liner.

    We could have a "Sorry not yet implemented" error if a we find that the set option matches ^[^=]+: to tell people that we expect to be able to specify deb822 options at some point.

  • Alper Nebi Yasak
    Alper Nebi Yasak @alpernebbi ·
    Developer

    Hi, sorry I missed this. Indeed, I meant it like you said in the beginning, apt-setup/$repo_name/check-valid-until ideally as a Type: boolean. Then we get everything from debconf and do something like:

    echo >/target/etc/sources.list.d/$repo_name.sources <<EOF
Types: deb
URIs: $repo_uri
Architectures: $repo_arch
...
EOF

    If so, presumably we'd need to have one for every possible option, which seems like a bit of a maintenance burden.

    I think in general, it's better to keep the debconf templates as a simple key-value store with strict typing where possible. That way, we would be offloading most of the parsing and input validation burden to debconf, and would need less complex code in the installer.

    Although in this case, I'm not sure how we can set Type: boolean for the dynamically-named template here, so we still might need if [ "$RET" = true ]; then opt=yes; else opt=no; fi kind of things. Regardless, I still think it'd be less of a burden than trying to parse the sources.list options field or deb822 options lines.

    (And, I guess we don't need to support all of the options immediately.)

    How about detecting that someone is using deb822 options (which would contain a ':') and if we're still generating one-line config, converting the options.

    Some of the deb822 options can have multiple values like Architectures: amd64 arm64. Signed-By can be multiline and include an entire PGP key in armored form. Suites can have $(ARCH) in it as a variable. We can inject a new source by putting \n\n\nTypes: deb\nURIs: ...\n as 'options'. There are just too many problems trying to parse deb822 to convert back to sources.list. We can detect them and raise errors, or we can even be sloppy about parsing it. But why, when all that can be avoided instead?

    Using \n as a delimiter seems pretty easy to deal with, and ought to work on the kernel command line (one would also want to handle \\ and replace it with \).

    It's hard to parse spaces (and escapes) properly in the kernel command line, especially in shell (IFS=? $(echo "$options" | sed -ne "s/^Option:\s*//p")?). This way would require spaces to use these repo options. Debconf might handle that fine and put it into the db, but even then the installer won't be the only thing that needs to handle them. Bootloaders and bootloader-adjacent programs that create bootable images might get it wrong.

    It seems like a good idea to use the deb822 version of the option names, because one can get most of the one-line versions of an option name by lower-casing the deb822 version, and we could have a lookup function for the few that are not (e.g. 'Architecture' -> 'arch').

    I imagine we wouldn't need a name lookup function either, with simple per-option templates.

    I'd say it's not worth making it possible to specify an old-style one-line option and have it generate a deb822 config, because people will quickly discover their options are not working and fix them - we could always throw an error if we notice an '=' once we're generating deb822 config.

    It seems to me that we could leave writing that code until we're actually generating deb822 config, and at present continue with simply putting whatever's set into the options bit of the one-liner.

    We could have a "Sorry not yet implemented" error if a we find that the set option matches ^[^=]+: to tell people that we expect to be able to specify deb822 options at some point.

    If we design the templates to overfit one of the current sources.list (or deb822), we might have to carry the code we use to parse that format forever at the risk of breaking preseed users who would rely on the new templates. People will fix their things if we break it, sure, but they will complain loudly while doing so. Why should we plan now to break this new template later, when we can avoid breaking it altogether?

  • Please register or sign in to reply
  • Philip Hands added 5 commits

    added 5 commits

    • 91162074 - 1 commit from branch installer-team:master
    • c8eafa94 - Allow preseeding extra options for the local sources.list lines
    • 92f3d7cf - avoid need to remove extra space, by not adding it
    • 3db109ff - avoid double space when $options unset
    • 6869af5c - set $options immediately, avoiding $key_path check

    Compare with previous version

    Toggle commit list
  • Philip Hands
    Philip Hands @philh ·
    Owner

    BTW I've worked out what gitlab didn't like about the previous coverage reports, so now one can see red/green highlighting of lines in merges: e.g. philh/apt-setup!1 (diffs) (see line 27 of 60local for an uncovered line marked in red)

  • Roland Mas added 11 commits

    added 11 commits

    Compare with previous version

  • Philip Hands added 2 commits

    added 2 commits

    • 9b11d2aa - break if into shorter lines
    • 807a132d - add coverage for new 'options' preseed code

    Compare with previous version

  • Roland Mas added 9 commits

    added 9 commits

    Compare with previous version

  • Roland Mas added 2 commits

    added 2 commits

    • a46ffc4e - 1 commit from branch installer-team:master
    • 33414b23 - Merge branch 'master' into preseed-extra-options

    Compare with previous version

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
    • Please register or sign in to reply