Solution and tooling managing security updates of libraries in statically-linked language ecosystems

Problem statement

The release team (and subsequently the security team) lacks the ability (tooling and workflow) to effectively schedule and monitor all of the necessary rebuilds when a Go or Rust library is updated.

Problem discussion

Develop a solution for managing security updates of libraries in statically-linked language ecosystems (e.g., Go, Rust, and so on).

All Debian releases since buster contain the following in the release notes:

The Debian infrastructure currently doesn't properly enable rebuilding packages that statically link parts of other packages on a large scale. Until buster that hasn't been a problem in practice, but with the growth of the Go ecosystem it means that Go based packages won't be covered by regular security support until the infrastructure is improved to deal with them maintainably.

If updates are warranted, they can only come via regular point releases, which may be slow in arriving.

Starting with bookworm, the above statement also includes Rust along with Go.

This issue was discussed today in the Freexian Sprint and we have developed a way forward.

Proposed solution summary

We need to develop a tool that enables the release team to manage the update of a Go or Rust (or some other similar statically-linked language) library by identifying all reverse dependencies of the library (and the reverse dependencies of those, and so on, as needed) and scheduling the necessary no-change sourceful uploads.

Detailed discussion

After a wide-ranging discussion involving multiple people (including a member of the security team and a member of the release team), we concluded that the core problem is as stated above:

The release team (and subsequently the security team) lacks the ability (tooling and workflow) to effectively schedule and monitor all of the necessary rebuilds when a Go or Rust library is updated.

Rather than targeting a solution starting with LTS or even the security team, we collectively decided that it makes the most sense to first develop a solution for unstable, and then successively work toward having the solution adopted for testing/stable, and then for LTS.

The initial stakeholders (outside of the LTS team) have been identified as: the Go/Rust teams, and the release team. The first step in developing a solution is to contact these teams and engage them with the following discussion points:

• Go/Rust teams:
  • what current tooling do you have available for managing coordinated uploads (e.g., along the lines of the issues described in this issue) of packages in your language ecosystem?
  • how do you monitor the progress of these coordinated uploads and what tooling do you currently have available to support this?
• Release team:
  • how would a "coordinated upload/rebuild" of a group of packages in a statically-linked language ecosystem need to function in order to best integrate with your existing workflows?

There are doubtless many other considerations that will impact the understanding of the problem and the proposed solution. As the discussion progresses, this issue should be updated, and additional issues/sub-issues/etc should be created and linked.

/cc @santiago