Skip to content

Implement changes for sub-workflow coordination

Please implement the design in https://freexian-team.pages.debian.net/debusine/reference/devel-blueprints/workflow-coordination.html (see !1079 (merged) and !1084 (merged)). We need the following changes:

  • debusine:promise bare data: add category name to debusine.artifacts.models.BareDataCategory, and add a pydantic model for it (we don't have any models for bare data yet, but I guess somewhere in debusine.artifacts.models would do; it will need extra = pydantic.Extra.allow in its configuration, plus a root validator to forbid extra names starting with promise_)
  • arrange to validate bare data in collections against pydantic models if available; I guess this could go in CollectionItemManager.create_from_bare_data, maybe?
  • change WorkflowInternalManager.do_add_bare_data to forbid names starting with promise_ unless the category is debusine:promise
  • change WorkflowInternalManager.do_add_artifact to copy variables into per-item data, and to forbid names starting with promise_
  • change WorkflowInternalManager.do_add_artifact to allow an artifact to replace either bare data or another artifact (see DebianPackageBuildLogsManager.do_add_artifact), so that a real artifact can replace a promise
  • change WorkRequest._retry_supersede to update the work request ID in any promises associated with the work request being retried
  • change the scheduler to only call populate_workflow for root workflows, not sub-workflows (we'll work out the exact way that root workflows populate sub-workflows once we're implementing a real example of such a thing)
  • add Workflow or WorkRequest helper methods to declare that a work request provides output / requires input (I'm not sure exactly what their signatures should be; let me know if you need more help defining these)
  • change SbuildWorkflow to make each of its sbuild task children provide build-{architecture} output

The last two tasks here should be done together to make sure that the helper methods have useful interfaces, but the others are mostly independent.

Edited by Colin Watson
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information