debci-status: Consistency with the results and output (!270) · Merge requests · Debian CI / debci

g0tmi1k requested to merge g0tmi1k/debci:status_json_output into master Feb 05, 2024

I started on/forked from: !269 (closed)

Ive noticed some discrepancies with debci-status. First command is often what is original/upstream version, second is this MR:

Normal

Singe valid package:

# sudo -u debci debci status_upstream exploitdb
neutral
#
# sudo -u debci debci status exploitdb
exploitdb neutral
#

Slightly different output

Single invalid package:

# sudo -u debci debci status_upstream randompackage
unknown
#
# sudo -u debci debci status randompackage
randompackage unknown
#

Slightly different output

Multiple packages:

# sudo -u debci debci status_upstream randompackage exploitdb
randompackage unknown
exploitdb     neutral
#
# sudo -u debci debci status randompackage exploitdb
randompackage unknown
exploitdb     neutral
#

Same output

JSON

Singe valid package, JSON output:

# sudo -u debci debci status_upstream --json exploitdb
"neutral"
#
# sudo -u debci debci status --json exploitdb
[
  {
    "package": "exploitdb",
    "status": "neutral"
  }
]
#

Different output, now valid JSON

Single invalid package, JSON output:

# sudo -u debci debci status_upstream --json randompackage
null
#
# sudo -u debci debci status --json randompackage
[
  {
    "package": "randompackage",
    "status": "unknown"
  }
]
#

Different output, now valid JSON

Multiple packages, JSON output:

# sudo -u debci debci status_upstream --json randompackage exploitdb
[
  {
    "run_id": 5175947,
    "created_at": "2024-02-01T13:24:09.309Z",
    "updated_at": "2024-02-01T13:59:20.093Z",
    "suite": "kali-dev",
    "arch": "amd64",
    "version": "20240124-0kali1",
    "trigger": null,
    "status": "neutral",
    "pin_packages": [

    ],
    "date": "2024-02-01T13:59:20.000Z",
    "duration_seconds": 165,
    "last_pass_date": null,
    "last_pass_version": null,
    "message": "No tests in this package or all skipped",
    "previous_status": "neutral",
    "package_id": 18147,
    "requestor_id": 7,
    "is_private": false,
    "extra_apt_sources": [

    ],
    "worker_id": 5,
    "log_size": 3365,
    "artifacts_size": 2807,
    "duration_human": "2m 45s",
    "package": "exploitdb"
  }
]
#
# sudo -u debci debci status --json randompackage exploitdb
[
  {
    "package": "randompackage",
    "status": "unknown"
  },
  {
    "package": "exploitdb",
    "status": "neutral"
  }
]
#

Different output, was behaving like --status-file before

status-file

Singe valid package, complete JSON output:

# sudo -u debci debci status_upstream --status-file exploitdb
{
  "run_id": 5175947,
  "created_at": "2024-02-01T13:24:09.309Z",
  "updated_at": "2024-02-01T13:59:20.093Z",
  "suite": "kali-dev",
  "arch": "amd64",
  "version": "20240124-0kali1",
  "trigger": null,
  "status": "neutral",
  "pin_packages": [

  ],
  "date": "2024-02-01T13:59:20.000Z",
  "duration_seconds": 165,
  "last_pass_date": null,
  "last_pass_version": null,
  "message": "No tests in this package or all skipped",
  "previous_status": "neutral",
  "package_id": 18147,
  "requestor_id": 7,
  "is_private": false,
  "extra_apt_sources": [

  ],
  "worker_id": 5,
  "log_size": 3365,
  "artifacts_size": 2807,
  "duration_human": "2m 45s",
  "package": "exploitdb"
}
#
# sudo -u debci debci status --status-file exploitdb
[
  {
    "run_id": 5175947,
    "created_at": "2024-02-01T13:24:09.309Z",
    "updated_at": "2024-02-01T13:59:20.093Z",
    "suite": "kali-dev",
    "arch": "amd64",
    "version": "20240124-0kali1",
    "trigger": null,
    "status": "neutral",
    "pin_packages": [

    ],
    "date": "2024-02-01T13:59:20.000Z",
    "duration_seconds": 165,
    "last_pass_date": null,
    "last_pass_version": null,
    "message": "No tests in this package or all skipped",
    "previous_status": "neutral",
    "package_id": 18147,
    "requestor_id": 7,
    "is_private": false,
    "extra_apt_sources": [

    ],
    "worker_id": 5,
    "log_size": 3365,
    "artifacts_size": 2807,
    "duration_human": "2m 45s",
    "package": "exploitdb"
  }
]
#

Slightly different output

Single invalid package, complete JSON output:

# sudo -u debci debci status_upstream --status-file randompackage
/home/debci/bin/debci-status:82:in `<main>': undefined method `job' for nil:NilClass (NoMethodError)

      puts JSON.pretty_generate(status.job.as_json)
                                      ^^^^
#
# sudo -u debci debci status --status-file randompackage
[
  {
    "package": "randompackage",
    "status": "unknown"
  }
]
#

Improved output

Multiple packages, complete JSON output:

# sudo -u debci debci status_upstream --status-file randompackage exploitdb
[
  {
    "run_id": 5175947,
    "created_at": "2024-02-01T13:24:09.309Z",
    "updated_at": "2024-02-01T13:59:20.093Z",
    "suite": "kali-dev",
    "arch": "amd64",
    "version": "20240124-0kali1",
    "trigger": null,
    "status": "neutral",
    "pin_packages": [

    ],
    "date": "2024-02-01T13:59:20.000Z",
    "duration_seconds": 165,
    "last_pass_date": null,
    "last_pass_version": null,
    "message": "No tests in this package or all skipped",
    "previous_status": "neutral",
    "package_id": 18147,
    "requestor_id": 7,
    "is_private": false,
    "extra_apt_sources": [

    ],
    "worker_id": 5,
    "log_size": 3365,
    "artifacts_size": 2807,
    "duration_human": "2m 45s",
    "package": "exploitdb"
  }
]
# sudo -u debci debci status --status-file randompackage exploitdb
[
  {
    "package": "randompackage",
    "status": "unknown"
  },
  {
    "run_id": 5175947,
    "created_at": "2024-02-01T13:24:09.309Z",
    "updated_at": "2024-02-01T13:59:20.093Z",
    "suite": "kali-dev",
    "arch": "amd64",
    "version": "20240124-0kali1",
    "trigger": null,
    "status": "neutral",
    "pin_packages": [

    ],
    "date": "2024-02-01T13:59:20.000Z",
    "duration_seconds": 165,
    "last_pass_date": null,
    "last_pass_version": null,
    "message": "No tests in this package or all skipped",
    "previous_status": "neutral",
    "package_id": 18147,
    "requestor_id": 7,
    "is_private": false,
    "extra_apt_sources": [

    ],
    "worker_id": 5,
    "log_size": 3365,
    "artifacts_size": 2807,
    "duration_human": "2m 45s",
    "package": "exploitdb"
  }
]
#

Includes the invalid package which was requested

--all

Checking normal output, for all values:

# sudo -u debci debci status_upstream --all | awk '{print $1}' | tee /tmp/status_upstream | wc -l
19551
# sudo -u debci debci status --all | awk '{print $1}' | tee /tmp/status | wc -l
21074
#

There is now extra values Digging into these extra values:

# while read -r line; do grep -q "^${line}$" /tmp/status_upstream || echo ${line}; done < /tmp/status | tee /tmp/diff | wc -l
1524
#
# expr 1524 + 19551
21075
#
# while read -r line; do grep -q "^${line}$" /tmp/status || echo ${line}; done < /tmp/status_upstream 
#

1524 extra values. Checking it out:

# head /tmp/diff
4ti2
aafigure
adabrowse
adacontrol
aewm
afl
afterstep
aiocoap
aiorwlock
aiowsgi
#
# sudo -u debci debci status_upstream 4ti2
pass
#
# sudo -u debci debci status 4ti2
4ti2 pass
#
# sudo -u debci debci status_upstream --all | grep 4ti2
#
# sudo -u debci debci status --all | grep 4ti2
4ti2                                                                    pass
#

So debci status can see it when selecting single package, but not when doing --all! Its missing out packages...

Checking JSON output, for all values:

# sudo -u debci debci status_upstream --all --json | jq '.[].name' | wc -l
13434
# sudo -u debci debci status --all --json | jq '.[].name' | wc -l
21074
#

13434 is different to 19551, which is what was non-JSON output (missing out packages...). 21074 is again static...

Checking status_file output, for all values:

# sudo -u debci debci status_upstream --all --status-file | jq '.[].name' | wc -l
find: Failed to restore initial working directory: /root: Permission denied
13434
# sudo -u debci debci status --all --status-file | jq '.[].name' | wc -l
21074
#

13434 is the same to --json output, but different to 19551, which is what was non-JSON output (still missing out packages...). 21074 is once again static...

debci-status: Consistency with the results and output

Normal

JSON

status-file

--all

Merge request reports