Skip to content

debcargo looks everywhere in debian/ , even in missing sources.

Hello

When packaging simd-json, the crate’s test data requires including a data test folder. I copied this folder into debian/missing-sources/, but debcargo scans all files under the debian/ folder. Some of the test files in that data set are intentionally corrupted JSON with invalid UTF-8 (e.g. \x80), which causes debcargo itself to crash instead of skipping them.

debcargo failed: stream did not contain valid UTF-8

Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
             at /home/lechatp/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.86/src/backtrace.rs:27:14
   1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /home/lechatp/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/result.rs:2050:27
   2: debcargo::package::PackageProcess::post_package_checks
             at /home/lechatp/deploy/debcargo/src/package.rs:222:22
   3: debcargo::real_main
             at /home/lechatp/deploy/debcargo/src/bin/debcargo.rs:57:13
   4: debcargo::main
             at /home/lechatp/deploy/debcargo/src/bin/debcargo.rs:79:21
   5: core::ops::function::FnOnce::call_once
             at /home/lechatp/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
   6: std::sys::backtrace::__rust_begin_short_backtrace
             at /home/lechatp/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/backtrace.rs:152:18
   7: std::rt::lang_start::{{closure}}
             at /home/lechatp/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:199:18
   8: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/core/src/ops/function.rs:284:13
   9: std::panicking::try::do_call
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/std/src/panicking.rs:589:40
  10: std::panicking::try
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/std/src/panicking.rs:552:19
  11: std::panic::catch_unwind
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/std/src/panic.rs:359:14
  12: std::rt::lang_start_internal::{{closure}}
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/std/src/rt.rs:168:24
  13: std::panicking::try::do_call
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/std/src/panicking.rs:589:40
  14: std::panicking::try
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/std/src/panicking.rs:552:19
  15: std::panic::catch_unwind
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/std/src/panic.rs:359:14
  16: std::rt::lang_start_internal
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/std/src/rt.rs:164:5
  17: std::rt::lang_start
             at /home/lechatp/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:198:5
  18: main
  19: <unknown>
  20: __libc_start_main
  21: _start

As the debcargo/src/package.rs:222:22 is quite condensed, I have some strace details where it start to crash :

[pid 832457] openat(AT_FDCWD, "/home/lechatp/deploy/debcargo-conf/build/simd-json/debian/missing-sources/data/fail/fail72.json", O_RDONLY|O_CLOEXEC) = 7
[pid 832457] read(7, "[\"contains bad UTF-8 \200\"]\n", 8192) = 25

Then it follows with a bunch of stack gathering and brk and the error message printing. that shows it's indeed the messy json file that makes debcargo crash.

Maybe the grep -R-style scan inside debian/ should filter out binary / non-UTF-8 files instead of every file?