1. 18 Dec, 2016 1 commit
  2. 25 Oct, 2016 1 commit
  3. 04 Sep, 2016 1 commit
  4. 26 Aug, 2016 1 commit
  5. 04 Aug, 2016 1 commit
  6. 12 May, 2016 1 commit
  7. 17 Apr, 2016 1 commit
  8. 17 Mar, 2016 3 commits
  9. 11 Dec, 2015 1 commit
  10. 22 Nov, 2015 1 commit
  11. 14 Nov, 2015 2 commits
  12. 17 Oct, 2015 1 commit
  13. 13 Oct, 2015 1 commit
  14. 26 Jul, 2015 3 commits
  15. 24 Jul, 2015 2 commits
  16. 04 Feb, 2015 2 commits
  17. 19 Jan, 2015 1 commit
  18. 16 Dec, 2014 1 commit
  19. 15 Dec, 2014 1 commit
  20. 15 Nov, 2014 2 commits
  21. 24 Sep, 2014 1 commit
  22. 03 Sep, 2014 1 commit
  23. 16 Jun, 2014 1 commit
    • Martin Pitt's avatar
      Separate controller and workers through AMQP queues · e1ded63f
      Martin Pitt authored
      Stop issuing debci-test calls directly from debci-batch, in order to
       - scale to multiple machines,
       - remove single point of failures,
       - and allow other controllers like britney to issue test requests,
       - eventually support multiple different backends in parallel.
      
      Instead we put test requests into an AMQP queue and let them get picked up and
      processed by arbitrarily many workers (which can be on remote hosts, or run in
      parallel on the same machine). AMQP will ensure proper load distribution and
      transactions (i. e. requests only get removed from the queue once the test has
      fully finished, otherwise requeued).
      
      We use RabbitMQ as AMQP server. In principle you can use others too, but that's
      what we test with and recommend.
      
      Introduce a new shared --amqp option to specify a remote AMQP server (defaults
      to amqp://localhost). This is mostly useful for workers; in a common setup the
      controller (debci-batch and the web UI) will run on the same host as the debci
      AMQP server.
      
      Add a new "debci-worker" script that picks up AMQP requests and calls
      debci-test on them.
      
      debci-batch now puts test requests into AMQP queues (<suite>-<arch>-<backend>,
      e. g. unstable-amd64-schroot). Drop the -j option as tests will naturally
      parallelize now depending on how many workers are available. Introduce --wait
      option to wait until all test results have arrived in the data directory to
      keep the previous behaviour and measuring of total time for now. Without that,
      you have to call generate-index manually regularly to pick up new test results.
      
      debci-batch is now a bash instead of POSIX shell script to do the scanning for
      incoming results with builtin instead of external commands for efficiency, in
      particular nullglob, string substitutions, and lexicographical string
      comparison.
      
      The tests which use debci-batch now need to start a local worker to actually
      get back results. Introduce a new start_worker() test helper for those, and
      clean it up in tearDown(). We also start and use a local rabbitmq-server, so
      that we don't interfere with a production one, and can run the tests during
      package build.
      
      TODO:
       - Eventually we want to use a distributed file system for <data>/autopkgtest/,
         so that the workers can run remotely. For now this just assumes that the
         files that workers write somehow end up at the controller (run both on the
         same machine or set up rsync). For that we will trigger a "sync to remote
         file system" before acknowledging the test result (marked with TODO in
         debci-worker).
      
       - Eventually we want to support putting a test request into multiple different
         queues, to test on multiple architectures and take isolation restrictions
         into account (e. g. put a test with isolation-machine on a -qemu queue
         instead of -schroot). For now we only issue reqests on the queue
         corresponding to the one configured backend.
      
       - Accept that debci-batch behaves asynchronously, drop --wait, and just call
         generate-index regularly from a cron job. With that one can also issue test
         requests manually to retry a test, or from britney.
      
      start local server
      e1ded63f
  24. 28 May, 2014 1 commit
  25. 13 May, 2014 2 commits
  26. 12 May, 2014 1 commit
  27. 11 May, 2014 1 commit
  28. 10 May, 2014 1 commit
    • Antonio Terceiro's avatar
      first stab at Ruby bindings · 66954fb5
      Antonio Terceiro authored
      This will need some serious optimization in the future, but it's a good
      start to a high-level interface to the debci data store
      66954fb5
  29. 07 Apr, 2014 1 commit
    • Martin Pitt's avatar
      Use chdist instead of local schroot for package queries · bcb8dc17
      Martin Pitt authored
       - Add scripts/setup-chdist to create/update a chdist for the selected release.
       - Use that in list-dependencies instead of querying the schroot. Move that
         script to scripts/ as it is now not backend specific any more.
       - Update lib/functions.sh to use chdist instead of the schroot.
       - Add devscripts dependency for chdist, and demote schroot dependency to
         recommends.
      
      This allows us to use other backends, like QEMU or remote ones.
      bcb8dc17
  30. 06 Apr, 2014 1 commit
  31. 05 Apr, 2014 1 commit