1. 30 Oct, 2018 1 commit
  2. 24 Oct, 2018 1 commit
  3. 23 Oct, 2018 4 commits
    • Matthew Leeds's avatar
      Rename core.repo-finders to core.default-repo-finders · ed41822b
      Matthew Leeds authored
      This renames a config key to make its semantics more obvious. Despite
      what the commit message says, it only applies when a set of repo finders
      is not specified (either on the command line or in a library API call).
      This also renames the corresponding ostree_repo_get function. We can do
      this since it hasn't been released yet.
      
      Closes: #1763
      Approved by: pwithnall
      ed41822b
    • Matthew Leeds's avatar
      lib/repo-pull: Add an explanatory comment · 3fc46f37
      Matthew Leeds authored
      Closes: #1763
      Approved by: pwithnall
      3fc46f37
    • Colin Walters's avatar
      README: Add bindings section · c9a9e6c3
      Colin Walters authored
      Since rust-libostree now exists too, let's make sure people
      know about it.
      
      Closes: #1762
      Approved by: jlebon
      c9a9e6c3
    • Jonathan Lebon's avatar
      boot: Add ostree-finalize-staged.path · ac1a919f
      Jonathan Lebon authored
      Rather than manually starting the `ostree-finalize-staged.service` unit,
      we can leverage systemd's path units for this. It fits quite nicely too,
      given that we already have a path we drop iif we have a staged
      deployment.
      
      To give some time for the preset to make it to systems, we don't yet
      drop the explicit call to `systemctl start`. Though we do make it
      conditional based on a DEBUG env var so that we can actually test it in
      CI for now. Once we're sure this has propagated, we can drop the
      `systemctl start` path and the env var together.
      
      Closes: #1740
      Approved by: cgwalters
      ac1a919f
  4. 22 Oct, 2018 1 commit
  5. 21 Oct, 2018 2 commits
    • Matthew Leeds's avatar
      lib/repo-pull: Disable LAN updates by default · 1d6347fe
      Matthew Leeds authored
      This commit disables searching on the local network for refs, unless
      explicitly requested by the user either by changing the value of the
      "core.repo-finders" config option, or by passing an OstreeRepoFinderAvahi to
      ostree_repo_find_remotes_async() / ostree_repo_finder_resolve_async(),
      or by specifying "lan" in the --finders option of the find-remotes
      command.
      
      The primary reason for this is that ostree_repo_find_remotes_async()
      takes about 40% longer to complete with the LAN finder enabled, and that
      API is used widely (e.g. in every flatpak operation). It's also probable
      that some users don't want ostree doing potentially unexpected traffic
      on the local network, even though everything pulled from a peer is GPG
      verified.
      
      Flathub will soon deploy collection IDs to everyone[1] so these code
      paths will soon see a lot more use and that's why this change is being
      made now.
      
      Endless is the only potential user of the LAN updates feature, and we
      can revert this patch on our fork of ostree. For it to be used outside
      Endless OS we will need to upstream eos-updater-avahi and
      eos-update-server into ostree.
      
      [1] https://github.com/flathub/flathub/issues/676
      
      Closes: #1758
      Approved by: cgwalters
      1d6347fe
    • Matthew Leeds's avatar
      Allow disabling pulling from LAN/USB/Internet · 3956fc88
      Matthew Leeds authored
      Currently libostree essentially has two modes when it's pulling refs:
      the "legacy" code paths pull only from the Internet, and the code paths
      that are aware of collection IDs try to pull from the Internet, the
      local network, and mounted filesystems (such as USB drives). The problem
      is that while we eventually want to migrate everyone to using collection
      IDs, we don't want to force checking LAN and USB sources if the user
      just wants to pull from the Internet, since the LAN/USB code paths can
      have privacy[1], security[2], and performance[3] implications.
      
      So this commit implements a new repo config option called "repo-finders"
      which can be configured to, for example, "config;lan;mount;" to check
      all three sources or "config;mount;" to disable searching the LAN. The
      set of values mirror those used for the --finders option of the
      find-remotes command. This configuration affects pulls in three places:
      1. the ostree_repo_find_remotes_async() API, regardless of whether or
      not the user of the API provided a list of OstreeRepoFinders
      2. the ostree_repo_finder_resolve_async() /
      ostree_repo_finder_resolve_all_async() API
      3. the find-remotes command
      
      This feature is especially important right now since we soon want to
      have Flathub publish a metadata key which will have Flatpak clients
      update the remote config to add a collection ID.[4]
      
      This effectively fixes https://github.com/flatpak/flatpak/issues/1863
      but I'll patch Flatpak too, so it doesn't pass finders to libostree only
      to then have them be removed.
      
      [1] https://github.com/flatpak/flatpak/issues/1863#issuecomment-404128824
      [2] https://github.com/ostreedev/ostree/issues/1527
      [3] Based on how long the "ostree find-remotes" command takes to
        complete, having the LAN finder enabled slows down that step of the
        pull process by about 40%. See also
        https://github.com/flatpak/flatpak/issues/1862
      [4] https://github.com/flathub/flathub/issues/676
      
      Closes: #1758
      Approved by: cgwalters
      3956fc88
  6. 19 Oct, 2018 2 commits
    • Jonathan Lebon's avatar
      ostree-prepare-root.service: Run earlier in initrd · a4a49724
      Jonathan Lebon authored
      Previously, we were preparing the root very late in the boot process;
      right before we switch root. The issue with that is that most services
      in the initrd that run `After=initrd-root-fs.target` expect that
      `/sysroot` already points to the rootfs we'll be pivoting to. Running
      this late violates that assumption.
      
      This patch fixes this by making `ostree-prepare-root.service` instead
      run right after `sysroot.mount` (the physical sysroot mounted by
      systemd) but still before `initrd-root-fs.target` (which is the target
      signalling that `/sysroot` is now valid and ready).
      
      This should make it easier to integrate OSTree with other initrd
      services such as Ignition.
      
      Related: https://github.com/dustymabe/ignition-dracut/issues/20
      
      Closes: #1759
      Approved by: cgwalters
      a4a49724
    • Jonathan Lebon's avatar
      ostree-prepare-root.service: Use RemainAfterExit=yes · ae99b9cc
      Jonathan Lebon authored
      For the same reasons as #1697. This is especially important in services
      that are likely to be used as an `After/Before=` target in other units.
      `ostree-prepare-root.service` is one such service.
      
      Closes: #1759
      Approved by: cgwalters
      ae99b9cc
  7. 17 Oct, 2018 2 commits
  8. 16 Oct, 2018 1 commit
    • Colin Walters's avatar
      finalize-staged: Bump timeout to 5 minutes · e242033f
      Colin Walters authored
      See https://github.com/projectatomic/rpm-ostree/issues/1568
      
      Basically for people on e.g. rotational media, the default 90
      second timeout can be too small.
      
      We're in a tough situation here, because delaying shutdown
      can be problematic too if the user is trying to shut down their
      laptop to put in a backpack, etc.
      
      There's potential optimizations here to make; I think we
      could pre-copy the kernel/initramfs for example.
      
      I suspect for some people the grub2 os-prober is a factor here too,
      if that tries to e.g. inspect attached USB rotational hard drives.
      But hopefully we'll get rid of that soon.
      
      Closes: #1755
      Approved by: jlebon
      e242033f
  9. 12 Oct, 2018 2 commits
    • Colin Walters's avatar
      rofiles-fuse: Improve error message for failure to open root · 04aff9c1
      Colin Walters authored
      I was debugging some rpm-ostree work and saw:
      `openat: No such file or directory`
      and it wasn't immediately obvious it was stderr from `rofiles-fuse`.
      
      Use the `err` API which is better in many ways; in this case
      it automatically prefixes with `argv0`.
      
      Closes: #1747
      Approved by: jlebon
      04aff9c1
    • Dan Nicholson's avatar
      lib/commit: Don't chown objects to repo target owner · 43d9cac4
      Dan Nicholson authored
      The idea is that if the process is running as root, it can change
      ownership of newly written files to match the owner of the repo.
      Unfortunately, it currently applies in the other direction, too - a
      non-root user writing to a root owned repository. If the repo is
      writable by the user but owned by root, it can still create files and
      directories there, but it can't change ownership of them.
      
      This feature comes from
      https://bugzilla.gnome.org/show_bug.cgi?id=738954. As it turns out, this
      feature was never completed. It only works on content objects and not
      metadata objects, refs, deltas, summaries, etc. Rather than try to fix
      all of those, remove the feature until someone has interest in
      completing it.
      
      Closes: #1754
      Approved by: cgwalters
      43d9cac4
  10. 11 Oct, 2018 3 commits
    • Colin Walters's avatar
      checkout: Support --union-identical and --force-copy{,--zerosized} · 9367a1be
      Colin Walters authored
      Actually testing the patch to add `--force-copy-zerosized` to
      rpm-ostree tripped over the fact that it uses `--union-identical`,
      and we just hit an assertion failure with that combination.
      
      Fix this by copying over the logic we have for the hardlink case.
      
      Closes: #1753
      Approved by: jlebon
      9367a1be
    • Colin Walters's avatar
      repo: Add a checkout option to not hardlink zero-sized files · 673cacd6
      Colin Walters authored
      In rpm-ostree we've hit a few cases where hardlinking zero-sized
      files causes us problems.  The most prominent is lock files in
      `/usr/etc`, such as `/usr/etc/selinux/semanage.LOCK`.  If there
      are two zero-sized lock files to grab, but they're hardlinked,
      then locking will fail.
      
      Another case here is if one is using ostree inside a container
      and don't have access to FUSE (i.e. `rofiles-fuse`), then the
      ostree hardlinking can cause files that aren't ordinarily hardlinked
      to become so, and mutation of one mutates all.  An example where
      this is concerning is Python `__init__.py` files.
      
      Now, these lock files should clearly not be in the tree to begin
      with, but - we're not gaining a huge amount by hardlinking these
      files either, so let's add an option to disable it.
      
      Closes: #1752
      Approved by: jlebon
      673cacd6
    • Sinny Kumari's avatar
      src/ostree: Don't delete refs having aliases · c7052684
      Sinny Kumari authored
      Deleting a ref with aliases makes them dangling. In such
      cases, display an error message to the user.
      
      Fixes #1597
      Signed-off-by: 's avatarSinny Kumari <sinny@redhat.com>
      
      Closes: #1749
      Approved by: cgwalters
      c7052684
  11. 05 Oct, 2018 3 commits
    • Jonathan Lebon's avatar
      lib/sysroot-deploy: Write to journal when finalizing · 05e99da7
      Jonathan Lebon authored
      Write to the journal when starting to finalize a staged deployment.
      Combined with the "Transaction completed" message we already emit, this
      makes it easy later on to determine whether the operation was successful
      by inspecting the journal. This will be used by `rpm-ostree status`.
      
      Closes: #1750
      Approved by: cgwalters
      05e99da7
    • Jonathan Lebon's avatar
      boot: Add Documentation= lines to services · 9161eb8c
      Jonathan Lebon authored
      It's a neat way to point folks to the documentation (of course, better
      would be to have man pages for each of those services). Also
      consistently use Title Case everywhere.
      
      Closes: #1750
      Approved by: cgwalters
      9161eb8c
    • Jonathan Lebon's avatar
      boot: Remove [Install] from ostree-finalize-staged · 367be40a
      Jonathan Lebon authored
      Let's just make this service not installable anymore. It should only be
      activated manually.
      
      Closes: #1750
      Approved by: cgwalters
      367be40a
  12. 02 Oct, 2018 1 commit
    • Matthew Leeds's avatar
      man/create-usb: Don't recommend summary updates · 039bbe56
      Matthew Leeds authored
      This commit removes the recommendation in the create-usb man page for
      the user to update the summary in the source repo before using the
      create-usb command. I'm not sure where I got the idea that create-usb
      depends on a summary in the source repo. I went back to the first commit
      that introduced the create-usb command and even using that a summary
      isn't required, so it seems unlikely that this changed recently.
      
      This is good news because the exclusive lock that's taken for summary
      updates has been causing problems on Endless (due to other processes
      having a lock for the duration of the 30 second acquire time out
      period).
      
      Closes: #1746
      Approved by: cgwalters
      039bbe56
  13. 01 Oct, 2018 2 commits
    • Colin Walters's avatar
      Only verify OSTREE_MAX_METADATA_SIZE for HTTP fetches · 2c55bc69
      Colin Walters authored
      There are use cases for libostree as a local content store
      for content derived or delivered via other mechanisms (e.g. OCI
      images, RPMs, etc.).  rpm-ostree today imports RPMs into OSTree
      branches, and puts the RPM header value as commit metadata.
      Some of these can be quite large because the header includes
      permissions for each file.  Similarly, some OCI metadata is large.
      
      Since there's no security issues with this, support committing
      such content.
      
      We still by default limit the size of metadata fetches, although
      for good measure we make this configurable too via a new
      `max-metadata-size` value.
      
      Closes: https://github.com/ostreedev/ostree/issues/1721
      
      Closes: #1744
      Approved by: jlebon
      2c55bc69
    • Matthew Leeds's avatar
      bash-completion: Fix --repo autocomplete · 5cada0f0
      Matthew Leeds authored
      This commit fixes the bash tab completion handling of the "--repo"
      argument. Before this commit, the completion only works if "--repo"
      comes after the main command. After this commit, you can use "--repo"
      directly after "ostree" in the command line, as is natural.
      
      Closes: #1745
      Approved by: jlebon
      5cada0f0
  14. 28 Sep, 2018 6 commits
    • Jonathan Lebon's avatar
      lib/mutable-tree: Port to new style · 39d5db7e
      Jonathan Lebon authored
      Some therapeutic style conversion to finish off the week. Pretty
      straightforward overall.
      
      Closes: #1742
      Approved by: cgwalters
      39d5db7e
    • Colin Walters's avatar
      lib/progress: Fix leak of GSource · 899b0bfa
      Colin Walters authored
      Closes: https://github.com/ostreedev/ostree/issues/1738
      
      Closes: #1741
      Approved by: jlebon
      899b0bfa
    • Jonathan Lebon's avatar
      ci: Bump rpm-ostree tag to 2018.8 · 6e3d4f90
      Jonathan Lebon authored
      2018.7 started failing `test-ucontainer.sh`. I don't have the cycles to
      look more deeply into what was going on there, but bumping to 2018.8
      fixes it at least. (And of course, it's passing in rpm-ostree too.)
      
      Closes: #1728
      Approved by: cgwalters
      6e3d4f90
    • Matthew Leeds's avatar
      man/ostree.repo-config: Document locking options · 1a7536c0
      Matthew Leeds authored
      This commit documents the "locking" and "lock-timeout-secs" options
      which have been around for a few releases.
      
      Closes: #1737
      Approved by: jlebon
      1a7536c0
    • Matthew Leeds's avatar
      lib/repo: Fix minor mistake in locking docs · 7892d35c
      Matthew Leeds authored
      The config option is "lock-timeout-secs" not "lock-timeout".
      
      Closes: #1737
      Approved by: jlebon
      7892d35c
    • Matthew Leeds's avatar
      lib/repo: Allow disabling lock timeout · 0c8a6d64
      Matthew Leeds authored
      Currently the locking code checks if the value -1 was set for the config
      key "lock-timeout-secs" and if so, a thread trying to acquire a lock
      will block indefinitely. Positive values specify how long to attempt to
      acquire a lock in a non-blocking way (the attempt is made once every
      second). But when the value is read from the config file,
      g_ascii_strtoull() is used, which converts it to an unsigned integer.
      This commit makes libostree use g_ascii_strtoll() instead, so that it's
      possible to set that key to -1 as intended.
      
      Closes: #1737
      Approved by: jlebon
      0c8a6d64
  15. 25 Sep, 2018 3 commits
    • Matthew Leeds's avatar
      create-usb: Add a --commit option · 9a06c540
      Matthew Leeds authored
      Currently on Endless OS, the OSTree ref for the operating system is
      something like os/eos/amd64/eos3, so that's what gets passed to `ostree
      create-usb` when copying the OS to a USB drive (for offline updates).
      However, when eos-updater checks for updates it pulls the metadata for a
      candidate commit and in so doing updates that eos3 ref to point to the
      partial commit being examined as a potential update rather than the
      deployed commit. This causes `ostree create-usb` to fail with an error
      like "No such metadata object
      7fb045cb2d1f1f3a81bfc157c6128ff443eb56350315b9536bdb56aee0659863.dirtree".
      
      OSTree creates deployment refs that look like "ostree/1/1/0" to maintain
      a pointer to the deployed commit, but create-usb can't use these because
      it shows up in the summary as just a ref, not a collection-ref.
      
      So this commit adds a --commit option to the create-usb command, so we
      can use the appropriate ref but copy the deployed commit rather than a
      (potentially partial) update commit.
      
      Closes: #1735
      Approved by: cgwalters
      9a06c540
    • Colin Walters's avatar
      lib/commit: Don't copy xattrs for metadata objects · c141fe61
      Colin Walters authored
      Copying the xattrs on metadata objects is wrong in general, we
      don't "own" them.  Notably this would fail in the situation of
      doing a pull from e.g. a `bare-user` source to a destination
      that was on a different mount point (so we couldn't hardlink),
      and the source had e.g. a `security.selinux` attribute.
      
      Closes: #1734
      
      Closes: #1736
      Approved by: jlebon
      c141fe61
    • Matthew Leeds's avatar
      create-usb: Always use archive mode · fc357adb
      Matthew Leeds authored
      Change the create-usb command so that it always creates the destination
      repository using the "archive" mode, rather than using archive mode when
      xattrs aren't supported and bare-user otherwise. This has a few
      advantages:
      
      1. The archive mode works with FAT filesystems, which is what most
      USB drives are, and which doesn't support xattrs.
      
      2. At least in some quick testing I did, archive mode is about
      twice as performant as bare-user mode, in terms of how long it takes for
      the create-usb command to complete.
      
      3. This ensures that a tool can safely change the permissions on
      ".ostree/repo" and subdirectories after create-usb completes, which is
      important for Endless since otherwise you can't use `ostree create-usb`
      as root and then `flatpak create-usb` as a non-root user on the same USB
      drive (or in other words copy OS updates and apps to the same USB).
      
      Closes: #1733
      Approved by: cgwalters
      fc357adb
  16. 21 Sep, 2018 6 commits