Commit 719e41a2 authored by Louis-Philippe Véronneau's avatar Louis-Philippe Véronneau Committed by Stefano Rivera

Document the USB installer in sphinx

parent f0d29958
.. _advanced_usage:
Advanced Usage
==============
......
.. _gateway:
Adding a Gateway
================
......
.. _getting_started:
Getting Started
===============
......
......@@ -10,4 +10,5 @@ Ansible Documentation
gateway.rst
opsis.rst
advanced_usage.rst
usb.rst
contact.rst
.. _physical_layout:
The Physical Layout
===================
......
.. _usb:
Installing machines with a USB installer
========================================
Installing Debian manually on machines and running ansible by hand can be
tedious and error-prone.
Although :ref:`our advanced usage documentation <advanced\_usage>` covers
imaging via PXE, you might find that for a small number of machines, using a
pre-built USB installer works best. It's also useful if you want to bootstrap
the PXE server.
Like the PXE server, this installer first installs a preseeded image and then
runs ansible when the machine first boots.
The ``usbinst`` directory contains the required scripts to make such a USB
installer and to test it properly.
``mk_usb_installer.sh``
-----------------------
This is the main script to build the USB installer. It makes a USB installer
with a preseed configuration and runs ansible. It currently supports Debian and
Ubuntu.
The script is not self contained and needs bits & pieces from the ansible
repository and from the internet to work properly. It takes for granted that you
already have a working ansible setup.
Disclaimer: it uses ``sudo`` to write to the USB key, so you may be prompted for
a ``sudo`` password.
Here is a summary of what it does:
1. Generates a ``late_command.cfg`` file in ``roles/tftp-server/files/scripts``
for ansible to configure the ansible ran in ``late_command.sh``
2. Downloads a installation image (Debian or Ubuntu)
3. Sets kernel parameters to preseed the installer
4. Formats and mounts the USB key
5. Installs the installation image on the USB key
6. Unmounts the USB key
7. Starts a simple web server on port 8007 using the :ref:`httpserver` script
to serve additional content from ``roles/tftp-server/files/``
The Debian Installer will fetch preseed configuration and `late_command scripts`
from ``roles/tftp-server/files/``, so the machine you are using to build the USB
installer needs to be available on the same network as the target machine,
until d-i is done.
Usage
^^^^^
You will need to install these dependencies to be able to run the script
properly::
$ sudo apt install curl git pmount dcfldd syslinux-efi syslinux-common
The script has two parameters:
``usb-device``: path to the usb device (for example: ``/dev/sdb``).
``config-filename``: path to the configuration file containing the settings to build the machine and run ansible.
You can call it this way::
$ ./mk_usb_installer.sh usb-device config-filename
Configuration File
^^^^^^^^^^^^^^^^^^
Recommended parameters
""""""""""""""""""""""
The following parameters should most of the time be explicitly set in your
``config-filename`` file:
``hostname``: (default: voctotest). Hostname of the target machine. If your DHCP server attributes a hostname to the machine, this value will be overwritten by it.
``playbook_repo``: (default: https://salsa.debian.org/debconf-video-team/ansible). The repository containing the playbooks for setting up the target machine.
``inventory_repo``: (default: blank). The repository containing the inventory/hosts files to use. If it is blank, the inventory in the playbook repository is used.
``timezone``: (default: Europe/Brussels). Timezone string respecting tzdata format.
``username``: (default: videoteam). Username of the main user.
``user_password_crypted``: (default: changeme). Crypted password using ``mkpasswd -m sha-256``.
Optional parameters
""""""""""""""""""""
These other parameters can be useful, but their defaults are more sane and can be
left as is:
``arch``: (default: amd64). Architecture of the target machine.
``bootdev``: (default: /dev/sda). The device on the target machine to install the bootloader on.
``disk``: (default: /dev/sda). The device on the target machine to install onto.
``debian_ver``: (default: 9.7.0). Debian version number.
``suite``: (default: stretch). Distribution suite to use.
``booting_loc``: (default: http://deb.debian.org/debian/dists/${suite}/main/installer-${arch}/current/images) Location of the boot images with the ``SA256SUMS`` and ``hd-media/boot.img.gz`` boot image files.
``iso``: (default: debian-${debian_ver}-${arch}-netinst.iso) Name of the ISO file
``iso_loc``: (default: https://cdimage.debian.org/debian-cd/current/${arch}/iso-cd) URL to the directory containing the ISO file.
``playbook_branch``: (default: master). The git branch in the playbook repository to use.
``inventory_repo_dir``: (default: ${playbook_repo}/inventory). The path to the inventory repository to use.
``load_firmware``: (default: false). Boolean. When true, d-i will prompt the user to provide non-free firmware blobs.
``vault_pw``: (default: blank). The password for ansible vault encoded in base64.
``preseed``: (default: url=$(hostname):8007). The location of the http server with additional preseed configuration and additional files. This is the machine the ``http_server.sh`` script is running on, or a PXE server configured by the tftp-server role. Specify a ``hostname:port``.
``more_appends``: (default: blank). Additional configuration to be appended to the kernel command line.
Step by step
^^^^^^^^^^^^
The script needs to be run in the ``usbinit`` directory::
$ cd usbinst
You can then create a new ``config-filename`` to set preseed values::
$ cp configs/blank.cfg configs/mybox.cfg
Once that is done, you can change the values listed before that seem important
to you. Finally, you can run the installer and use the key to image machines::
$ ./mk_usb_installer.sh sdb configs/mybox.cfg
.. _httpserver:
``http_server.sh``
------------------
This script runs an HTTP server so that the USB installer can pull additional
configuration files. The root directory being served is
``roles/tftp-server/files``.
``test_pxe.sh``
---------------
This script tests the PXE booting environment by booting it in a QEMU x86_64
system. You will need to install those dependencies to run it properly::
$ sudo apt install qemu brctl
You can then run the script this way::
$ ./test_pxe.sh
``test_thumb.sh``
-----------------
This script tests the USB stick by booting it in a QEMU x86_64 system. You will
need to install this dependency to run it properly::
$ sudo apt install qemu qemu-utils qemu-system ovmf
It only takes one argument - path to the USB device, for example ``/dev/sdb``::
$ ./test_thumb.sh /dev/sdb
# Scripts
The scripts in this directory accomplish various tasks to do with setting up
the initial USB installer and testing the setup using QEMU.
## `mk_usb_installer.sh`
Makes a USB installer that loads a pressed and runs ansible. Currently supports
Debian. It is not self contained. It expects networking to get the preseed, deb
repos, `late_command` and ansible.
./mk_usb_installer.sh usb-device [config-filename]
`usb-device`: dev of usb stick to clobber with no `/dev/` prefix (for example
sdc).
`config-filename`: settings to build the machine and run ansible:
- distro/suite, where to get it, proxy
- where to get preseed file
- drive to install to (sda, nvme0n1...)
- non-free firmware setting that I am afraid to hardcode
- ansible playbook and inventory repos
### How to use
1. `sudo apt install curl git pmount dcfldd`
2. fork and clone the inventory repo
3. adjust ansible inventory file, host and group vars, commit and push back to
your public repo
4. `cd usbinst`
5. `cp configs/blank.cfg configs/mybox.cfg`
6. `vim configs/mybox.cfg`
- set `inventory_repo` to your public repo
- set `inventory_repo_dir` to a dir in your public repo, defaults to inventory/.
- set `hostname` to the target's hostname
- and consider the other settings.
7. `./mk_usb_installer.sh sdb configs/mybox.cfg`
`mk_usb_installer.sh` script will:
* setup bootable usb stick
* run a web server to serve up the preseed, early, `late_command.sh`
Boot target machine from the usb stick.
It will do this:
* install the OS using values from `mybox.cfg` and `preseed.cfg`
* `wget/run late_command.sh`
* `late_command.sh` will clone the repos
* and run something like: `ansible --local --limit=$(hostname)`
### Configuration File:
**hostname** - what the hostame will be. used by ansible (see above.) udefaults to voctotest, which will bring up a simple vocto box. To make the installer prompt, add a "?" to syslinux.cfg like this: hostname?=
**suite** - distro suite to use (stretch)
**arch** - architecture of the target machine (amd64)
**booting\_loc** - location of the boot images with the `SA256SUMS` and
`hd-media/boot.img.gz` files (http://deb.debian.org/debian/dists/${suite}/main/installer-${arch}/current/images)
**debian_ver** - debian version, currently 9.5.0.
**iso** - the ISO name to download (debian-${debian_ver}-${arch}-netinst.iso)
**iso\_loc** - the URL to the directory containing the ISO. (https://cdimage.debian.org/debian-cd/current/${arch}/iso-cd)
**preseed** - how the installer gets the file (defaults to http from this box).
There are several different options for this.
Easy:
Leave this as is. It will use the server run at the end of this script.
preseed="url=$(hostname):8007"
No local DNS:
preseed="url=$(hostname).local:8007"
preseed="url=10.100.7.247:8007"
Host the file on some other server hostname dc10b:
Set the other server up and point the configuration to its hostname.
preseed="url=dc10b"
Use the file on the USB stick:
`early_command` and `late_command` use `$url`, so they will changing in the
script.
preseed="file=preseed.cfg"
**disk** - the device on the target machine to install onto (/dev/sda)
**bootdev** - the device on the target machine to boot from (/dev/sda)
**playbook\_repo** - the repo containing the playbooks for setting up the target
machine (https://salsa.debian.org/debconf-video-team/ansible)
**playbook\_branch** - the git branch in the playbook repo to use (master)
**username** - username of the main user
**user\_password\_crypted** - password crypted using `mkpasswd -m sha-256`
**timezone** - timezone string respecting tzdata format
**inventory\_repo** - the repo containing the inventory/hosts file to use. If
it is blank, the playbook repo is used.
**inventory\_branch** - the git branch in the inventory repo to use. If it is
blank, the playbook repo branch is used.
**vault\_pw** - the password for ansible vault encoded in base64
**load\_firmware** - prompt about firmware in the installer (false)
**more\_appends** - additional configuration to be appended to the kernel
### Supported Distributions
#### Debian
This is the default configuration and does not require a configuration file if
none of the defaults need to be changed
#### Ubuntu
The default Xenial configuration is in `configs/xenial.cfg`. This can be
used for Artful by changing the suite. Unfortunately the Xenial ISO is too big
to fit, as `boot.img` only has 782M of free space. Thus `mk_usb_installer.sh`
will be unable to create a USB for Xenial. The config is kept here for when we
find a way of fixing this bug.
## `http_server.sh`
This runs an HTTP server so that the USB installer can pull the configuration
into the install. The configuration is written to the
[../roles/tftp-server/files/](../roles/tftp-server/files) directory by the
`mk_usb_installer.sh` script.
## `test_pxe.sh`
This tests the PXE booting environment by booting it in a qemu x86\_64 system.
$ ./test_pxe.sh
It requires qemu and brctl
$ sudo apt install qemu brctl
## `test_thumb.sh`
This tests the USB stick by booting it in a qemu x86\_64 system. It takes one
argument - the device, without the preceding `/dev/`. If the USB is `/dev/sdb`:
$ ./test_thumb.sh sdb
It requires qemu:
$ sudo apt install qemu
......@@ -2,8 +2,8 @@
# template of default parameters that can be overridden
# hostname=voctotest
# username=voctotest
# user_password_crypted=\$5\$Wif9syMJU1FcbvI\$zD4fP9LDI2rZoiTJMODG59WUdtulZaelqU1xHoiJFtC # 1234567890
# username=videoteam
# user_password_crypted=\$5\$Fh87LcPw74EXo\$cepyGtRvrEsKGd4zKeXXL3nyRUD7.LR9q9yk1Kboh20 # changeme
# timezone=Europe/Brussels
# domain=local.lan
......
......@@ -6,20 +6,21 @@ set -eux
# expects networking in order to get:
# preseed, deb repos, late_command, ansible, blobs
# $1 - dev of usb stick to clobber (like sdc, no /dev/ prefex)
# $1 - /dev/sdx for usb stick to clobber
# $2 - optional filename of parameters
dev=$1
if [ ! -b "/dev/${dev}" ]; then
echo "error: /dev/${dev} is not a block device."
label=${dev#/dev/}
if [ ! -b "${dev}" ]; then
echo "error: ${dev} is not a block device."
exit
fi
# default parameters that can be over ridden by $2 passed config file
hostname=voctotest
username=voctotest
user_password_crypted=\$5\$Wif9syMJU1FcbvI\$zD4fP9LDI2rZoiTJMODG59WUdtulZaelqU1xHoiJFtC # 1234567890
username=videoteam
user_password_crypted=\$5\$Fh87LcPw74EXo\$cepyGtRvrEsKGd4zKeXXL3nyRUD7.LR9q9yk1Kboh20 # changeme
timezone=Europe/Brussels
domain=local.lan
......@@ -117,17 +118,16 @@ grep ${iso} SHA256SUMS > ${iso}.SHA256SUM
sha256sum --check ${iso}.SHA256SUM
)
zcat ${cache}/hd-media/boot.img.gz|sudo dcfldd of=/dev/${dev}
zcat ${cache}/hd-media/boot.img.gz|sudo dcfldd of=${dev}
# mount the installer media
pmount /dev/${dev} ${dev}
pmount ${dev}
# fix boot.img (now on the usb dev) to be uefi bootable:
# workaround https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=913523
# work towards upstream fix:
# https://salsa.debian.org/carlfk-guest/debian-installer/commits/hd-efi
sudo apt install syslinux-efi syslinux-common
target=/media/$dev
target=/media/$label
mkdir -p $target/EFI/BOOT $target/EFI/syslinux
cp /usr/lib/SYSLINUX.EFI/efi64/syslinux.efi $target/EFI/BOOT/BOOTX64.EFI
cp /usr/lib/syslinux/modules/efi64/ldlinux.e64 $target/EFI/BOOT/
......@@ -135,7 +135,7 @@ cp /usr/lib/syslinux/modules/efi64/*.c32 $target/EFI/syslinux
# end workaround bug=913523
# append $appends to APPEND line
sed "\|^APPEND|s|$| fb=false ${appends}|" syslinux.cfg | tee /media/${dev}/syslinux.cfg
sed "\|^APPEND|s|$| fb=false ${appends}|" syslinux.cfg | tee /media/${label}/syslinux.cfg
# skip over pesky ubuntu builds that don't play well
case $suite in
......@@ -146,21 +146,21 @@ case $suite in
# you will have to figure out how to put it on a 2nd usb stick,
# or the unused space on the 8gig usb stick.
echo "Your problem to get ${iso} on the target box."
pumount /dev/${dev}
pumount ${dev}
exit
;;
*)
cp ${cache}/${iso} ${cache}/${iso}.SHA256SUM /media/${dev}
cp ${cache}/${iso} ${cache}/${iso}.SHA256SUM /media/${label}
# check the iso file, make sure it copied ok.
(cd /media/${dev}
(cd /media/${label}
sha256sum --check ${iso}.SHA256SUM
)
;;
esac
pumount /dev/${dev}
pumount ${dev}
sync
echo Starting HTTP server. Please boot the target machine from the USB now.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment