Commit Graph

1117 Commits

Author SHA1 Message Date
David Runge
960b988ac6
Merge remote-tracking branch 'nl6720/ARCHISO_GNUPG_FD-unbound-variable'
* nl6720/ARCHISO_GNUPG_FD-unbound-variable:
  mkarchiso: fix unbound variable when not using GPG signing
2021-06-05 11:38:29 +02:00
nl6720
62e74c7e79
mkarchiso: fix unbound variable when not using GPG signing
Fixes #138
2021-06-05 08:23:14 +03:00
David Runge
a9a4dc4f70
Merge remote-tracking branch 'nl6720/env-i'
* nl6720/env-i:
  archiso/mkarchiso: clear environment variables when chrooting
2021-06-04 17:37:22 +02:00
nl6720
3d6651abf7
archiso/mkarchiso: clear environment variables when chrooting
Environment variables can interfere with the commands in chroot.
This causes issues when the environment variables specify a path that does not exist inside the chroot, e.g. if TMPDIR is set to a custom value.

Fixes https://bugs.archlinux.org/task/70580 .
2021-06-01 16:48:54 +03:00
David Runge
0bc62940c8
Merge branch 'changelog/55'
* changelog/55:
  Add changelog for v55
2021-05-30 23:10:45 +02:00
David Runge
5751de9753
Add changelog for v55
CHANGELOG.rst:
Add changelog entries for v55.
2021-05-30 19:44:55 +02:00
David Runge
50c5a023a4
Merge branch 'mr/118'
* mr/118:
  Add pv to the list of releng packages
2021-05-30 19:44:29 +02:00
David Runge
d3b06a0aaa
Add pv to the list of releng packages
configs/releng/packages.x86_64:
Add pv, as it was not added in the context of !118.
2021-05-30 11:17:45 +02:00
David Runge
ef9207d02b
Merge remote-tracking branch 'nl6720/baseline-erofs'
* nl6720/baseline-erofs:
  configs/baseline: use EROFS based image
2021-05-30 00:52:37 +02:00
nl6720
0b94b049e9
configs/baseline: use EROFS based image
Showcase #59.
2021-05-29 18:04:15 +03:00
David Runge
85c7cbee69
Merge remote-tracking branch 'nl6720/networkd-replace-type-with-name'
* nl6720/networkd-replace-type-with-name:
  configs/releng/airootfs/etc/systemd/network/: match by globbing the interface name instead of matching the type
2021-05-29 16:55:35 +02:00
nl6720
3b2618ff84
configs/releng/airootfs/etc/systemd/network/: match by globbing the interface name instead of matching the type
Type=ether matches virtual Ethernet interfaces (veth*) which may break networking inside containers.

Fixes https://bugs.archlinux.org/task/70892 .

Partially reverts 8a521d0bfa .
2021-05-29 15:31:02 +03:00
David Runge
254194ad6f
Merge branch 'issues/135'
* issues/135:
  Adjust documentation on IRC support channel
2021-05-29 14:25:20 +02:00
David Runge
0e61ed0a65
Adjust documentation on IRC support channel
README.rst:
Update the documentation on IRC support channel, after Arch Linux has moved to Libera Chat.
2021-05-29 10:48:06 +02:00
David Runge
e3a7f02385
Merge remote-tracking branch 'sxw/copytoram'
* sxw/copytoram:
  Apply copytoram Boot Option Menu Entries also to releng
  Fence add_binary for pv
  Reorder UEFI boot menu entries
  Update README.bootparams
  Apply 1 suggestion(s) to 1 file(s)
  add optional pv tool
2021-05-29 10:47:24 +02:00
Simon Wilper
31427eca7a Apply copytoram Boot Option Menu Entries also to releng 2021-05-14 19:47:25 +02:00
Simon Wilper
15f4ef4938 Fence add_binary for pv 2021-05-14 19:47:25 +02:00
Simon Wilper
9c783a8e71 Reorder UEFI boot menu entries 2021-05-14 19:47:25 +02:00
Simon Wilper
16742d1290 Update README.bootparams
Explain the purpose of optionally adding the "pv" program to the package
list.
2021-05-14 19:47:25 +02:00
Simon Wilper
c9cdb6d4b9 Apply 1 suggestion(s) to 1 file(s) 2021-05-14 19:46:29 +02:00
Simon Wilper
bb503b9030 add optional pv tool
use pv to give feedback on copying the airootfs to RAM when copytoram
kernel parameter is given
2021-05-14 19:46:23 +02:00
David Runge
28ab118099
Add changelog for v54
CHANGELOG.rst:
Add changelog entry for v54.
2021-05-13 22:18:47 +02:00
David Runge
9827cb19bb
ci: Use environment variables to override build settings
.gitlab-ci.yml:
Use environment variables to override the build settings offered by the build-host.sh script.
Lower the RAM usage of the builds to 3072.
Lower the COW space used by archiso to 2GiB.
Remove dependency on libisoburn in the `before_script` section of the build target (libarchive and util-linux have
replaced this requirement).
Use size units in accordance with upstream documentation.
Use the new name of the build's metrics file.

Fixes #134
2021-05-13 18:32:22 +02:00
David Runge
cde7296e6a
ci: Consolidate build-host script
.gitlab/ci/build-host.sh:
Change all script-local variables to lower-case and make some of them overridable using environment variables (by
providing defaults).
Break down overly long commands by splitting them into a list of strings.
Use local variables where possible.
Change `main()` to use rsync instead of cp to copy the project to the build location more generically.
Change `main()` to use rsync instead of cp to copy the build artifacts on the VM from the project's build directory to
the output.
Remove all unnecessary `function` keywords for function declarations.
Replace the dependency on libisoburn's `xorriso` with libarchive's `bsdtar` and util-linux's `blkid` in
`prepare_boot()`.
Add `print_section_start()` and `print_section_end()` to reduce code duplication and error potential when printing lines
for gitlab's collapsible sections (https://docs.gitlab.com/ee/ci/jobs/#custom-collapsible-sections).
Document the script's behavior and expectations.
Document the understood environment variables and add links to documentation on understood units (in case of size
units).
2021-05-13 18:32:10 +02:00
David Runge
8e44a8b72e
ci: consolidate build script
.gitlab/ci/build-inside-vm.sh:
Add `print_section_start()` and `print_section_end()` to remove code duplication when printing gitlab collapsible
sections (https://docs.gitlab.com/ee/ci/jobs/#custom-collapsible-sections).
Document further script dependencies.
Remove the temporary directory base (located in the project directory) instead of only the tempdir.
Simplify setting file and directory ownership when running with sudo, by only doing it once, recursively on the output
directory.
Make the script's output more verbose by using verbose flags for removal, ownership changes and zsyncmake actions and by
displaying e.g. created metrics and checksum files.
Change `create_metrics()` to output to a `metrics.txt` by default.
Change `create_checksums()` to create the checksums relative to the files.
2021-05-13 18:31:39 +02:00
nl6720
d0d4fa56cb
configs/releng/airootfs/etc/systemd/system/pacman-init.service: don't hardcode the keyrings
If the keyring is not specified, pacman-key will simply use all keyrings from /usr/share/pacman/keyrings/.

Fixes #133.
2021-05-12 10:24:35 +03:00
nl6720
3678bba899
configs/releng/packages.x86_64: explicitly add wanted packages instead of relying on pulling them in as dependencies
Fixes #119.
2021-05-11 23:51:00 +03:00
nl6720
bd2b861aa3
configs/*: add VM guest packages and enable their services
* virtualbox-guest-utils-nox package and vboxservice.service for VirtualBox.
* qemu-guest-agent package and qemu-guest-agent.service for QEMU & libvirt.

Implements #118.
2021-05-11 21:47:12 +03:00
David Runge
f86cb0faa2
ci: Expand parallel matrix with build modes
.gitlab-ci.yml:
Expand the parallel matrix with build modes per profile.
Do not run netboot with the baseline profile, as codesigning is not supported yet (#132).
Remove tagging secure, as archiso builds will only ever be used for testing the project.
Build ISOs using fast-single-thread (they take very long otherwise and hit the job time limit) and use any available
runner for anything else.
2021-05-11 18:09:50 +02:00
David Runge
5630a23ba1
ci: build based on buildmodes
.gitlab/ci/build-inside-vm.sh:
Expand the script to allow building based on mkarchiso's buildmodes.
Accept two parameters now: profile and buildmode.

Add gitlab collapsable sections by adding specific printfs for it to all functions.
Add extglob to be able to more specifically target files.
Change `create_checksums()` to also generate the legacy md5 and sha1 checksums.
Change `create_checksums()` and `create_zsync_delta()` to accept and process an unspecified amount of files as
parameters.
Change `create_zsync_delta()` to lower the blocksize when creating a file for the bootstrap image.
Change `create_metrics()` to create metrics depending on build mode.
Rename `create_temp_pgp_key()` to `create_ephemeral_pgp_key()`.
Add `create_ephemeral_codesigning_key()` to create a codesigning key, that is used to sign the netboot artifacts.
Change `run_mkarchiso()` to also create an ephemeral codesigning key before running mkarchiso and to run
`create_checksums()` and `create_zsync_delta()` with files depending on build mode.

.gitlab/ci/build-host.sh:
Call `build-inside-vm.sh` using the PROFILE and BUILDMODE environment variables as parameters.
2021-05-11 16:44:23 +02:00
David Runge
6b11d7be7a
mkarchiso: Also create package list for netboot
archiso/mkarchiso:
Change `_make_pkglist()` to also generate the package list when using the netboot build mode.
2021-05-10 22:42:27 +02:00
David Runge
ace88aaaca
Update project documentation related to netboot
README.rst:
Add openssl as another dependency.
Mention the export of netboot artifacts in the project introduction.
Mention that archiso may be used on other operating systems as well.
2021-05-10 12:38:31 +02:00
David Runge
4d1e898833
docs: Add netboot to buildmodes documentation
docs/README.profile.rst:
Add documentation for the netboot buildmode.
2021-05-10 12:38:28 +02:00
David Runge
d54bf635cd
mkarchiso: Add buildmode to export netboot artifacts
archiso/mkarchiso:
Implement a buildmode to export artifacts required for netboot with IPXE.
When providing the buildmode 'netboot' via profiledef.sh or the `-m` option, all targets necessary to create an ISO
medium are built, but the components required for netboot are exported to the output dir.
Optionally, it is possible to provide a set of certificates for codsigning using the `-c` option, where the first file
is considered as the signer certificate and the second as the key.

Add `_export_netboot_artifacts()` to copy build artifacts to the output directory.
Add `_sign_netboot_artifacts()` to codesign the netboot artifacts in the work directory.
Add `_validate_requirements_buildmode_netboot()` to check for openssl.
Add `_build_iso_base()` to implement common function calls between the 'iso' and the 'netboot' buildmodes.
Add `_build_buildmode_netboot()` to make use of `_build_iso_base()`, (optionally) `_sign_netboot_artifacts()` and
`_export_netboot_artifacts()`.
Change `_build_buildmode_iso()` to make use of `_build_iso_base()`.
Add `-c` as an option to mkarchiso to read in a list of file names.
Unify the output of `_usage()` by using the same definition style for lists of strings provided to options that accept
them (e.g. `-c`, `-m`, `-p`).

Closes #128
2021-05-10 12:38:18 +02:00
David Runge
1fed84c795
Extend project overview with info on bootstrap images
README.rst:
Mention bootstrap images in the opening words of the project documentation.
Add awk, erofs-utils, findutils, gzip, libarchive, pacman and sed to dependencies.
2021-05-09 15:53:09 +02:00
David Runge
1630d76c87
docs: Add documentation for buildmodes and bootstrap build mode
docs/README.profile.rst:
Add documentation for the optional `buildmodes` array in profiledef.sh, the understood build modes `bootstrap` and
`iso` and the implicit default build mode 'iso'.
Add basic documentation for the bootstrap_packages.arch file.
Add missing backticks.
Fix indent.
2021-05-09 15:50:24 +02:00
David Runge
2cac53967b
mkarchiso: Implement buildmodes that allow building bootstrap images
archiso/mkarchiso:
Introduce a buildmodes array, that can be used to build towards more than one output artifact type.
Add a buildmode for building a bootstrap image (a compressed file containing a very minimal Arch installation).
The buildmodes can be set either using a `buildmodes` array in a `profiledef.sh` or by using the `-m` option flag to
mkarchiso and providing a space delimited, quoted list.
The 'iso' buildmode is always the default if no buildmodes are setup.
Implement building a bootstrap image, when using the 'bootstrap' `buildmode`, which uses a profile's
'bootstrap_packages.$arch' file to install packages using pacstrap and compressing it to a bootstrap image.
The name of the output file is currently constructed from the `iso_name` value by appending `-bootstrap`.

Replace the uses of `airootfs_dir` with the more generic `pacstrap_dir`, as the location denotes where pacstrap is
being used.
Replace uses of `img_name` with `image_name` and removing it from the global scope, so that it can be overridden per
each buildmode.
Rename `_cleanup_airootfs_dir()` to `_cleanup_pacstrap_dir()`.
Make `_run_once()` more generic by prepending the state files with a string defined by `run_once_mode`.
Add `_validate_requirements_buildmode_all()`, `_validate_requirements_buildmode_bootstrap()` and
`_validate_requirements_buildmode_iso()` to validate the general requirements of the different buildmodes.
Add `_build_bootstrap_image()` to generate the bootstrap image using bsdtar.
Rename `_build_iso()` to `_build_iso_image()` to fit the naming of the respective bootstrap function.
Extend `_read_profile()` to include the reading of bootstrap image specific packages from a file.
Extend `_validate_options()` to include testing of the bootstrap packages and running of validation functions for all
buildmodes.
Change `_set_overrides()` to override the buildmodes if they are specified via the `-m` option flag.
Change `_make_version()` to be used generically in all buildmodes.
Change `_make_pkglist()` to be used generically in all buildmodes.
Rename `_build_profile()` to `_build_buildmode_iso()` and set local variables that are specific to the buildmode, such
as `image_name`, `pacstrap_dir`, `run_once_mode` , `buildmode_packages` and `buildmode_pkg_list`.
Add `_build_buildmode_bootstrap()` and set local variables that are specific to the buildmode, such as `image_name`,
`pacstrap_dir`, `run_once_mode` , `buildmode_packages` and `buildmode_pkg_list`.
Add the `-m` option flag to the list of flags.

Closes #127
2021-05-09 15:50:08 +02:00
David Runge
535bc3c0da
baseline: Add bootstrap packages file
configs/baseline/bootstrap_packages.x86_64:
Add a packages file for bootstrap images using the baseline profile and add arch-install-scripts and base to it.
2021-05-09 11:38:03 +02:00
David Runge
b7fd696e1e
baseline: Add buildmodes to profiledef.sh
configs/baseline/profiledef.sh:
Add `buildmodes` array with default entry for the 'iso' buildmode.
2021-05-09 11:38:00 +02:00
David Runge
6afa6958f5
releng: Add buildmodes to profiledef.sh
configs/releng/profiledef.sh:
Add a `buildmodes` array to releng's profiledef.sh with the up-to-now default buildmode 'iso'.
2021-05-09 11:37:57 +02:00
David Runge
9c84b7ca5b
releng: Add packages file for bootstrap image
configs/releng/bootstrap_packages.x86_64:
Add packages file for bootstrap images and add arch-install-scripts and base.
2021-05-09 11:26:53 +02:00
David Runge
2a07aa2f24
Increase timeout for initial prompt in build VM
.gitlab/ci/build-host.sh:
Increase the allowed timeout for reaching the initial prompt in the build VM from 30 to 60 seconds.

Fixes #129
2021-05-06 10:57:18 +02:00
David Runge
4b14350e5f
Use QEMU 6.x options
scripts/run_archiso:
Change parameters to qemu's `-drive` option to make use of the explicit `read-only=on`, as the implicit `read-only` is
now obsolete.

Closes #126
2021-05-06 01:41:19 +02:00
David Runge
c58b44f016
Use QEMU 6.x options
.gitlab/ci/build-host.sh:
Change parameters to qemu's `-drive` option to make use of the explicit `read-only=on`, as the implicit `read-only` is
now obsolete.
2021-05-06 01:37:51 +02:00
David Runge
42cdf8674a
Set more generic output for signatures
archiso/mkarchiso:
Change the help output to reflect that the `-g` option is generically signing a rootfs (which may be e.g. squashfs or
erofs).
Change the output of `_mksignature()` to be more generic, as it signs any type of understood rootfs image (which may be
e.g. squashfs or erofs).
2021-05-01 17:16:56 +02:00
David Runge
cc735dbbfc
Force PGP signature file extension
archiso/mkarchiso:
Force the file extension in use for the PGP signatures of the rootfs to always be .sig.
When gnupg's 'armor' configuration option is used, the output otherwise defaults to using .asc.
As the verification hook in mkinitcpio-archiso expects the .sig file extension, verifying the rootfs will fail in that
scenario.
2021-05-01 16:56:53 +02:00
David Runge
73e3ccdb59
Add ephemeral signing key to CI setup
.gitlab/ci/build-inside-vm.sh:
Create an ephemeral signing key for signing the rootfs image (e.g. squashfs or erofs) when building the profiles.

Implements #125
2021-05-01 16:22:02 +02:00
David Runge
e2cce07df7
Add changelog for v53
CHANGELOG.rst:
Add changelog for v53
2021-05-01 10:24:54 +02:00
David Runge
9dbb600d4c
Add packages for unlocking LUKS2 volumes with systemd
configs/releng/packages.x86_64:
Add libfido2 for unlocking LUKS2 volumes with FIDO2 tokens.
Add tpm2-tss for unlocking LUKS2 volumes with TPM2.
2021-04-30 23:03:08 +02:00
David Runge
81da51835a
Add required packages to interact with smartcards
configs/releng/packages.x86_64:
Add libusb-compat and pcsclite as optional dependencies for gnupg to be able to interact with smartcards out-of-the-box.

Closes #122
2021-04-30 23:03:04 +02:00