Commit Graph

1084 Commits

Author SHA1 Message Date
David Runge
d54bf635cd
mkarchiso: Add buildmode to export netboot artifacts
archiso/mkarchiso:
Implement a buildmode to export artifacts required for netboot with IPXE.
When providing the buildmode 'netboot' via profiledef.sh or the `-m` option, all targets necessary to create an ISO
medium are built, but the components required for netboot are exported to the output dir.
Optionally, it is possible to provide a set of certificates for codsigning using the `-c` option, where the first file
is considered as the signer certificate and the second as the key.

Add `_export_netboot_artifacts()` to copy build artifacts to the output directory.
Add `_sign_netboot_artifacts()` to codesign the netboot artifacts in the work directory.
Add `_validate_requirements_buildmode_netboot()` to check for openssl.
Add `_build_iso_base()` to implement common function calls between the 'iso' and the 'netboot' buildmodes.
Add `_build_buildmode_netboot()` to make use of `_build_iso_base()`, (optionally) `_sign_netboot_artifacts()` and
`_export_netboot_artifacts()`.
Change `_build_buildmode_iso()` to make use of `_build_iso_base()`.
Add `-c` as an option to mkarchiso to read in a list of file names.
Unify the output of `_usage()` by using the same definition style for lists of strings provided to options that accept
them (e.g. `-c`, `-m`, `-p`).

Closes #128
2021-05-10 12:38:18 +02:00
David Runge
1fed84c795
Extend project overview with info on bootstrap images
README.rst:
Mention bootstrap images in the opening words of the project documentation.
Add awk, erofs-utils, findutils, gzip, libarchive, pacman and sed to dependencies.
2021-05-09 15:53:09 +02:00
David Runge
1630d76c87
docs: Add documentation for buildmodes and bootstrap build mode
docs/README.profile.rst:
Add documentation for the optional `buildmodes` array in profiledef.sh, the understood build modes `bootstrap` and
`iso` and the implicit default build mode 'iso'.
Add basic documentation for the bootstrap_packages.arch file.
Add missing backticks.
Fix indent.
2021-05-09 15:50:24 +02:00
David Runge
2cac53967b
mkarchiso: Implement buildmodes that allow building bootstrap images
archiso/mkarchiso:
Introduce a buildmodes array, that can be used to build towards more than one output artifact type.
Add a buildmode for building a bootstrap image (a compressed file containing a very minimal Arch installation).
The buildmodes can be set either using a `buildmodes` array in a `profiledef.sh` or by using the `-m` option flag to
mkarchiso and providing a space delimited, quoted list.
The 'iso' buildmode is always the default if no buildmodes are setup.
Implement building a bootstrap image, when using the 'bootstrap' `buildmode`, which uses a profile's
'bootstrap_packages.$arch' file to install packages using pacstrap and compressing it to a bootstrap image.
The name of the output file is currently constructed from the `iso_name` value by appending `-bootstrap`.

Replace the uses of `airootfs_dir` with the more generic `pacstrap_dir`, as the location denotes where pacstrap is
being used.
Replace uses of `img_name` with `image_name` and removing it from the global scope, so that it can be overridden per
each buildmode.
Rename `_cleanup_airootfs_dir()` to `_cleanup_pacstrap_dir()`.
Make `_run_once()` more generic by prepending the state files with a string defined by `run_once_mode`.
Add `_validate_requirements_buildmode_all()`, `_validate_requirements_buildmode_bootstrap()` and
`_validate_requirements_buildmode_iso()` to validate the general requirements of the different buildmodes.
Add `_build_bootstrap_image()` to generate the bootstrap image using bsdtar.
Rename `_build_iso()` to `_build_iso_image()` to fit the naming of the respective bootstrap function.
Extend `_read_profile()` to include the reading of bootstrap image specific packages from a file.
Extend `_validate_options()` to include testing of the bootstrap packages and running of validation functions for all
buildmodes.
Change `_set_overrides()` to override the buildmodes if they are specified via the `-m` option flag.
Change `_make_version()` to be used generically in all buildmodes.
Change `_make_pkglist()` to be used generically in all buildmodes.
Rename `_build_profile()` to `_build_buildmode_iso()` and set local variables that are specific to the buildmode, such
as `image_name`, `pacstrap_dir`, `run_once_mode` , `buildmode_packages` and `buildmode_pkg_list`.
Add `_build_buildmode_bootstrap()` and set local variables that are specific to the buildmode, such as `image_name`,
`pacstrap_dir`, `run_once_mode` , `buildmode_packages` and `buildmode_pkg_list`.
Add the `-m` option flag to the list of flags.

Closes #127
2021-05-09 15:50:08 +02:00
David Runge
535bc3c0da
baseline: Add bootstrap packages file
configs/baseline/bootstrap_packages.x86_64:
Add a packages file for bootstrap images using the baseline profile and add arch-install-scripts and base to it.
2021-05-09 11:38:03 +02:00
David Runge
b7fd696e1e
baseline: Add buildmodes to profiledef.sh
configs/baseline/profiledef.sh:
Add `buildmodes` array with default entry for the 'iso' buildmode.
2021-05-09 11:38:00 +02:00
David Runge
6afa6958f5
releng: Add buildmodes to profiledef.sh
configs/releng/profiledef.sh:
Add a `buildmodes` array to releng's profiledef.sh with the up-to-now default buildmode 'iso'.
2021-05-09 11:37:57 +02:00
David Runge
9c84b7ca5b
releng: Add packages file for bootstrap image
configs/releng/bootstrap_packages.x86_64:
Add packages file for bootstrap images and add arch-install-scripts and base.
2021-05-09 11:26:53 +02:00
David Runge
2a07aa2f24
Increase timeout for initial prompt in build VM
.gitlab/ci/build-host.sh:
Increase the allowed timeout for reaching the initial prompt in the build VM from 30 to 60 seconds.

Fixes #129
2021-05-06 10:57:18 +02:00
David Runge
4b14350e5f
Use QEMU 6.x options
scripts/run_archiso:
Change parameters to qemu's `-drive` option to make use of the explicit `read-only=on`, as the implicit `read-only` is
now obsolete.

Closes #126
2021-05-06 01:41:19 +02:00
David Runge
c58b44f016
Use QEMU 6.x options
.gitlab/ci/build-host.sh:
Change parameters to qemu's `-drive` option to make use of the explicit `read-only=on`, as the implicit `read-only` is
now obsolete.
2021-05-06 01:37:51 +02:00
David Runge
42cdf8674a
Set more generic output for signatures
archiso/mkarchiso:
Change the help output to reflect that the `-g` option is generically signing a rootfs (which may be e.g. squashfs or
erofs).
Change the output of `_mksignature()` to be more generic, as it signs any type of understood rootfs image (which may be
e.g. squashfs or erofs).
2021-05-01 17:16:56 +02:00
David Runge
cc735dbbfc
Force PGP signature file extension
archiso/mkarchiso:
Force the file extension in use for the PGP signatures of the rootfs to always be .sig.
When gnupg's 'armor' configuration option is used, the output otherwise defaults to using .asc.
As the verification hook in mkinitcpio-archiso expects the .sig file extension, verifying the rootfs will fail in that
scenario.
2021-05-01 16:56:53 +02:00
David Runge
73e3ccdb59
Add ephemeral signing key to CI setup
.gitlab/ci/build-inside-vm.sh:
Create an ephemeral signing key for signing the rootfs image (e.g. squashfs or erofs) when building the profiles.

Implements #125
2021-05-01 16:22:02 +02:00
David Runge
e2cce07df7
Add changelog for v53
CHANGELOG.rst:
Add changelog for v53
2021-05-01 10:24:54 +02:00
David Runge
9dbb600d4c
Add packages for unlocking LUKS2 volumes with systemd
configs/releng/packages.x86_64:
Add libfido2 for unlocking LUKS2 volumes with FIDO2 tokens.
Add tpm2-tss for unlocking LUKS2 volumes with TPM2.
2021-04-30 23:03:08 +02:00
David Runge
81da51835a
Add required packages to interact with smartcards
configs/releng/packages.x86_64:
Add libusb-compat and pcsclite as optional dependencies for gnupg to be able to interact with smartcards out-of-the-box.

Closes #122
2021-04-30 23:03:04 +02:00
nl6720
6287f72e8f
Remove docs/README.knownissues. Replaced by issue #83. 2021-04-30 20:51:24 +03:00
nl6720
76c80303b2
Remove docs/README.build. Superseded by README.rst 2021-04-30 20:51:24 +03:00
nl6720
a855dd4a5f
Move README.profile.rst to docs/
Keep all documentation except the main README in the docs directory.
2021-04-30 20:51:24 +03:00
nl6720
6294d1d985
Update README.profile.rst
* Don't nest code blocks inside quote blocks.
* Use monospace for paths, options, values, etc.
2021-04-30 20:51:24 +03:00
nl6720
5754000d2e
Update README.rst
* Don't nest code blocks inside quote blocks.
* Replace bash with sh, as there's nothing bash-specific in the examples.
* There is no syntax highlighting for grub, use sh.
* Use sentence case for headings.
* Use monospace for paths.
2021-04-30 20:51:23 +03:00
nl6720
06c3218786
configs/releng/syslinux/: increase serial baud rate to 115200
Additionally enable serial in baseline profile.

Related to #75.
2021-04-30 20:50:34 +03:00
nl6720
0406f9ca02
mkarchiso: create reproducible gzip archives
Use the gzip option -n/--no-name to prevent saving the original file name and timestamp.

Fixes #104.
2021-04-30 20:50:07 +03:00
nl6720
a771297e12
mkarchiso: make sure to remove potentially preexisting files from $airootfs_dir before creating them with output redirection
mkarchiso creates "${airootfs_dir}/etc/machine-id" by using output redirection. If this file is an existing symlink, then the printf output would be written to the symlink target. It can be a big issue in case the symlink resolves to a path outside ${airootfs_dir}.

Fixes #121.
2021-04-30 20:37:18 +03:00
nl6720
98c7b67697
mkarchiso: append IMAGE_ID and IMAGE_VERSION to /etc/os-release
This provides the ISO version information in the os-release file.

* IMAGE_ID is set to the value of $iso_name.
* IMAGE_VERSION is set to the value of $iso_version.

Implements #116.
2021-04-30 20:37:17 +03:00
nl6720
0ed1c61f1f
Add package count, El Torito EFI image size and initramfs image sizes to GitLab metrics
Metrics are now collected in build-inside-vm.sh since the files in question are in the work directory.

Implements #101 and #111.
2021-04-30 19:44:37 +03:00
David Runge
8bf95d37d3
Ignore SC3060 in initcpio hook
archiso/initcpio/hooks/archiso_pxe_common:
Disable shellcheck's SC3060, as ash is able to do bash-like string replacements.
2021-04-30 17:51:30 +02:00
David Runge
bde3971991
Fix shellcheck complains in CI scripts
.gitlab/ci/build-host.sh:
Change the readonly TMPDIR variable to a global tmpdir variable and set it in the `init()` function.

.gitlab/ci/build-inside-vm.sh:
Change assigning the readonly tmpdir variable directly to assigning it after declaring it.
Change `cleanup()` and `create_zsync_delta()` to use bash-style statements and also check whether SUDO_GID is set before
using it.
2021-04-30 17:51:27 +02:00
Christian Hesse
1a97109639 mkarchiso: also add iso name in grub environment block 2021-04-07 16:40:18 +00:00
nl6720
09b6127fe8
mkarchiso: use -isohybrid-gpt-basdat instead of -appended_part_as_gpt for ISOs that will support BIOS booting
Some hardware, like Lenovo Thinkpad T420, will not BIOS boot if the disk has a valid GPT.
See https://bbs.archlinux.org/viewtopic.php?id=264096 .

Instead of a valid GPT, change to a valid MBR and invalid GPT similar to what was used before 729d16b48c. That layout, despite having crazy partition tables, boots everywhere.
The difference is that -append_partition is still kept and specified before -isohybrid-gpt-basdat. Thus the appended partition will be listed as EFI system partition in MBR and as Microsoft basic partition in the invalid GPT.

Fixes #102.
2021-04-07 10:29:35 +03:00
David Runge
c8599788f0
Revert "configs/releng: improve UX"
This reverts commit 8b6f3545e3.
2021-03-30 22:40:37 +02:00
David Runge
495721a79c
Add changelog entry for v52
CHANGELOG.rst:
Add changelog entry for v52
2021-03-30 21:46:07 +02:00
nl6720
8b6f3545e3
configs/releng: improve UX
Implements #90
2021-03-30 21:09:44 +02:00
Giancarlo Razzolini
9875249e02 releng/packages: Add archinstall to the list of packages
Add archinstall to the list of packages thare are installed on the ISO.
2021-03-30 18:14:27 +00:00
David Runge
bc007ca5f3
Add releases section with PGP information
README.rst:
Add a "Releases" section that specifies who is creating releases and which PGP key ID is used to sign tags.
Additionally, information about how to retrieve the relevant public key and how to verify a tag in the repository is
added.

Fixes #114
2021-03-29 23:16:04 +02:00
David Runge
d178183c2e
Reduce amount of checked mirrors
configs/releng/airootfs/etc/xdg/reflector/reflector.conf:
Reduce the amount mirrors that reflector checks from 70 to 20.
This significantly reduces the time it takes to end up with an up-to-date mirrorlist during boot with the releng
profile.

Fixes #92
2021-03-29 21:00:06 +02:00
David Runge
0664efcf3c
Allow redirects for scripts
configs/releng/airootfs/root/.automated_script.sh:
Add the `--location` curl parameter (see `man 1 curl`) to allow for curl to retrieve a remote script even if the source
is being redirected (e.g. moved permanently) when using the `script=` kernel commandline parameter.

Fixes #113
2021-03-29 19:26:27 +02:00
nl6720
d86f8606dc
configs/releng: add and enable ModemManager
ModemManager's mmcli is the simplest way to connect with WWAN modems.

Mention mmcli in MOTD.

Implements #110.
2021-03-26 17:19:31 +02:00
nl6720
2c9eabd53f
configs/releng: start DHCP client for mobile broadband
Add /etc/systemd/network/20-wwan.network

Related to #110.
2021-03-26 12:28:07 +02:00
nl6720
edfb50ab6c
configs/releng: rename 20-wireless.network to 20-wlan.network
The file is limited to Wi-Fi (Type=wlan in networkd configuration).
2021-03-26 12:18:08 +02:00
nl6720
8a521d0bfa
Update configs/releng/airootfs/etc/systemd/network/20-{ethernet,wireless}.network
* Match the device type instead of the interface name.
* Replace DHCP section with DHCPv4/DHCPv6. systemd split the sections.
2021-03-26 11:35:20 +02:00
nl6720
8cbc548359
mkarchiso: do not set default mksquashfs options
Remove hardcoded '-comp xz', it prevents using mksquashfs defaults.

Fixes #112.
2021-03-26 08:37:38 +02:00
nl6720
8050fa4797
.gitlab/ci/build-host.sh: install erofs-utils
Allow building ISOs with EROFS airootfs images in CI.
2021-03-26 00:45:30 +02:00
nl6720
e847c4ad75
.gitlab/ci/build-host.sh: Do not upgrade packages with files in /usr/lib/modules/
Packages with files in /usr/lib/modules/ depend on the current kernel, thus they should not be updated.
2021-03-21 16:04:39 +02:00
Michael Gilchrist
6bb12552e4 Recursively change file permissions for folders listed in profiledef.sh
- if a folder listed in the associative array ends with a "/",
  recursively apply chmod and chown.
2021-03-21 14:00:13 +00:00
Alexander Epaneshnikov
e43017c955 add a sound card firmware 2021-03-14 20:34:43 +00:00
nl6720
702026e124
.gitlab-ci.yml: do not use build:secure on forks
Forks may not have access to secure runners. Restrict build:secure to https://gitlab.archlinux.org/archlinux/archiso/ only.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/106 .
2021-03-14 21:27:04 +02:00
nl6720
c241285c5f
.gitlab/ci/build-host.sh: increase mksquashfs timeout to 40 minutes
Some mksquashfs runs take a very long time and 1000 seconds might not be enough.
2021-03-14 20:20:46 +02:00
Jonathon Fernyhough
f502b56ec2 Create zsync control file for delta downloads 2021-03-14 17:42:54 +00:00