Commit Graph

399 Commits

Author SHA1 Message Date
hayao
025d6b828f mkarchiso: Fix mmd-command check
valication_error variable is not changed when mmd and/or mcopy are missing.
2022-01-08 02:18:09 +00:00
nl6720
dae5494ca1
mkarchiso: show "Code signing certificates: None" if no keys/certificates are specified
Fixes #151.
2021-12-10 13:15:13 +02:00
nl6720
a7f879fcdd
mkarchiso: use mkfs.erofs --quiet in quiet mode
erofs-utils 1.4 introduced a --quiet option.

Related to #148.
2021-11-22 12:04:35 +02:00
nl6720
8cfe29043f
mkarchiso: use mksquashfs -quiet instead of redirecting its stdout to /dev/null
mksquashfs supports a -quiet option since squashfs-tools 4.4.
Use this option in non-verbose mode instead of redirecting stdout of the whole command to /dev/null.

This allows to have only one instance of mksquashfs in _run_mksquashfs instead of multiple ones in if-then-else.

Related to #148.
2021-11-05 13:01:28 +02:00
nl6720
009140bd8b
mkarchiso: do not show subdirectory sizes in netboot mode
Only the total size is relevant.

Related to #148.
2021-11-05 12:44:03 +02:00
nl6720
5738f40aa9
mkarchiso: redirect command -v output to /dev/null
The output is irrelevant, we only need the return code.

Related to #148.
2021-11-05 12:44:01 +02:00
nl6720
ad6d65ab87
mkarchiso: silence xorriso's note about SOURCE_DATE_EPOCH
The `xorriso -as mkisofs` option `-quiet` is interpreted too late. Use native xorriso option `-report_about SORRY` instead and ensure it is the first option.

Related to #148.
2021-11-05 12:43:48 +02:00
nl6720
d2addb25c9
mkarchiso: silence mkfs.fat in quiet mode
mkfs.fat does not have a -q/--quiet option, so redirect its stdout to /dev/null instead.
See https://github.com/dosfstools/dosfstools/issues/103 .

Related to #148.
2021-11-04 07:58:12 +02:00
nl6720
662027ef80
mkarchiso: do not require setting boot modes when building only netboot artifacts
netboot does not need any of the boot loaders or their files.

Fixes #150.
2021-11-02 21:10:07 +02:00
nl6720
f8862c2621
mkarchiso: replace external date command with printf
printf is a bash builtin, so by using it an external command can be avoided.
Due to the differences between date(1) and strftime(3), the time zone output will not contain a colon anymore. Fortunately, that is still a supported format according to https://en.wikipedia.org/wiki/ISO_8601#Time_offsets_from_UTC .
2021-11-02 20:57:47 +02:00
nl6720
d2315bc98d
mkarchiso: error out of iso and netboot build modes if no boot modes are specified 2021-08-25 00:29:35 +03:00
nl6720
f3959d6ef0
mkarchiso: split out build mode specific checks from _validate_options to _validate_requirements_buildmode_*
The bootstrap build mode does not use packages.${arch}, ${bootmodes[@]} or $airootfs_image_type so there's no need to validate them.
Requirements common to iso and netboot are checked with the _validate_common_requirements_buildmode_iso_netboot function.

Fixes #149.

* Rename _validate_requirements_buildmode_all to _validate_common_requirements_buildmode_all to prevent potential conflicts since we are using _validate_requirements_buildmode_${_buildmode} to run the functions.
* Improve searching in an array. See https://stackoverflow.com/a/15394738.
2021-08-25 00:29:35 +03:00
nl6720
8d18d8f745
mkarchiso: fix unbound variable errors in _validate_options
$pkg_list_from_file and $bootstrap_pkg_list_from_file are arrays, they must be referenced as such. Fixes https://bugs.archlinux.org/task/71852.

Remove quotes from arithmetic expressions.
2021-08-25 00:29:35 +03:00
nl6720
59dffcf11a
mkarchiso: support setting gpg sender
Add new -G option to set gpg's --sender. This allows to see who signed the rootfs image without needing to import the gpg key from the keyring in initramfs.
2021-08-25 00:26:07 +03:00
nl6720
ea9572b98e
mkarchiso: add some sane gpg options to override those set in user's gpg.conf
* Add --batch, since gpg is run in a script.
* Add --no-armor (this is the default). Armored output provides no benifit here.
* Add --no-include-key-block (this is the default). There is no need to have the gpg key in the signature. The mkinitcpio hook will verify the signature against the included keyring.

Remove the output files before running gpg. Otherwise gpg --batch will fail if they exist.
2021-08-25 00:26:07 +03:00
nl6720
0a58431dde
mkarchiso: put version files in netboot artifacts
Fixes #147.
2021-08-24 23:47:00 +03:00
nl6720
b040ef1774
mkarchiso: ensure there are no existing image files before trying to create them
This could prevent a few issues when creating an image file when re-running mkarchiso after a failure.

Remove useless test before running rm -f. The command will always succeed.
2021-08-15 16:39:55 +03:00
nl6720
6185448477
mkarchiso: copy files to ext4 image using mkfs.ext4's -d option instead of mounting the file system
mkfs.ext4 with its -d option can "copy the contents of the given directory into the root directory of the filesystem".
This allows to get rid of the last directly used mount and umount commands in mkarchiso.

Additionally try to make the ext4 image somewhat reproducible by setting E2FSPROGS_FAKE_TIME to SOURCE_DATE_EPOCH, clearing the UUID and using a reproducible hash seed.
See https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/commit/?id=e1f7100643a46456be107b33098f6034b0835e6d .

Place mkfs.ext4 options in an array to avoid duplicating the command.

Related to #40.
2021-08-15 16:37:13 +03:00
David Runge
75d36d2124
Remove mkinitcpio-archiso files
archiso/initcpio/*:
Remove mkinitcpio-archiso scripts as they have been split out into a separate project.

docs/README.{altbootmethods,bootparams}:
Remove mkinitcpio-archiso specific documentation.
2021-07-31 16:58:57 +02:00
David Runge
0fa1c0e945
mkarchiso: Do not run _build_iso_base once
archiso/mkarchiso:
Do not run `_build_iso_base` with `_run_once`, as it sets variables which are important to functions that are called
after it.
2021-07-30 15:48:09 +02:00
nl6720
610ff2ac26
mkarchiso: improve functions and comments
* Merge _make_efi_dir_on_iso9660 into _make_bootmode_uefi-x64.systemd-boot.eltorito since it's specific to that boot mode.
* Split off FAT image creation into _make_efibootimg, so it can be reused by other functions.
* Add/improve comments to better document what the script does.
2021-07-29 17:12:47 +03:00
nl6720
1e5b3a57f5
mkarchiso: set LC_ALL instead of LANG
LC_ALL overrides LANG and all LC_* variables, so use it instead.
See https://www.gnu.org/software/gettext/manual/html_node/Locale-Environment-Variables.html#Locale-Environment-Variables .
2021-07-11 20:02:39 +03:00
nl6720
d65682a765
mkarchiso: unset TMPDIR to work around FS#70580
Alternative to 3d6651abf7 which caused issues.

Fixes https://bugs.archlinux.org/task/70580 .
2021-07-02 15:54:34 +03:00
nl6720
c4d1109e49
Revert "archiso/mkarchiso: clear environment variables when chrooting"
This reverts commit 3d6651abf7.

A missing $PATH breaks scripts that do not hardcode binary paths.
Fixes https://bugs.archlinux.org/task/71416 .
2021-07-02 15:54:33 +03:00
David Runge
19861e4742
archiso/mkarchiso: Generalize handling of ucodes
archiso/mkarchiso:
Generalize the handling of ucode files (e.g. copying, signing, size calculation) by introducing the global readonly
array of expected ucode names instead of hardcoding them in various functions.
When making use of ucode files in `_make_boot_on_iso9660()`, `_make_bootmode_uef-x86.systemd-boot.esp()` and
`_sign_netboot_artifacts()` create a list of files that is guaranteed to contain existing files.

Fixes #132
2021-06-30 14:36:31 +02:00
nl6720
cbac95ec9d
mkarchiso: pass SOURCE_DATE_EPOCH to chroot 2021-06-22 09:12:22 +03:00
nl6720
6e1ca35e59
Update ArchWiki article URLs
Use the new, prettier short URLs.
2021-06-18 21:17:47 +03:00
nl6720
62e74c7e79
mkarchiso: fix unbound variable when not using GPG signing
Fixes #138
2021-06-05 08:23:14 +03:00
nl6720
3d6651abf7
archiso/mkarchiso: clear environment variables when chrooting
Environment variables can interfere with the commands in chroot.
This causes issues when the environment variables specify a path that does not exist inside the chroot, e.g. if TMPDIR is set to a custom value.

Fixes https://bugs.archlinux.org/task/70580 .
2021-06-01 16:48:54 +03:00
Simon Wilper
15f4ef4938 Fence add_binary for pv 2021-05-14 19:47:25 +02:00
Simon Wilper
c9cdb6d4b9 Apply 1 suggestion(s) to 1 file(s) 2021-05-14 19:46:29 +02:00
Simon Wilper
bb503b9030 add optional pv tool
use pv to give feedback on copying the airootfs to RAM when copytoram
kernel parameter is given
2021-05-14 19:46:23 +02:00
David Runge
6b11d7be7a
mkarchiso: Also create package list for netboot
archiso/mkarchiso:
Change `_make_pkglist()` to also generate the package list when using the netboot build mode.
2021-05-10 22:42:27 +02:00
David Runge
d54bf635cd
mkarchiso: Add buildmode to export netboot artifacts
archiso/mkarchiso:
Implement a buildmode to export artifacts required for netboot with IPXE.
When providing the buildmode 'netboot' via profiledef.sh or the `-m` option, all targets necessary to create an ISO
medium are built, but the components required for netboot are exported to the output dir.
Optionally, it is possible to provide a set of certificates for codsigning using the `-c` option, where the first file
is considered as the signer certificate and the second as the key.

Add `_export_netboot_artifacts()` to copy build artifacts to the output directory.
Add `_sign_netboot_artifacts()` to codesign the netboot artifacts in the work directory.
Add `_validate_requirements_buildmode_netboot()` to check for openssl.
Add `_build_iso_base()` to implement common function calls between the 'iso' and the 'netboot' buildmodes.
Add `_build_buildmode_netboot()` to make use of `_build_iso_base()`, (optionally) `_sign_netboot_artifacts()` and
`_export_netboot_artifacts()`.
Change `_build_buildmode_iso()` to make use of `_build_iso_base()`.
Add `-c` as an option to mkarchiso to read in a list of file names.
Unify the output of `_usage()` by using the same definition style for lists of strings provided to options that accept
them (e.g. `-c`, `-m`, `-p`).

Closes #128
2021-05-10 12:38:18 +02:00
David Runge
2cac53967b
mkarchiso: Implement buildmodes that allow building bootstrap images
archiso/mkarchiso:
Introduce a buildmodes array, that can be used to build towards more than one output artifact type.
Add a buildmode for building a bootstrap image (a compressed file containing a very minimal Arch installation).
The buildmodes can be set either using a `buildmodes` array in a `profiledef.sh` or by using the `-m` option flag to
mkarchiso and providing a space delimited, quoted list.
The 'iso' buildmode is always the default if no buildmodes are setup.
Implement building a bootstrap image, when using the 'bootstrap' `buildmode`, which uses a profile's
'bootstrap_packages.$arch' file to install packages using pacstrap and compressing it to a bootstrap image.
The name of the output file is currently constructed from the `iso_name` value by appending `-bootstrap`.

Replace the uses of `airootfs_dir` with the more generic `pacstrap_dir`, as the location denotes where pacstrap is
being used.
Replace uses of `img_name` with `image_name` and removing it from the global scope, so that it can be overridden per
each buildmode.
Rename `_cleanup_airootfs_dir()` to `_cleanup_pacstrap_dir()`.
Make `_run_once()` more generic by prepending the state files with a string defined by `run_once_mode`.
Add `_validate_requirements_buildmode_all()`, `_validate_requirements_buildmode_bootstrap()` and
`_validate_requirements_buildmode_iso()` to validate the general requirements of the different buildmodes.
Add `_build_bootstrap_image()` to generate the bootstrap image using bsdtar.
Rename `_build_iso()` to `_build_iso_image()` to fit the naming of the respective bootstrap function.
Extend `_read_profile()` to include the reading of bootstrap image specific packages from a file.
Extend `_validate_options()` to include testing of the bootstrap packages and running of validation functions for all
buildmodes.
Change `_set_overrides()` to override the buildmodes if they are specified via the `-m` option flag.
Change `_make_version()` to be used generically in all buildmodes.
Change `_make_pkglist()` to be used generically in all buildmodes.
Rename `_build_profile()` to `_build_buildmode_iso()` and set local variables that are specific to the buildmode, such
as `image_name`, `pacstrap_dir`, `run_once_mode` , `buildmode_packages` and `buildmode_pkg_list`.
Add `_build_buildmode_bootstrap()` and set local variables that are specific to the buildmode, such as `image_name`,
`pacstrap_dir`, `run_once_mode` , `buildmode_packages` and `buildmode_pkg_list`.
Add the `-m` option flag to the list of flags.

Closes #127
2021-05-09 15:50:08 +02:00
David Runge
42cdf8674a
Set more generic output for signatures
archiso/mkarchiso:
Change the help output to reflect that the `-g` option is generically signing a rootfs (which may be e.g. squashfs or
erofs).
Change the output of `_mksignature()` to be more generic, as it signs any type of understood rootfs image (which may be
e.g. squashfs or erofs).
2021-05-01 17:16:56 +02:00
David Runge
cc735dbbfc
Force PGP signature file extension
archiso/mkarchiso:
Force the file extension in use for the PGP signatures of the rootfs to always be .sig.
When gnupg's 'armor' configuration option is used, the output otherwise defaults to using .asc.
As the verification hook in mkinitcpio-archiso expects the .sig file extension, verifying the rootfs will fail in that
scenario.
2021-05-01 16:56:53 +02:00
nl6720
0406f9ca02
mkarchiso: create reproducible gzip archives
Use the gzip option -n/--no-name to prevent saving the original file name and timestamp.

Fixes #104.
2021-04-30 20:50:07 +03:00
nl6720
a771297e12
mkarchiso: make sure to remove potentially preexisting files from $airootfs_dir before creating them with output redirection
mkarchiso creates "${airootfs_dir}/etc/machine-id" by using output redirection. If this file is an existing symlink, then the printf output would be written to the symlink target. It can be a big issue in case the symlink resolves to a path outside ${airootfs_dir}.

Fixes #121.
2021-04-30 20:37:18 +03:00
nl6720
98c7b67697
mkarchiso: append IMAGE_ID and IMAGE_VERSION to /etc/os-release
This provides the ISO version information in the os-release file.

* IMAGE_ID is set to the value of $iso_name.
* IMAGE_VERSION is set to the value of $iso_version.

Implements #116.
2021-04-30 20:37:17 +03:00
David Runge
8bf95d37d3
Ignore SC3060 in initcpio hook
archiso/initcpio/hooks/archiso_pxe_common:
Disable shellcheck's SC3060, as ash is able to do bash-like string replacements.
2021-04-30 17:51:30 +02:00
Christian Hesse
1a97109639 mkarchiso: also add iso name in grub environment block 2021-04-07 16:40:18 +00:00
nl6720
09b6127fe8
mkarchiso: use -isohybrid-gpt-basdat instead of -appended_part_as_gpt for ISOs that will support BIOS booting
Some hardware, like Lenovo Thinkpad T420, will not BIOS boot if the disk has a valid GPT.
See https://bbs.archlinux.org/viewtopic.php?id=264096 .

Instead of a valid GPT, change to a valid MBR and invalid GPT similar to what was used before 729d16b48c. That layout, despite having crazy partition tables, boots everywhere.
The difference is that -append_partition is still kept and specified before -isohybrid-gpt-basdat. Thus the appended partition will be listed as EFI system partition in MBR and as Microsoft basic partition in the invalid GPT.

Fixes #102.
2021-04-07 10:29:35 +03:00
nl6720
8cbc548359
mkarchiso: do not set default mksquashfs options
Remove hardcoded '-comp xz', it prevents using mksquashfs defaults.

Fixes #112.
2021-03-26 08:37:38 +02:00
Michael Gilchrist
6bb12552e4 Recursively change file permissions for folders listed in profiledef.sh
- if a folder listed in the associative array ends with a "/",
  recursively apply chmod and chown.
2021-03-21 14:00:13 +00:00
nl6720
bc67933af1
Support EROFS
EROFS, like Squashfs, is a read-only file system. It can be used to store airootfs in an image file.
Its advantage is the support for POSIX ACLs. EROFS downside is that currently it only supports LZ4 compression (LZMA support is not yet fully implemented).

A difference from Squashfs is that, EROFS stores change time (ctime) not modification time (mtime). The reverse is true for Squashfs.

Implements https://gitlab.archlinux.org/archlinux/archiso/-/issues/59
2021-03-09 16:25:45 +02:00
nl6720
711ab4cd1e
archiso/initcpio/hooks/archiso: remove redundant /sfs/ from airootfs mount point
Remove /run/archiso/bootmnt directory if nothing is mounted there. An empty directory is just confusing.
2021-03-09 16:25:45 +02:00
nl6720
0f20a11bb7
Support setting more variables in profiledef.sh and rework the way overrides are applied
- Apply overrides before validating the options.
- Parse all paths with realpath. Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/84 .
2021-01-29 00:01:24 +02:00
Christian Hesse
a2c8dd3173 archiso_pxe_common: remove resolv.conf before copy
Booting via PXE we want to keep our DNS configuration. So remove
/etc/resolv.conf in new root before copying the current file.

Without this systemd-resolved fallback nameservers are used and we see an
error message when the root ships a symbolic link to systemd-resolved's
stub-resolv.conf:

cp: not writing through dangling symlink '/new_root/etc/resolv.conf'
2021-01-26 21:59:01 +00:00
Christian Hesse
b30d1cad9b mkarchiso: add version information
To date the iso version was used for iso volume information and iso file name.
In my custom builds I do use it a lot more:

* Inside the root fs: The system knows about its own version. I use this to:
  -> report the version to a server (poor man's inventory)
  -> let the system update itself

* On the iso fs: The files are served via rsync, running systems transfer
  version file first to check for available update.

* A grub environment file on the iso fs: Booting the iso from grub allows
  to create cow directory per version:

    loopback loop archlinux.iso
    load_env -f (loop)/arch/grubenv
    linux (loop)/arch/boot/x86_64/vmlinuz-linux ... \
        cow_directory=archlinux/${VERSION} ...

So let's just create these files.
2021-01-26 10:00:38 +01:00