Commit Graph

364 Commits

Author SHA1 Message Date
David Runge
42cdf8674a
Set more generic output for signatures
archiso/mkarchiso:
Change the help output to reflect that the `-g` option is generically signing a rootfs (which may be e.g. squashfs or
erofs).
Change the output of `_mksignature()` to be more generic, as it signs any type of understood rootfs image (which may be
e.g. squashfs or erofs).
2021-05-01 17:16:56 +02:00
David Runge
cc735dbbfc
Force PGP signature file extension
archiso/mkarchiso:
Force the file extension in use for the PGP signatures of the rootfs to always be .sig.
When gnupg's 'armor' configuration option is used, the output otherwise defaults to using .asc.
As the verification hook in mkinitcpio-archiso expects the .sig file extension, verifying the rootfs will fail in that
scenario.
2021-05-01 16:56:53 +02:00
nl6720
0406f9ca02
mkarchiso: create reproducible gzip archives
Use the gzip option -n/--no-name to prevent saving the original file name and timestamp.

Fixes #104.
2021-04-30 20:50:07 +03:00
nl6720
a771297e12
mkarchiso: make sure to remove potentially preexisting files from $airootfs_dir before creating them with output redirection
mkarchiso creates "${airootfs_dir}/etc/machine-id" by using output redirection. If this file is an existing symlink, then the printf output would be written to the symlink target. It can be a big issue in case the symlink resolves to a path outside ${airootfs_dir}.

Fixes #121.
2021-04-30 20:37:18 +03:00
nl6720
98c7b67697
mkarchiso: append IMAGE_ID and IMAGE_VERSION to /etc/os-release
This provides the ISO version information in the os-release file.

* IMAGE_ID is set to the value of $iso_name.
* IMAGE_VERSION is set to the value of $iso_version.

Implements #116.
2021-04-30 20:37:17 +03:00
David Runge
8bf95d37d3
Ignore SC3060 in initcpio hook
archiso/initcpio/hooks/archiso_pxe_common:
Disable shellcheck's SC3060, as ash is able to do bash-like string replacements.
2021-04-30 17:51:30 +02:00
Christian Hesse
1a97109639 mkarchiso: also add iso name in grub environment block 2021-04-07 16:40:18 +00:00
nl6720
09b6127fe8
mkarchiso: use -isohybrid-gpt-basdat instead of -appended_part_as_gpt for ISOs that will support BIOS booting
Some hardware, like Lenovo Thinkpad T420, will not BIOS boot if the disk has a valid GPT.
See https://bbs.archlinux.org/viewtopic.php?id=264096 .

Instead of a valid GPT, change to a valid MBR and invalid GPT similar to what was used before 729d16b48c. That layout, despite having crazy partition tables, boots everywhere.
The difference is that -append_partition is still kept and specified before -isohybrid-gpt-basdat. Thus the appended partition will be listed as EFI system partition in MBR and as Microsoft basic partition in the invalid GPT.

Fixes #102.
2021-04-07 10:29:35 +03:00
nl6720
8cbc548359
mkarchiso: do not set default mksquashfs options
Remove hardcoded '-comp xz', it prevents using mksquashfs defaults.

Fixes #112.
2021-03-26 08:37:38 +02:00
Michael Gilchrist
6bb12552e4 Recursively change file permissions for folders listed in profiledef.sh
- if a folder listed in the associative array ends with a "/",
  recursively apply chmod and chown.
2021-03-21 14:00:13 +00:00
nl6720
bc67933af1
Support EROFS
EROFS, like Squashfs, is a read-only file system. It can be used to store airootfs in an image file.
Its advantage is the support for POSIX ACLs. EROFS downside is that currently it only supports LZ4 compression (LZMA support is not yet fully implemented).

A difference from Squashfs is that, EROFS stores change time (ctime) not modification time (mtime). The reverse is true for Squashfs.

Implements https://gitlab.archlinux.org/archlinux/archiso/-/issues/59
2021-03-09 16:25:45 +02:00
nl6720
711ab4cd1e
archiso/initcpio/hooks/archiso: remove redundant /sfs/ from airootfs mount point
Remove /run/archiso/bootmnt directory if nothing is mounted there. An empty directory is just confusing.
2021-03-09 16:25:45 +02:00
nl6720
0f20a11bb7
Support setting more variables in profiledef.sh and rework the way overrides are applied
- Apply overrides before validating the options.
- Parse all paths with realpath. Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/84 .
2021-01-29 00:01:24 +02:00
Christian Hesse
a2c8dd3173 archiso_pxe_common: remove resolv.conf before copy
Booting via PXE we want to keep our DNS configuration. So remove
/etc/resolv.conf in new root before copying the current file.

Without this systemd-resolved fallback nameservers are used and we see an
error message when the root ships a symbolic link to systemd-resolved's
stub-resolv.conf:

cp: not writing through dangling symlink '/new_root/etc/resolv.conf'
2021-01-26 21:59:01 +00:00
Christian Hesse
b30d1cad9b mkarchiso: add version information
To date the iso version was used for iso volume information and iso file name.
In my custom builds I do use it a lot more:

* Inside the root fs: The system knows about its own version. I use this to:
  -> report the version to a server (poor man's inventory)
  -> let the system update itself

* On the iso fs: The files are served via rsync, running systems transfer
  version file first to check for available update.

* A grub environment file on the iso fs: Booting the iso from grub allows
  to create cow directory per version:

    loopback loop archlinux.iso
    load_env -f (loop)/arch/grubenv
    linux (loop)/arch/boot/x86_64/vmlinuz-linux ... \
        cow_directory=archlinux/${VERSION} ...

So let's just create these files.
2021-01-26 10:00:38 +01:00
Christian Hesse
d153b48613 mkarchiso: fix typos 2021-01-23 21:04:17 +01:00
nl6720
64091a1802
Combine sed commands to reduce file writes 2021-01-07 14:33:28 +02:00
David Runge
c10004dfec
Fix issues with file ownerships/modes
archiso/mkarchiso:
Make sure to always compare absolute paths in `_make_custom_airootfs()` (as `realpath` is used).
Remove `echo` calls that prevent the setting of actual file ownerships and modes.

configs/releng/profiledef.sh:
Set file mode of /root/.automated_script.sh to 755.

Fixes #82
2020-11-30 21:48:08 +01:00
nl6720
863247d0a6
Keep all SYSLINUX files in /syslinux
This gets rid of the duplicate ldlinux.c32 and the useless isolinux.cfg which only points to syslinux.cfg.

Implements https://gitlab.archlinux.org/archlinux/archiso/-/issues/46 .
2020-11-30 16:00:15 +02:00
nl6720
183ae52792
Prevent path traversal outside of $airootfs_dir 2020-11-30 09:21:35 +02:00
nl6720
42d9e4f983
Allow specifying ownership and mode of custom airootfs files and directories
profiledef.sh can now contain an associative array called file_permissions which can be used to set custom ownership and mode of custom airootfs files. The array's keys contain the path and the value is a colon separated list of owner UID, owner GID and access mode.
For example:

    file_permissions=(
      ["/etc/shadow"]="0:0:400"
    )

This means that mkarchiso now copies airootfs files (and directores) without permissions and anything that should be owned by a user other than root and/or if the mode should be something other than 644 for files and 755 for directories must to be listed in ${file_permission[@]} in profiledef.sh.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/61 .
2020-11-30 08:46:24 +02:00
fdupoux
2c99df5c9b Reset network interfaces at the end of the PXE boot to allow DHCP to run 2020-11-18 21:16:27 +00:00
David Runge
3160db0e9e
Fix evaluation bugs in mkarchiso
archiso/mkarchiso:
Guard the call to `_mksignature()` in `_prepare_airootfs_image()` by an if statement.
Using the `&&` logic leads to `_prepare_airootfs_image()` evaluating to false if `$gpg_key` is not set.

Add `_msg_info()` calls to `_set_override()` which prevent the function from evaluating to false if no override is
being done. Additionally this is great for debugging purposes.

Add `_msg_info()` calls to `_read_profile()` (which is great for debugging purposes).

Fixes #81
2020-11-18 19:24:02 +01:00
nl6720
96ac5e2454
mkarchiso: add xorrisofs options from boot mode specific functions instead of hardcoding them in _build_iso 2020-11-14 15:36:45 +02:00
nl6720
4dfb473748
mkarchiso: validate profile right after reading it
Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/76 .
2020-11-14 15:29:51 +02:00
nl6720
57d510fe7f
mkarchiso: general cleanup and simplification
- Remove remnants of the now removed legacy commands.
- Improve readability by getting rid of some "if" statements when performing string comparisons.
- Rename functions to make their purpose more clear.
- Move some conditions from functions to their invocations.
2020-11-14 14:49:33 +02:00
fdupoux
9f16862acd Configure the image type and image creation options using profiles (#54) 2020-10-30 21:33:08 +00:00
David Runge
1bd944ee7c
Remove build.sh support from profiles and mkarchiso
configs/{baseline,releng}/build.sh:
Remove `build.sh` scripts. They were deprecated with v47.

archiso/mkarchiso:
Remove all `build.sh` related functionality (i.e. `command_pkglist()`, `command_iso()`, `command_prepare()`,
`command_install()`, `command_init()`, `command_run()`).

Rename `command_build_profile()` to `_build_profile()` to be more in line with the style of the other function naming.

Change `_show_config()` to only print info about the profile and make no more use of parameters.

Remove all help output related to legacy `build.sh` commands.

Fixes #51
2020-10-29 12:32:48 +01:00
David Runge
f3af569205
Set CacheDir and HookDir for profile more sanely
archiso/mkarchiso:
Change `_pacman()` to use the *modified* pacman.conf from the work_dir, instead of using the *unmodified* pacman.conf from
the profile.

Change `_make_pacman_conf()` to compare the system's and the profile's CacheDir setting and use the profile's CacheDir
setting only if it's not the default and not the same as the system's.

Always set the HookDir to the airootfs' override directory, so that no hooks from the host system are being run.

Remove DBPath, LogFile and RootDir settings from the work_dir pacman.conf as they are otherwise referring to the host
system, **even if** pacman is being called with the `-r` flag.

Fix a typo in _make_custom_airootfs().

README.profile.rst:
Add information about the pacman.conf in a profile and how configuration options behave, when used by mkarchiso.

Fixes #73
Fixes #74
2020-10-24 17:00:59 +02:00
nl6720
729d16b48c
Move FAT image to a separate partition outside the ISO 9660 file system
Support bios.syslinux.eltorito boot mode without bios.syslinux.mbr.
bios.syslinux.mbr does not work without bios.syslinux.eltorito because -isohybrid-mbr requires the El Torito boot image.

Support uefi-x64.systemd-boot.esp boot mode without uefi-x64.systemd-boot.eltorito and vice versa.
If uefi-x64.systemd-boot.eltorito is used without uefi-x64.systemd-boot.esp, the El Torito boot image will be placed in the ISO 9660 file system as before.
Note that an ISO created with only uefi-x64.systemd-boot.eltorito will still be bootable as a "hard disk" on OVMF. OVMF will boot the El Torito image.

This change has the following effect on the partition tables:

- *.eltorito options add El Torito boot catalog entries. MBR and GPT are not affected.
- uefi-x64.systemd-boot.esp creates a protective MBR partition table and a GPT table that includes a Linux filesystem data partition for the ISO 9660 volume, an EFI system partition and a Microsoft basic data partition that maps the 300 KiB padding added by xorriso.
- bios.syslinux.mbr (without uefi-x64.systemd-boot.esp): adds a MBR partition table and maps the ISO 9660 volume as a partition of type 0x83. No GPT is produced.
- bios.syslinux.mbr (with uefi-x64.systemd-boot.esp): adds an second partition in the MBR (after 0xEE) starting from sector 0 to sector 1 with type 0 and marks it as bootable. This violates the GPT specification, but allows some systems to succesfully boot in BIOS mode from GPT.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/49
2020-10-24 13:15:37 +03:00
nl6720
bfab360529
Ensure that _make_boot_uefi-x64.systemd-boot.esp can be re-run if it fails
Remove efiboot.img if it exists so that mkfs.fat does not fail.
2020-10-07 11:42:08 +03:00
nl6720
f45fc29fe0
Save SOURCE_DATE_EPOCH to a file and read it from the file when resuming a build
Print the build date.
Don't re-run mksquashfs if it succeeded, but the failure was in gpg signing. For example, if the gpg pinentry timed out.
2020-10-07 11:42:08 +03:00
nl6720
455e2b94c0
Check command line parameter count instead of option count, and do it after evaluating option arguments
Fixes b6241cb1d0 .
2020-10-07 11:33:56 +03:00
nl6720
b6241cb1d0
Don't require root privileges just to show a "No command specified" error
Check if a profile or command is specified before looking at EUID.
2020-10-04 11:31:14 +03:00
David Runge
55cfb8ba02
Replace bash arithmetic with awk functions
archiso/mkarchiso:
The bash arithmethics in _make_boot_uefi-x64.systemd-boot.esp() introduced rounding issues, that can lead to
insufficient FAT image size for the files.
Conversion functions for awk now replace the bash arithmetics and additionally a ceil() function rounds the calculated
size up to the next full MiB.
Add an info message about the size of the created FAT image.

Fixes #70
2020-10-03 22:55:31 +02:00
David Runge
2e1ddec0a7
Fix info message for legacy command
archiso/mkarchiso:
Change the way _show_config() displays information about the build environment, as displaying a profile directory as a
legacy command to mkarchiso is confusing.
The function now prints a deprecation message if '$command_name' is not a directory (i.e. not a profile).
Remove 'command_' prefix from help output for legacy commands (the prefix is only in use for internal functions).

Relates to !69
Fixes #60
2020-09-29 17:29:15 +02:00
nl6720
f34c95797d
Calculate required efiboot.img size instead of hardcoding it
Use du to count the file size of the kernel(s), initramfs images and boot loader (and its configuration).
This allows to compress initramfs with something other than xz, or have more than one kernel installed.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/62 .
2020-09-27 20:54:11 +03:00
nl6720
8276616f6f
archiso/mkarchiso: do not hardcode the kernel and initramfs file names
All kernels and initramfs images from airootfs are copied to ${install_dir}/boot/ on both ISO 9660 and FAT.
This allows providing multiple kernels. The kernel package needs to be added to packages.x86_64 and the boot loader configuration files created/adjusted appropriately.

Support all possible microcode initrd file names.
2020-09-27 20:54:11 +03:00
nl6720
83e4cb9fb9
Use the same file paths in both ISO 9660 and FAT
This allows to use only one systemd-boot configuration file per kernel.
Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/44 .
2020-09-27 20:54:11 +03:00
nl6720
136152e672
Do not rename the initramfs image
This is a breaking change. archweb, archiso-manager and various documentation must be updated.

* https://github.com/archlinux/archweb/blob/master/templates/releng/archlinux.ipxe
* https://github.com/pierres/archiso-manager/blob/master/Makefile
2020-09-27 20:54:10 +03:00
nl6720
486b1910dd
Do not overwrite existing files when copying from /etc/skel/
Copy /etc/skel/ only for users with UID in range 1000–60000.
Correct user home directory permission after copying files.
Fixes all /etc/skel issues from https://bugs.archlinux.org/task/67729 .
2020-09-05 11:23:00 +03:00
nl6720
45a5d229b3
Create directories and copy files to the FAT image using mtools instead of mounting the file system
- mtools supports SOURCE_DATE_EPOCH.
- The image file is operated on directly instead of mounting the file system. This is a prerequisite to limit the commands that run with root privileges. Related to https://gitlab.archlinux.org/archlinux/archiso/-/issues/40 .
- Add a reminder comment to not get rid of the dosfstools dependency, since a mformat made FAT image can fail to boot on some systems.
2020-09-02 22:31:04 +03:00
David Runge
4b401931bd
Add missing declaration of override_install_dir
**archiso/mkarchiso**:

In d90184a the unbound variable `override_install_dir` was introduced by accident.

It is required to be set to empty string to provide override functionality for `install_dir`.

Fixes #57
2020-08-25 22:02:50 +02:00
David Runge
d90184a7f0
Pass profile directory as parameter to mkarchiso
**archiso/mkarchiso**:

Change all override option parameters (i.e. `-A`, `-C`, `-D`, `-L`, `-P` and `-g`) to not directly override the global
variable they are tied to, but instead using an `override_` prefixed variable.

Add `_set_overrides()` to use `override_` prefixed variables (if set) to override those without a prefix.

Remove `-B` (a profile directory) from the list of parameters. The profile directory is now provided as separate
non-option parameter.

Add a call to `_read_profile()`, `_set_overrides()` and `command_build_profile()` to the fallthrough option of the
switch-case checking `command_name` - a non-option parameter to mkarchiso. This effectively provides the possibility to
set the profile directory using a non-option parameter, while still maintaining compatibility to legacy named arguments
used in the configs' `build.sh` scripts.

Extend the warning in regards to legacy `build.sh` based commands to mkarchiso by providing an EOL with archiso v49.

Change the help output to reflect the changes and further elaborate on the legacy commands used by `build.sh` scripts.

Change help output to be ordered alphabetically.

Add help output for `-r` and `-g` options.

Call `_set_overrides()` for legacy commands that accept one or more of the overriden options (i.e. `command_init`,
`command_install`, `command_prepare` and `command_iso`).

Various style fixes.

**configs/{baseline,releng}/build.sh**:

Change call to mkarchiso to use the profile's directory as a named argument instead of an option-argument.

**README.rst**:

Fix documentation on how to call mkarchiso with a profile directory.

Fix wording and ordering of option arguments for run_archiso documentation.

Fixes #52
2020-08-25 19:06:43 +02:00
nl6720
e2c5b4527f
archiso/mkarchiso: reduce duplication and add more info messages
Fixes to issues introduced in https://gitlab.archlinux.org/archlinux/archiso/-/merge_requests/59 :
- _make_boot_on_fat(): copy initramfs from "${airootfs_dir}/boot/" not "${isofs_dir}/". Otherwise UEFI-only ISOs cannot be built.

Some general fixes:
- Replace mkdir with install. Unlike mkdir, install does not complain when the target exists.
- Reduce excess newlines produced by messages.
- Ensure FAT image gets unmounted in case the script is interrupted.
- Create the ext4 image with mkfs.ext4 instead of truncate.
- Do not rely on user and group names for chown commands. Use numeric UID and GID instead.
- Minimize the times stderr is redirected to /dev/null.
- Add missing '?' to getopts.
- Standardize function definitions by removing spaces between the function name and () .
2020-08-21 14:16:32 +03:00
nl6720
13b8c7d099
archiso/mkarchiso: correct the path of airootfs/etc/machine-id
Use "${airootfs_dir}".
Fixes a mistake introduced when rebasing https://gitlab.archlinux.org/archlinux/archiso/-/merge_requests/64 .
2020-08-18 21:59:18 +03:00
nl6720
41d3d7d489
Add Joliet file system to the ISO
Joliet ensures correct file names capitalization on operating systems that support Joliet but not support Rock Ridge.
Implements https://gitlab.archlinux.org/archlinux/archiso/-/issues/45 .
2020-08-18 21:41:37 +03:00
nl6720
e6455b885b
archiso/mkarchiso: create an empty /etc/machine-id
Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/42 .
2020-08-18 21:27:52 +03:00
nl6720
e9f209efbf
Deprecate build.sh scripts and old mkarchiso commands
Replace build.sh scripts with calls to mkarchiso -B "profiledir" build_profile.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/37 .
2020-08-17 21:33:20 +03:00
nl6720
31b1dfdbe4
archiso/mkarchiso: delete all files in /boot not just the kernel and initramfs
There is nothing useful in /boot for the live system.
2020-08-17 17:14:06 +03:00