archiso/mkarchiso:
Make sure to always compare absolute paths in `_make_custom_airootfs()` (as `realpath` is used).
Remove `echo` calls that prevent the setting of actual file ownerships and modes.
configs/releng/profiledef.sh:
Set file mode of /root/.automated_script.sh to 755.
Fixes#82
profiledef.sh can now contain an associative array called file_permissions which can be used to set custom ownership and mode of custom airootfs files. The array's keys contain the path and the value is a colon separated list of owner UID, owner GID and access mode.
For example:
file_permissions=(
["/etc/shadow"]="0:0:400"
)
This means that mkarchiso now copies airootfs files (and directores) without permissions and anything that should be owned by a user other than root and/or if the mode should be something other than 644 for files and 755 for directories must to be listed in ${file_permission[@]} in profiledef.sh.
Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/61 .
archiso/mkarchiso:
Guard the call to `_mksignature()` in `_prepare_airootfs_image()` by an if statement.
Using the `&&` logic leads to `_prepare_airootfs_image()` evaluating to false if `$gpg_key` is not set.
Add `_msg_info()` calls to `_set_override()` which prevent the function from evaluating to false if no override is
being done. Additionally this is great for debugging purposes.
Add `_msg_info()` calls to `_read_profile()` (which is great for debugging purposes).
Fixes#81
- Remove remnants of the now removed legacy commands.
- Improve readability by getting rid of some "if" statements when performing string comparisons.
- Rename functions to make their purpose more clear.
- Move some conditions from functions to their invocations.
configs/{baseline,releng}/build.sh:
Remove `build.sh` scripts. They were deprecated with v47.
archiso/mkarchiso:
Remove all `build.sh` related functionality (i.e. `command_pkglist()`, `command_iso()`, `command_prepare()`,
`command_install()`, `command_init()`, `command_run()`).
Rename `command_build_profile()` to `_build_profile()` to be more in line with the style of the other function naming.
Change `_show_config()` to only print info about the profile and make no more use of parameters.
Remove all help output related to legacy `build.sh` commands.
Fixes#51
archiso/mkarchiso:
Change `_pacman()` to use the *modified* pacman.conf from the work_dir, instead of using the *unmodified* pacman.conf from
the profile.
Change `_make_pacman_conf()` to compare the system's and the profile's CacheDir setting and use the profile's CacheDir
setting only if it's not the default and not the same as the system's.
Always set the HookDir to the airootfs' override directory, so that no hooks from the host system are being run.
Remove DBPath, LogFile and RootDir settings from the work_dir pacman.conf as they are otherwise referring to the host
system, **even if** pacman is being called with the `-r` flag.
Fix a typo in _make_custom_airootfs().
README.profile.rst:
Add information about the pacman.conf in a profile and how configuration options behave, when used by mkarchiso.
Fixes#73Fixes#74
Support bios.syslinux.eltorito boot mode without bios.syslinux.mbr.
bios.syslinux.mbr does not work without bios.syslinux.eltorito because -isohybrid-mbr requires the El Torito boot image.
Support uefi-x64.systemd-boot.esp boot mode without uefi-x64.systemd-boot.eltorito and vice versa.
If uefi-x64.systemd-boot.eltorito is used without uefi-x64.systemd-boot.esp, the El Torito boot image will be placed in the ISO 9660 file system as before.
Note that an ISO created with only uefi-x64.systemd-boot.eltorito will still be bootable as a "hard disk" on OVMF. OVMF will boot the El Torito image.
This change has the following effect on the partition tables:
- *.eltorito options add El Torito boot catalog entries. MBR and GPT are not affected.
- uefi-x64.systemd-boot.esp creates a protective MBR partition table and a GPT table that includes a Linux filesystem data partition for the ISO 9660 volume, an EFI system partition and a Microsoft basic data partition that maps the 300 KiB padding added by xorriso.
- bios.syslinux.mbr (without uefi-x64.systemd-boot.esp): adds a MBR partition table and maps the ISO 9660 volume as a partition of type 0x83. No GPT is produced.
- bios.syslinux.mbr (with uefi-x64.systemd-boot.esp): adds an second partition in the MBR (after 0xEE) starting from sector 0 to sector 1 with type 0 and marks it as bootable. This violates the GPT specification, but allows some systems to succesfully boot in BIOS mode from GPT.
Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/49
archiso/mkarchiso:
The bash arithmethics in _make_boot_uefi-x64.systemd-boot.esp() introduced rounding issues, that can lead to
insufficient FAT image size for the files.
Conversion functions for awk now replace the bash arithmetics and additionally a ceil() function rounds the calculated
size up to the next full MiB.
Add an info message about the size of the created FAT image.
Fixes#70
archiso/mkarchiso:
Change the way _show_config() displays information about the build environment, as displaying a profile directory as a
legacy command to mkarchiso is confusing.
The function now prints a deprecation message if '$command_name' is not a directory (i.e. not a profile).
Remove 'command_' prefix from help output for legacy commands (the prefix is only in use for internal functions).
Relates to !69Fixes#60
Use du to count the file size of the kernel(s), initramfs images and boot loader (and its configuration).
This allows to compress initramfs with something other than xz, or have more than one kernel installed.
Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/62 .
All kernels and initramfs images from airootfs are copied to ${install_dir}/boot/ on both ISO 9660 and FAT.
This allows providing multiple kernels. The kernel package needs to be added to packages.x86_64 and the boot loader configuration files created/adjusted appropriately.
Support all possible microcode initrd file names.
Copy /etc/skel/ only for users with UID in range 1000–60000.
Correct user home directory permission after copying files.
Fixes all /etc/skel issues from https://bugs.archlinux.org/task/67729 .
- mtools supports SOURCE_DATE_EPOCH.
- The image file is operated on directly instead of mounting the file system. This is a prerequisite to limit the commands that run with root privileges. Related to https://gitlab.archlinux.org/archlinux/archiso/-/issues/40 .
- Add a reminder comment to not get rid of the dosfstools dependency, since a mformat made FAT image can fail to boot on some systems.
**archiso/mkarchiso**:
In d90184a the unbound variable `override_install_dir` was introduced by accident.
It is required to be set to empty string to provide override functionality for `install_dir`.
Fixes#57
**archiso/mkarchiso**:
Change all override option parameters (i.e. `-A`, `-C`, `-D`, `-L`, `-P` and `-g`) to not directly override the global
variable they are tied to, but instead using an `override_` prefixed variable.
Add `_set_overrides()` to use `override_` prefixed variables (if set) to override those without a prefix.
Remove `-B` (a profile directory) from the list of parameters. The profile directory is now provided as separate
non-option parameter.
Add a call to `_read_profile()`, `_set_overrides()` and `command_build_profile()` to the fallthrough option of the
switch-case checking `command_name` - a non-option parameter to mkarchiso. This effectively provides the possibility to
set the profile directory using a non-option parameter, while still maintaining compatibility to legacy named arguments
used in the configs' `build.sh` scripts.
Extend the warning in regards to legacy `build.sh` based commands to mkarchiso by providing an EOL with archiso v49.
Change the help output to reflect the changes and further elaborate on the legacy commands used by `build.sh` scripts.
Change help output to be ordered alphabetically.
Add help output for `-r` and `-g` options.
Call `_set_overrides()` for legacy commands that accept one or more of the overriden options (i.e. `command_init`,
`command_install`, `command_prepare` and `command_iso`).
Various style fixes.
**configs/{baseline,releng}/build.sh**:
Change call to mkarchiso to use the profile's directory as a named argument instead of an option-argument.
**README.rst**:
Fix documentation on how to call mkarchiso with a profile directory.
Fix wording and ordering of option arguments for run_archiso documentation.
Fixes#52
Fixes to issues introduced in https://gitlab.archlinux.org/archlinux/archiso/-/merge_requests/59 :
- _make_boot_on_fat(): copy initramfs from "${airootfs_dir}/boot/" not "${isofs_dir}/". Otherwise UEFI-only ISOs cannot be built.
Some general fixes:
- Replace mkdir with install. Unlike mkdir, install does not complain when the target exists.
- Reduce excess newlines produced by messages.
- Ensure FAT image gets unmounted in case the script is interrupted.
- Create the ext4 image with mkfs.ext4 instead of truncate.
- Do not rely on user and group names for chown commands. Use numeric UID and GID instead.
- Minimize the times stderr is redirected to /dev/null.
- Add missing '?' to getopts.
- Standardize function definitions by removing spaces between the function name and () .
Boot mode names are:
- bios_syslinux.mbr: SYSLINUX in MBR
- bios.syslinux.eltorito: SYSLINUX (ISOLINUX) via El Torito
- uefi-x64.systemd-boot.esp: systemd-boot on ESP in MBR
- uefi-x64.systemd-boot.eltorito: systemd-boot on ESP via El Torito
It is not yet possible to create an ISO with only El Torito or only MBR boot modes!
Quote all variables.
Terminate option processing using '--' for commands that support it.
Do not hardcode file descriptor.
Compare integers with arithmetic comparison instead of string comparison.
Replace echo with printf.
Use heredoc for usage text.
Don't print INFO messages when quiet is set.
Export SOURCE_DATE_EPOCH.
LICENSE:
Add GPL-3.0 license.
{{archiso,configs}/*,.editorconfig,.gitlab-ci.yml}:
Add SPDX license identifier.
Makefile:
Add SPDX license identifier.
Install the `run_archiso.sh` script as global executable `run_archiso`.
Use -D and -t flags to install to install files more generically (without a previous call to install the directory).
README.rst:
Add README outlining the project's scope, how to build images from the profiles and how to test.
AUTHORS.rst:
Add list of all direct contributors to the repository.
CONTRIBUTING.rst:
Add basic contribution guidelines, explaining the linter and the license in use.
Closes#7Closes#3
archiso/initcpio/install/*:
Setting bash shebang for all scripts and making them comform with shellcheck.
archiso/initcpio/{hooks,script}/*:
Setting ash shebang for all scripts and making them comform with shellcheck (for dash, as shellcheck has no ash specific
ruleset). Essentially the ash based scripts should be POSIX compliant as much as possible to have an easier time
writing, debugging and maintaining them.
Ensuring that variables are not treated as options and introducing variable quoting.
.gitlab-ci.yml:
Integrating shellcheck for initcpio scripts.
Closes#32
archiso/mkarchiso:
Calls to _pacman() need to be done with multiple parameters (e.g. array) instead of one string, as string splitting is
not done in that function anymore.
Turning _iso_efi_boot_args from string into an array to have an easier time of passing it to xorriso.
Calling xorriso within the if statements instead of providing -quiet via variable.
Fixing command_install() to provide packages separately to _pacman()
configs/releng/build.sh:
Replacing all newlines when retrieving the packages from packages.x86_64 with spaces so they will be properly provided
to "mkarchiso install".
archiso/mkarchiso:
Quoting all variables.
Changing pkg_list to be an array instead of a string for easier
handling. Using read to properly populate pkg_list from OPTARG with
stripped whitespaces.
Not exporting iso_label anymore as there seems to be no reason to do so.
Introducing line breaks.
.editorconfig:
Setting max_line_length to 120.
Adding a section for YAML files (e.g. .gitlab-ci.yml).
configs/releng/build.sh
Quting nearly all variables.
Introducing line breaks.
configs/baseline/build.sh:
Quoting all variables.
Introducing line breaks.
.gitlab-ci.yml:
Adding gitlab CI for shelleck linting of the config build scripts, mkarchiso and startup scripts in releng.
Closes#19
Using the sync operation with list option fails with --sysroot when
signed database files are around. Instead use the query operation, which
uses the local databases of installed pakages only.
The only downside is that we do no longer record the originating
repository.
Signed-off-by: Christian Hesse <mail@eworm.de>
The kernel has an open file handle after setting up the mapping. We can
remove it early to make sure it is gone on shutdown.
This helps to keep the cow_device clean for non-persistent systems where
cow_directory contains a version specific string.
Signed-off-by: Christian Hesse <mail@eworm.de>
If the ARCHISO_GNUPG_FD environment variable is set, its contents will be interpreted as an open file
descriptor and its contents will be used to create a keyring in the initramfs in /gpg.
Having files on btrfs subvolumes requires to give mount options. Add
boot params archisoflags= and cow_flags= for this purpose. Boot
parameters could look like this:
... archisodevice=/dev/sdaX archisoflags=subvolume=isos
cow_device=/dev/sdaX cow_flags=subvolume=persist ...
Signed-off-by: Christian Hesse <mail@eworm.de>
Currently, when booting loopmounted iso file, it is necessary to
specify not only img_dev and img_loop (which should be sufficient),
but also archisolabel or archisodevice. With this patch, archisodevice
is directly populated with the correct loop device, and it is not
necessary to specify the label when booting from loopmounted iso,
which makes for leaner and cleaner grub.cfg.
We received an IP address from DHCP server and configure it statically.
This is required if we continue to use network connectivity to access
the root device (for example via NBD or NFS).
The lease is not updated, though. This can cause trouble in networks
with low lease times. So let's flush the addresses if root filesystem
has been copied to RAM. A dhcp client in main system can handle the
network connectivity then.
Signed-off-by: Christian Hesse <mail@eworm.de>
Umount detaches the loop device automatically, but let's make it
explicit to be sure. Additionally losetup gives:
losetup: /dev/loop0: detach failed: No such device or address
This is kind of expected, let's silent the error message.
Signed-off-by: Christian Hesse <mail@eworm.de>
I see cases where a stale loop device stays around and fills up my
partition as image file is still in use and does not get unlinked.
Explicitly detach loop device on umount to fix that.
Signed-off-by: Christian Hesse <mail@eworm.de>
This is the first attemp to test overlayfs in archiso.
The current dm-snapshot mode is keep and is enabled by default,
while the new mode is enabled via "-s sfs" to mkarchiso.
No new boot parameters are added, since archiso hooks detects
if the .sfs file is for dm-snapshot (airootfs.img inside)
or for overlayfs.
Persistence is supported in overlayfs mode using the same options
(cowlabel or cowdevice), but warning while in dm-snapshot mode,
only one file is used (airootfs.cow), in overlayfs mode internal
files for workdir/ and upperdir/ are allocated, so you can not use
VFAT or NTFS.
To test this, you need to enable [testing] in pacman.conf from
releng profile and edit build.sh then add "-s sfs" in make_prepare()
Look at:
setarch ${arch} mkarchiso ${verbose} -w "${work_dir}" -D "${install_dir}" prepare
Replace with:
setarch ${arch} mkarchiso ${verbose} -w "${work_dir}" -s sfs -D "${install_dir}" prepare
The build requires just half of space that the build for dm-snapshot,
since there is no ext4 img ;)
Just to remember: there is no space gain in .sfs (just about 2M)
There is at least one thing during boot with machine-id service:
Dec 24 03:31:39 archiso systemd-machine-id-commit[183]: Failed to unmount transient /etc/machine-id file in our private namespace: Invalid argument
Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386@gmail.com>
Copying big amount of data results in bad performance as data is
written in chunks of 4kiB (8 * 512 bytes).
The default is not changed but can be overwritten.
Signed-off-by: Christian Hesse <mail@eworm.de>
Remove unused feature, just to save space
sizeof ext4 image (empty): 4.3M vs 403K (du airootfs.img)
sizeof used space (empty): 48M vs 20K (df)
Thanks Christian Hesse for initial tip.
Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386@gmail.com>
Currently, when booting via HTTP, the airootfs.md5 file is downloaded to
/run/archiso/bootmnt/arch/. However, the checksum file is later assumed
to be located at "/run/archiso/bootmnt/arch/${arch}". Fix the _curl_get
invocation and directly place the file in the right directory.
Fixes a regression introduced in commit b3e1d31 ([archiso] Rework
checksum function, 2014-06-28).
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Among other things, 36459f3 ([archiso] Drop aitab support, 2014-06-28)
removed the possibility to manually set a specific architecture by using
kernel parameters. This, however, is useful, e.g. when installing Arch
Linux on a device that reports itself as i586 but works fine with the
i686 flavor.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
