Commit Graph

337 Commits

Author SHA1 Message Date
David Runge
1bd944ee7c
Remove build.sh support from profiles and mkarchiso
configs/{baseline,releng}/build.sh:
Remove `build.sh` scripts. They were deprecated with v47.

archiso/mkarchiso:
Remove all `build.sh` related functionality (i.e. `command_pkglist()`, `command_iso()`, `command_prepare()`,
`command_install()`, `command_init()`, `command_run()`).

Rename `command_build_profile()` to `_build_profile()` to be more in line with the style of the other function naming.

Change `_show_config()` to only print info about the profile and make no more use of parameters.

Remove all help output related to legacy `build.sh` commands.

Fixes #51
2020-10-29 12:32:48 +01:00
David Runge
f3af569205
Set CacheDir and HookDir for profile more sanely
archiso/mkarchiso:
Change `_pacman()` to use the *modified* pacman.conf from the work_dir, instead of using the *unmodified* pacman.conf from
the profile.

Change `_make_pacman_conf()` to compare the system's and the profile's CacheDir setting and use the profile's CacheDir
setting only if it's not the default and not the same as the system's.

Always set the HookDir to the airootfs' override directory, so that no hooks from the host system are being run.

Remove DBPath, LogFile and RootDir settings from the work_dir pacman.conf as they are otherwise referring to the host
system, **even if** pacman is being called with the `-r` flag.

Fix a typo in _make_custom_airootfs().

README.profile.rst:
Add information about the pacman.conf in a profile and how configuration options behave, when used by mkarchiso.

Fixes #73
Fixes #74
2020-10-24 17:00:59 +02:00
nl6720
729d16b48c
Move FAT image to a separate partition outside the ISO 9660 file system
Support bios.syslinux.eltorito boot mode without bios.syslinux.mbr.
bios.syslinux.mbr does not work without bios.syslinux.eltorito because -isohybrid-mbr requires the El Torito boot image.

Support uefi-x64.systemd-boot.esp boot mode without uefi-x64.systemd-boot.eltorito and vice versa.
If uefi-x64.systemd-boot.eltorito is used without uefi-x64.systemd-boot.esp, the El Torito boot image will be placed in the ISO 9660 file system as before.
Note that an ISO created with only uefi-x64.systemd-boot.eltorito will still be bootable as a "hard disk" on OVMF. OVMF will boot the El Torito image.

This change has the following effect on the partition tables:

- *.eltorito options add El Torito boot catalog entries. MBR and GPT are not affected.
- uefi-x64.systemd-boot.esp creates a protective MBR partition table and a GPT table that includes a Linux filesystem data partition for the ISO 9660 volume, an EFI system partition and a Microsoft basic data partition that maps the 300 KiB padding added by xorriso.
- bios.syslinux.mbr (without uefi-x64.systemd-boot.esp): adds a MBR partition table and maps the ISO 9660 volume as a partition of type 0x83. No GPT is produced.
- bios.syslinux.mbr (with uefi-x64.systemd-boot.esp): adds an second partition in the MBR (after 0xEE) starting from sector 0 to sector 1 with type 0 and marks it as bootable. This violates the GPT specification, but allows some systems to succesfully boot in BIOS mode from GPT.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/49
2020-10-24 13:15:37 +03:00
nl6720
bfab360529
Ensure that _make_boot_uefi-x64.systemd-boot.esp can be re-run if it fails
Remove efiboot.img if it exists so that mkfs.fat does not fail.
2020-10-07 11:42:08 +03:00
nl6720
f45fc29fe0
Save SOURCE_DATE_EPOCH to a file and read it from the file when resuming a build
Print the build date.
Don't re-run mksquashfs if it succeeded, but the failure was in gpg signing. For example, if the gpg pinentry timed out.
2020-10-07 11:42:08 +03:00
nl6720
455e2b94c0
Check command line parameter count instead of option count, and do it after evaluating option arguments
Fixes b6241cb1d0 .
2020-10-07 11:33:56 +03:00
nl6720
b6241cb1d0
Don't require root privileges just to show a "No command specified" error
Check if a profile or command is specified before looking at EUID.
2020-10-04 11:31:14 +03:00
David Runge
55cfb8ba02
Replace bash arithmetic with awk functions
archiso/mkarchiso:
The bash arithmethics in _make_boot_uefi-x64.systemd-boot.esp() introduced rounding issues, that can lead to
insufficient FAT image size for the files.
Conversion functions for awk now replace the bash arithmetics and additionally a ceil() function rounds the calculated
size up to the next full MiB.
Add an info message about the size of the created FAT image.

Fixes #70
2020-10-03 22:55:31 +02:00
David Runge
2e1ddec0a7
Fix info message for legacy command
archiso/mkarchiso:
Change the way _show_config() displays information about the build environment, as displaying a profile directory as a
legacy command to mkarchiso is confusing.
The function now prints a deprecation message if '$command_name' is not a directory (i.e. not a profile).
Remove 'command_' prefix from help output for legacy commands (the prefix is only in use for internal functions).

Relates to !69
Fixes #60
2020-09-29 17:29:15 +02:00
nl6720
f34c95797d
Calculate required efiboot.img size instead of hardcoding it
Use du to count the file size of the kernel(s), initramfs images and boot loader (and its configuration).
This allows to compress initramfs with something other than xz, or have more than one kernel installed.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/62 .
2020-09-27 20:54:11 +03:00
nl6720
8276616f6f
archiso/mkarchiso: do not hardcode the kernel and initramfs file names
All kernels and initramfs images from airootfs are copied to ${install_dir}/boot/ on both ISO 9660 and FAT.
This allows providing multiple kernels. The kernel package needs to be added to packages.x86_64 and the boot loader configuration files created/adjusted appropriately.

Support all possible microcode initrd file names.
2020-09-27 20:54:11 +03:00
nl6720
83e4cb9fb9
Use the same file paths in both ISO 9660 and FAT
This allows to use only one systemd-boot configuration file per kernel.
Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/44 .
2020-09-27 20:54:11 +03:00
nl6720
136152e672
Do not rename the initramfs image
This is a breaking change. archweb, archiso-manager and various documentation must be updated.

* https://github.com/archlinux/archweb/blob/master/templates/releng/archlinux.ipxe
* https://github.com/pierres/archiso-manager/blob/master/Makefile
2020-09-27 20:54:10 +03:00
nl6720
486b1910dd
Do not overwrite existing files when copying from /etc/skel/
Copy /etc/skel/ only for users with UID in range 1000–60000.
Correct user home directory permission after copying files.
Fixes all /etc/skel issues from https://bugs.archlinux.org/task/67729 .
2020-09-05 11:23:00 +03:00
nl6720
45a5d229b3
Create directories and copy files to the FAT image using mtools instead of mounting the file system
- mtools supports SOURCE_DATE_EPOCH.
- The image file is operated on directly instead of mounting the file system. This is a prerequisite to limit the commands that run with root privileges. Related to https://gitlab.archlinux.org/archlinux/archiso/-/issues/40 .
- Add a reminder comment to not get rid of the dosfstools dependency, since a mformat made FAT image can fail to boot on some systems.
2020-09-02 22:31:04 +03:00
David Runge
4b401931bd
Add missing declaration of override_install_dir
**archiso/mkarchiso**:

In d90184a the unbound variable `override_install_dir` was introduced by accident.

It is required to be set to empty string to provide override functionality for `install_dir`.

Fixes #57
2020-08-25 22:02:50 +02:00
David Runge
d90184a7f0
Pass profile directory as parameter to mkarchiso
**archiso/mkarchiso**:

Change all override option parameters (i.e. `-A`, `-C`, `-D`, `-L`, `-P` and `-g`) to not directly override the global
variable they are tied to, but instead using an `override_` prefixed variable.

Add `_set_overrides()` to use `override_` prefixed variables (if set) to override those without a prefix.

Remove `-B` (a profile directory) from the list of parameters. The profile directory is now provided as separate
non-option parameter.

Add a call to `_read_profile()`, `_set_overrides()` and `command_build_profile()` to the fallthrough option of the
switch-case checking `command_name` - a non-option parameter to mkarchiso. This effectively provides the possibility to
set the profile directory using a non-option parameter, while still maintaining compatibility to legacy named arguments
used in the configs' `build.sh` scripts.

Extend the warning in regards to legacy `build.sh` based commands to mkarchiso by providing an EOL with archiso v49.

Change the help output to reflect the changes and further elaborate on the legacy commands used by `build.sh` scripts.

Change help output to be ordered alphabetically.

Add help output for `-r` and `-g` options.

Call `_set_overrides()` for legacy commands that accept one or more of the overriden options (i.e. `command_init`,
`command_install`, `command_prepare` and `command_iso`).

Various style fixes.

**configs/{baseline,releng}/build.sh**:

Change call to mkarchiso to use the profile's directory as a named argument instead of an option-argument.

**README.rst**:

Fix documentation on how to call mkarchiso with a profile directory.

Fix wording and ordering of option arguments for run_archiso documentation.

Fixes #52
2020-08-25 19:06:43 +02:00
nl6720
e2c5b4527f
archiso/mkarchiso: reduce duplication and add more info messages
Fixes to issues introduced in https://gitlab.archlinux.org/archlinux/archiso/-/merge_requests/59 :
- _make_boot_on_fat(): copy initramfs from "${airootfs_dir}/boot/" not "${isofs_dir}/". Otherwise UEFI-only ISOs cannot be built.

Some general fixes:
- Replace mkdir with install. Unlike mkdir, install does not complain when the target exists.
- Reduce excess newlines produced by messages.
- Ensure FAT image gets unmounted in case the script is interrupted.
- Create the ext4 image with mkfs.ext4 instead of truncate.
- Do not rely on user and group names for chown commands. Use numeric UID and GID instead.
- Minimize the times stderr is redirected to /dev/null.
- Add missing '?' to getopts.
- Standardize function definitions by removing spaces between the function name and () .
2020-08-21 14:16:32 +03:00
nl6720
13b8c7d099
archiso/mkarchiso: correct the path of airootfs/etc/machine-id
Use "${airootfs_dir}".
Fixes a mistake introduced when rebasing https://gitlab.archlinux.org/archlinux/archiso/-/merge_requests/64 .
2020-08-18 21:59:18 +03:00
nl6720
41d3d7d489
Add Joliet file system to the ISO
Joliet ensures correct file names capitalization on operating systems that support Joliet but not support Rock Ridge.
Implements https://gitlab.archlinux.org/archlinux/archiso/-/issues/45 .
2020-08-18 21:41:37 +03:00
nl6720
e6455b885b
archiso/mkarchiso: create an empty /etc/machine-id
Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/42 .
2020-08-18 21:27:52 +03:00
nl6720
e9f209efbf
Deprecate build.sh scripts and old mkarchiso commands
Replace build.sh scripts with calls to mkarchiso -B "profiledir" build_profile.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/37 .
2020-08-17 21:33:20 +03:00
nl6720
31b1dfdbe4
archiso/mkarchiso: delete all files in /boot not just the kernel and initramfs
There is nothing useful in /boot for the live system.
2020-08-17 17:14:06 +03:00
nl6720
7c2247f615
archiso/mkarchiso: allow choosing boot modes from profiledef.sh
Boot mode names are:

- bios_syslinux.mbr: SYSLINUX in MBR
- bios.syslinux.eltorito: SYSLINUX (ISOLINUX) via El Torito
- uefi-x64.systemd-boot.esp: systemd-boot on ESP in MBR
- uefi-x64.systemd-boot.eltorito: systemd-boot on ESP via El Torito

It is not yet possible to create an ISO with only El Torito or only MBR boot modes!
2020-08-17 17:14:06 +03:00
nl6720
41b9b89746
archiso/mkarchiso: copy make_* functions from configs/releng/build.sh
Adapt _make_* functions to mkarchiso.

Related to https://gitlab.archlinux.org/archlinux/archiso/-/issues/37 .
2020-08-17 17:13:37 +03:00
nl6720
1cc7375c25
archiso/mkarchiso: start preparing a build_profile command
Nothing is implemented yet!

configs/releng/profiledef.sh:
A test profile.
2020-08-17 14:22:36 +03:00
nl6720
0387b253c8
archiso/mkarchiso: general bash improvements
Quote all variables.
Terminate option processing using '--' for commands that support it.
Do not hardcode file descriptor.
Compare integers with arithmetic comparison instead of string comparison.
Replace echo with printf.
Use heredoc for usage text.
Don't print INFO messages when quiet is set.
Export SOURCE_DATE_EPOCH.
2020-08-17 14:22:24 +03:00
nl6720
8e82bbbe4a
archiso/initcpio/hooks/archiso: remove option terminator from the blockdev command
Apparently blockdev does not support it.
In an ISO made using '-s img' (Squashfs with dm-snapshot), it results in:

    blockdev: Unknown command: --
2020-08-11 06:58:06 +03:00
nl6720
6312ccc9bc
Ensure all files in the ISO's Rock Ridge file system are owned by root
archiso/mkarchiso:
Add the -rational-rock option to xorriso.
This is a preparatory step for creating ISOs as a regular user. See https://gitlab.archlinux.org/archlinux/archiso/-/issues/40 .
2020-07-30 22:01:27 +03:00
nl6720
945f3834c2
Show mksquashfs progress bar when mkarchiso is run in verbose mode 2020-07-30 21:57:27 +03:00
nl6720
b8dc4484e8
archiso/initcpio/install/archiso: fix GnuPG home directory permissions
Gets rid of a gpg warning:
gpg: WARNING: unsafe permissions on homedir '/tmp/mkinitcpio.*/root/gpg'
2020-07-29 18:05:16 +03:00
David Runge
e264b44682
Add license and basic documentation
LICENSE:
Add GPL-3.0 license.

{{archiso,configs}/*,.editorconfig,.gitlab-ci.yml}:
Add SPDX license identifier.

Makefile:
Add SPDX license identifier.
Install the `run_archiso.sh` script as global executable `run_archiso`.
Use -D and -t flags to install to install files more generically (without a previous call to install the directory).

README.rst:
Add README outlining the project's scope, how to build images from the profiles and how to test.

AUTHORS.rst:
Add list of all direct contributors to the repository.

CONTRIBUTING.rst:
Add basic contribution guidelines, explaining the linter and the license in use.

Closes #7
Closes #3
2020-07-29 14:27:48 +02:00
nl6720
b40fb2ae53
Separate custom comments from shellcheck comments
Having a shellcheck directive and custom comments in the same line can trigger SC1107 on old versions of shellcheck.
2020-07-17 10:42:14 +03:00
David Runge
e2032db4e7
Adding linting for initcpio scripts
archiso/initcpio/install/*:
Setting bash shebang for all scripts and making them comform with shellcheck.

archiso/initcpio/{hooks,script}/*:
Setting ash shebang for all scripts and making them comform with shellcheck (for dash, as shellcheck has no ash specific
ruleset). Essentially the ash based scripts should be POSIX compliant as much as possible to have an easier time
writing, debugging and maintaining them.
Ensuring that variables are not treated as options and introducing variable quoting.

.gitlab-ci.yml:
Integrating shellcheck for initcpio scripts.

Closes #32
2020-07-11 20:58:01 +02:00
nl6720
24a68d5879
Add amdgpu and mgag200 to the archiso_kms mkinitcpio hook
See https://wiki.archlinux.org/index.php/Kernel_mode_setting#Early_KMS_start .
2020-07-01 12:05:52 +03:00
David Runge
5e43a63b3c
Fixing issues with variable quoting and arrays
archiso/mkarchiso:
Calls to _pacman() need to be done with multiple parameters (e.g. array) instead of one string, as string splitting is
not done in that function anymore.
Turning _iso_efi_boot_args from string into an array to have an easier time of passing it to xorriso.
Calling xorriso within the if statements instead of providing -quiet via variable.
Fixing command_install() to provide packages separately to _pacman()

configs/releng/build.sh:
Replacing all newlines when retrieving the packages from packages.x86_64 with spaces so they will be properly provided
to "mkarchiso install".
2020-06-30 19:35:56 +02:00
David Runge
47533fd974
Introducing shellcheck in gitlab CI
archiso/mkarchiso:
Quoting all variables.
Changing pkg_list to be an array instead of a string for easier
handling. Using read to properly populate pkg_list from OPTARG with
stripped whitespaces.
Not exporting iso_label anymore as there seems to be no reason to do so.
Introducing line breaks.

.editorconfig:
Setting max_line_length to 120.
Adding a section for YAML files (e.g. .gitlab-ci.yml).

configs/releng/build.sh
Quting nearly all variables.
Introducing line breaks.

configs/baseline/build.sh:
Quoting all variables.
Introducing line breaks.

.gitlab-ci.yml:
Adding gitlab CI for shelleck linting of the config build scripts, mkarchiso and startup scripts in releng.

Closes #19
2020-06-29 20:10:23 +02:00
Christian Hesse
01521adc42 add zstd in supported squashfs compression list
Signed-off-by: Christian Hesse <mail@eworm.de>
2019-09-03 09:34:40 -03:00
Christian Hesse
55cdc1e9bc make sure the root filesystem is owned by root
Signed-off-by: Christian Hesse <mail@eworm.de>
2019-01-09 15:01:51 -03:00
Christian Hesse
3a95b7cafb generate pkglist with query (not sync) operation
Using the sync operation with list option fails with --sysroot when
signed database files are around. Instead use the query operation, which
uses the local databases of installed pakages only.

The only downside is that we do no longer record the originating
repository.

Signed-off-by: Christian Hesse <mail@eworm.de>
2018-07-26 17:15:29 -03:00
Christian Hesse
a00ba0f499 Remove cow file early for non-persistent systems
The kernel has an open file handle after setting up the mapping. We can
remove it early to make sure it is gone on shutdown.

This helps to keep the cow_device clean for non-persistent systems where
cow_directory contains a version specific string.

Signed-off-by: Christian Hesse <mail@eworm.de>
2018-06-29 13:08:24 -03:00
Gerardo Exequiel Pozzi
4b7c87fe76 [archiso] Drop --config when using --sysroot
https://lists.archlinux.org/pipermail/arch-releng/2018-June/003834.html

Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386@gmail.com>
2018-06-25 13:30:23 -03:00
Gerardo Exequiel Pozzi
5a57d83964 [archiso] Use --sysroot when list installed packages.
Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386@gmail.com>
2018-06-24 12:49:40 -03:00
Gerardo Exequiel Pozzi
b200ce82f7 Ensure correct umask for scripts
Fix for FS#58473

Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386@gmail.com>
2018-06-23 16:09:33 -03:00
Sean Enck via arch-releng
5f57352398 Drop the '-d' from pacstrap calls
The '-d' in pacstrap is a noop nowadays, keeping it may cause confusion

Signed-off-by: Sean Enck <enckse@gmail.com>
2018-06-17 12:21:37 -03:00
Christian Hesse
acf3ac9884 use a stronger hashing algorithm
Nobody wants to use md5 these days...

Signed-off-by: Christian Hesse <mail@eworm.de>
2017-10-17 23:54:55 -03:00
Gerardo Exequiel Pozzi
bfc0c81530 [archiso] Add ca-certificates.crt
Fix FS#54275

Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386@gmail.com>
2017-06-04 16:07:54 -03:00
David Thurstenson
3fec501225 Handle case where ip=dhcp
Signed-off-by: David Thurstenson <thurstylark@gmail.com>
2017-03-14 11:12:29 -03:00
David Thurstenson
ae854ed712 Add timeout to ipconfig call
Signed-off-by: David Thurstenson <thurstylark@gmail.com>
2017-03-14 11:12:29 -03:00
Christian Hesse
a04548ce2a pxe-common: not only search, but also domain entry in resolv.conf
Signed-off-by: Christian Hesse <mail@eworm.de>
2016-09-29 19:34:51 -03:00