Commit Graph

1214 Commits

Author SHA1 Message Date
nl6720
f34c95797d
Calculate required efiboot.img size instead of hardcoding it
Use du to count the file size of the kernel(s), initramfs images and boot loader (and its configuration).
This allows to compress initramfs with something other than xz, or have more than one kernel installed.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/62 .
2020-09-27 20:54:11 +03:00
nl6720
8c047047ab
Split SYSLINUX configuration into per-kernel configuration files
Slightly simplifies adding boot loader configuration for more kernels.
Unfortunately the INCLUDE statement doesn't support wildcards, so each new file must be manually included in the main syslinux configuration file (syslinux.cfg for baseline and archiso_sys.cfg for releng).
2020-09-27 20:54:11 +03:00
nl6720
8276616f6f
archiso/mkarchiso: do not hardcode the kernel and initramfs file names
All kernels and initramfs images from airootfs are copied to ${install_dir}/boot/ on both ISO 9660 and FAT.
This allows providing multiple kernels. The kernel package needs to be added to packages.x86_64 and the boot loader configuration files created/adjusted appropriately.

Support all possible microcode initrd file names.
2020-09-27 20:54:11 +03:00
nl6720
83e4cb9fb9
Use the same file paths in both ISO 9660 and FAT
This allows to use only one systemd-boot configuration file per kernel.
Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/44 .
2020-09-27 20:54:11 +03:00
nl6720
136152e672
Do not rename the initramfs image
This is a breaking change. archweb, archiso-manager and various documentation must be updated.

* https://github.com/archlinux/archweb/blob/master/templates/releng/archlinux.ipxe
* https://github.com/pierres/archiso-manager/blob/master/Makefile
2020-09-27 20:54:10 +03:00
nl6720
9544bbfdf1
configs/releng: remove custom reflector.service and use the service provided by the package
archiso specific options are placed in a /etc/systemd/system/reflector.service.d/archiso.conf drop-in.
2020-09-27 19:55:38 +03:00
nl6720
486b1910dd
Do not overwrite existing files when copying from /etc/skel/
Copy /etc/skel/ only for users with UID in range 1000–60000.
Correct user home directory permission after copying files.
Fixes all /etc/skel issues from https://bugs.archlinux.org/task/67729 .
2020-09-05 11:23:00 +03:00
nl6720
45a5d229b3
Create directories and copy files to the FAT image using mtools instead of mounting the file system
- mtools supports SOURCE_DATE_EPOCH.
- The image file is operated on directly instead of mounting the file system. This is a prerequisite to limit the commands that run with root privileges. Related to https://gitlab.archlinux.org/archlinux/archiso/-/issues/40 .
- Add a reminder comment to not get rid of the dosfstools dependency, since a mformat made FAT image can fail to boot on some systems.
2020-09-02 22:31:04 +03:00
David Runge
4b401931bd
Add missing declaration of override_install_dir
**archiso/mkarchiso**:

In d90184a the unbound variable `override_install_dir` was introduced by accident.

It is required to be set to empty string to provide override functionality for `install_dir`.

Fixes #57
2020-08-25 22:02:50 +02:00
David Runge
d90184a7f0
Pass profile directory as parameter to mkarchiso
**archiso/mkarchiso**:

Change all override option parameters (i.e. `-A`, `-C`, `-D`, `-L`, `-P` and `-g`) to not directly override the global
variable they are tied to, but instead using an `override_` prefixed variable.

Add `_set_overrides()` to use `override_` prefixed variables (if set) to override those without a prefix.

Remove `-B` (a profile directory) from the list of parameters. The profile directory is now provided as separate
non-option parameter.

Add a call to `_read_profile()`, `_set_overrides()` and `command_build_profile()` to the fallthrough option of the
switch-case checking `command_name` - a non-option parameter to mkarchiso. This effectively provides the possibility to
set the profile directory using a non-option parameter, while still maintaining compatibility to legacy named arguments
used in the configs' `build.sh` scripts.

Extend the warning in regards to legacy `build.sh` based commands to mkarchiso by providing an EOL with archiso v49.

Change the help output to reflect the changes and further elaborate on the legacy commands used by `build.sh` scripts.

Change help output to be ordered alphabetically.

Add help output for `-r` and `-g` options.

Call `_set_overrides()` for legacy commands that accept one or more of the overriden options (i.e. `command_init`,
`command_install`, `command_prepare` and `command_iso`).

Various style fixes.

**configs/{baseline,releng}/build.sh**:

Change call to mkarchiso to use the profile's directory as a named argument instead of an option-argument.

**README.rst**:

Fix documentation on how to call mkarchiso with a profile directory.

Fix wording and ordering of option arguments for run_archiso documentation.

Fixes #52
2020-08-25 19:06:43 +02:00
nl6720
e2c5b4527f
archiso/mkarchiso: reduce duplication and add more info messages
Fixes to issues introduced in https://gitlab.archlinux.org/archlinux/archiso/-/merge_requests/59 :
- _make_boot_on_fat(): copy initramfs from "${airootfs_dir}/boot/" not "${isofs_dir}/". Otherwise UEFI-only ISOs cannot be built.

Some general fixes:
- Replace mkdir with install. Unlike mkdir, install does not complain when the target exists.
- Reduce excess newlines produced by messages.
- Ensure FAT image gets unmounted in case the script is interrupted.
- Create the ext4 image with mkfs.ext4 instead of truncate.
- Do not rely on user and group names for chown commands. Use numeric UID and GID instead.
- Minimize the times stderr is redirected to /dev/null.
- Add missing '?' to getopts.
- Standardize function definitions by removing spaces between the function name and () .
2020-08-21 14:16:32 +03:00
nl6720
13b8c7d099
archiso/mkarchiso: correct the path of airootfs/etc/machine-id
Use "${airootfs_dir}".
Fixes a mistake introduced when rebasing https://gitlab.archlinux.org/archlinux/archiso/-/merge_requests/64 .
2020-08-18 21:59:18 +03:00
nl6720
41d3d7d489
Add Joliet file system to the ISO
Joliet ensures correct file names capitalization on operating systems that support Joliet but not support Rock Ridge.
Implements https://gitlab.archlinux.org/archlinux/archiso/-/issues/45 .
2020-08-18 21:41:37 +03:00
nl6720
e6455b885b
archiso/mkarchiso: create an empty /etc/machine-id
Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/42 .
2020-08-18 21:27:52 +03:00
nl6720
ba3e8345ad
scripts/run_archiso.sh: support booting the image as a hard disk or an optical disc
Use virtio-scsi to attach the image. virtio-blk (if=virtio) is inconsistent about the used media type.
2020-08-18 21:21:23 +03:00
nl6720
55d9d1ffc4
scripts/run_archiso.sh: disable PXE ROM
ISO testing doesn't require the emulated firmware's PXE stack.
For UEFI, it only disable iPXE in favor of EDK II network stack. https://github.com/tianocore/tianocore.github.io/wiki/EDKII-Network-Over-QEMU#verification-result
2020-08-18 21:21:23 +03:00
nl6720
1e0ffd0297
scripts/run_archiso.sh: don't duplicate qemu commands for each boot mode
Various bash script improvements.
2020-08-18 21:21:22 +03:00
nl6720
e9f209efbf
Deprecate build.sh scripts and old mkarchiso commands
Replace build.sh scripts with calls to mkarchiso -B "profiledir" build_profile.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/37 .
2020-08-17 21:33:20 +03:00
nl6720
31b1dfdbe4
archiso/mkarchiso: delete all files in /boot not just the kernel and initramfs
There is nothing useful in /boot for the live system.
2020-08-17 17:14:06 +03:00
nl6720
7c2247f615
archiso/mkarchiso: allow choosing boot modes from profiledef.sh
Boot mode names are:

- bios_syslinux.mbr: SYSLINUX in MBR
- bios.syslinux.eltorito: SYSLINUX (ISOLINUX) via El Torito
- uefi-x64.systemd-boot.esp: systemd-boot on ESP in MBR
- uefi-x64.systemd-boot.eltorito: systemd-boot on ESP via El Torito

It is not yet possible to create an ISO with only El Torito or only MBR boot modes!
2020-08-17 17:14:06 +03:00
nl6720
41b9b89746
archiso/mkarchiso: copy make_* functions from configs/releng/build.sh
Adapt _make_* functions to mkarchiso.

Related to https://gitlab.archlinux.org/archlinux/archiso/-/issues/37 .
2020-08-17 17:13:37 +03:00
nl6720
1cc7375c25
archiso/mkarchiso: start preparing a build_profile command
Nothing is implemented yet!

configs/releng/profiledef.sh:
A test profile.
2020-08-17 14:22:36 +03:00
nl6720
0387b253c8
archiso/mkarchiso: general bash improvements
Quote all variables.
Terminate option processing using '--' for commands that support it.
Do not hardcode file descriptor.
Compare integers with arithmetic comparison instead of string comparison.
Replace echo with printf.
Use heredoc for usage text.
Don't print INFO messages when quiet is set.
Export SOURCE_DATE_EPOCH.
2020-08-17 14:22:24 +03:00
nl6720
951b217813
configs/releng/packages.x86_64: replace exfat-utils with exfatprogs
exfatprogs uses the kernel's driver unlike exfat-utils which provides a FUSE driver.
2020-08-11 11:05:44 +03:00
nl6720
e467c69e94
configs/releng/syslinux/archiso_pxe.cfg: add "checksum" and "verify" boot options
An ISO's checksum and GPG validation primarily matters when PXE booting, so it would be appropriate to enable these options for releng.
They should protect against booting a corrupt image.
2020-08-11 09:52:53 +03:00
nl6720
8e82bbbe4a
archiso/initcpio/hooks/archiso: remove option terminator from the blockdev command
Apparently blockdev does not support it.
In an ISO made using '-s img' (Squashfs with dm-snapshot), it results in:

    blockdev: Unknown command: --
2020-08-11 06:58:06 +03:00
nl6720
6312ccc9bc
Ensure all files in the ISO's Rock Ridge file system are owned by root
archiso/mkarchiso:
Add the -rational-rock option to xorriso.
This is a preparatory step for creating ISOs as a regular user. See https://gitlab.archlinux.org/archlinux/archiso/-/issues/40 .
2020-07-30 22:01:27 +03:00
nl6720
945f3834c2
Show mksquashfs progress bar when mkarchiso is run in verbose mode 2020-07-30 21:57:27 +03:00
nl6720
d52d36363e
releng: copy UEFI shell from airootfs instead of the host system
Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/35 .
2020-07-30 18:31:25 +03:00
nl6720
0ada52d47f
releng: place custom sshd_config in airootfs
Properly track the file instead of modifying it on-the-fly with customize_airootfs.sh.
See https://gitlab.archlinux.org/archlinux/archiso/-/issues/21 .
2020-07-30 15:53:46 +03:00
nl6720
eec944cdf7
CONTRIBUTING.rst: use make lint to run shellcheck
`make check` could get extended to include other things besides just linting.
2020-07-30 13:37:55 +03:00
nl6720
d7fc56e006
Move shell script linting into a separate make target
Makefile:
Move shellcheck to a "lint" target and change "check" so that it runs "lint".
This provides a more finer grained control and allows to extend "check" with things beyond just linting.

.gitlab-ci.yml:
Run `make lint` in the lint job.
2020-07-30 13:09:53 +03:00
David Runge
a96330699f
Use latest synced mirrors instead of age
configs/releng/airootfs/etc/systemd/system/reflector.service:
Use the 70 mirrors synced most recently (--latest) instead of a specific time since last sync (--age).
According to FS#67399 using 1hour as sync age is too short, as it sometimes leads to empty list.
2020-07-30 12:06:57 +02:00
nl6720
b5f04dba2b
Install mkinitcpio-archiso-hooks in airootfs instead of copying the hooks from host 2020-07-30 12:55:12 +03:00
nl6720
e9303ca175
Makefile: do not install mkinitcpio hooks by default
They are not required in the host system. See https://gitlab.archlinux.org/archlinux/archiso/-/issues/34 .
2020-07-29 19:14:59 +03:00
nl6720
ff0c13f90c
Build initramfs images only once
Place custom mkinitcpio.conf in airootfs.
Use a custom mkinitcpio preset to specify generated image file path.
2020-07-29 18:19:42 +03:00
nl6720
a5886ad71e
Remove unnecessary files from releng's airootfs
configs/releng/airootfs/etc/fstab:
/etc/fstab shipped by the filesystem package has only comments. There is no reason to replace it with an empty file.

configs/releng/airootfs/etc/systemd/system/default.target:
There is no harm in booting to graphical.target. releng does not enable (or even install) any service that has {Required,Wanted}By=graphical.target.
2020-07-29 18:10:25 +03:00
nl6720
b8dc4484e8
archiso/initcpio/install/archiso: fix GnuPG home directory permissions
Gets rid of a gpg warning:
gpg: WARNING: unsafe permissions on homedir '/tmp/mkinitcpio.*/root/gpg'
2020-07-29 18:05:16 +03:00
David Runge
e264b44682
Add license and basic documentation
LICENSE:
Add GPL-3.0 license.

{{archiso,configs}/*,.editorconfig,.gitlab-ci.yml}:
Add SPDX license identifier.

Makefile:
Add SPDX license identifier.
Install the `run_archiso.sh` script as global executable `run_archiso`.
Use -D and -t flags to install to install files more generically (without a previous call to install the directory).

README.rst:
Add README outlining the project's scope, how to build images from the profiles and how to test.

AUTHORS.rst:
Add list of all direct contributors to the repository.

CONTRIBUTING.rst:
Add basic contribution guidelines, explaining the linter and the license in use.

Closes #7
Closes #3
2020-07-29 14:27:48 +02:00
David Runge
47e11125e4
Copy files more generically
configs/*:
Copy all files that do not need a rename generically (not specifying a destination file name).
Do not rename vmlinuz-linux to vmlinuz or vmlinuz.efi (as this serves no purpose and makes the scripts more
complicated).
Do not rename microcode (i.e. {amd,intel}-ucode.img) when copying them and change all boot loader configuration files
that assume a renamed microcode image.
Add note and link to Arch Linux wiki to state why memtest.bin is renamed to memtest.
Copy license files for {amd,intel}-ucode and memtest more generically by placing them into subdirectories with the same
name as the package (to circumenvent overwriting one other).

Closes #33
2020-07-28 12:13:45 +02:00
nl6720
ba6b44f80a
Copy /etc/skel/ to user's home from build.sh instead of customize_airootfs.sh
Additionally copy the files to all custom user homes, not just root's.
2020-07-28 11:56:59 +03:00
nl6720
b922e8bc9c
Set up custom user home directories and their permissions 2020-07-28 11:56:59 +03:00
nl6720
78fe656042
Deprecate customize_airootfs.sh
Show a deprecation notice if airootfs/root/customize_airootfs.sh is found.
2020-07-28 11:56:58 +03:00
nl6720
df93289b73
Install all packages in one go and don't copy pacman.conf to airootfs
configs/{baseline,releng}/build.sh:
Copy custom files to airootfs before installing packages.
Instead of calling `mkarchiso init`, list all required packages in packages.x86_64 and install them all at once with `mkarchiso install`. The mkdir command which `mkarchiso init` performs is now done by make_custom_airootfs.

configs/releng/build.sh:
Don't copy configs/releng/pacman.conf to airootfs, it is only meant to provide a unmodified pacman.conf durring pacstrap. In airootfs, an unmodified /etc/pacman.conf will be installed with the pacman package.
2020-07-28 11:50:25 +03:00
nl6720
c54a41fb3d
Fix baseline profile
Install linux and mkinitcpio packages.
Remove root user's password.

Fixes https://bugs.archlinux.org/task/64236 .
2020-07-28 11:47:42 +03:00
nl6720
6689537563
Set root user's shell and password with custom /etc/passwd and /etc/shadow
Correct shadow file permissions from build.sh.
2020-07-20 13:39:41 +03:00
nl6720
12b02406eb
Revert "Add GitHub Actions workflow"
This reverts commit 2993b80816.

Actions are disabled in archiso's GitHub mirror.
2020-07-17 22:39:54 +03:00
nl6720
b40fb2ae53
Separate custom comments from shellcheck comments
Having a shellcheck directive and custom comments in the same line can trigger SC1107 on old versions of shellcheck.
2020-07-17 10:42:14 +03:00
nl6720
2993b80816
Add GitHub Actions workflow
.github/workflows/lint.yml:
Run `make check`.
2020-07-17 10:42:14 +03:00
nl6720
ecd339278d
Add check target to Makefile
Makefile:
Move shellcheck commands from .gitlab-ci.yml.

.gitlab-ci.yml:
Run `make check`.
Prevent partial upgrades, i.e. use '-Syu'.
Prevent reinstalling already installed packages by using pacman's '--needed' option.
Don't force-download the package database.
2020-07-17 10:42:14 +03:00