Commit Graph

1056 Commits

Author SHA1 Message Date
David Runge
bde3971991
Fix shellcheck complains in CI scripts
.gitlab/ci/build-host.sh:
Change the readonly TMPDIR variable to a global tmpdir variable and set it in the `init()` function.

.gitlab/ci/build-inside-vm.sh:
Change assigning the readonly tmpdir variable directly to assigning it after declaring it.
Change `cleanup()` and `create_zsync_delta()` to use bash-style statements and also check whether SUDO_GID is set before
using it.
2021-04-30 17:51:27 +02:00
Christian Hesse
1a97109639 mkarchiso: also add iso name in grub environment block 2021-04-07 16:40:18 +00:00
nl6720
09b6127fe8
mkarchiso: use -isohybrid-gpt-basdat instead of -appended_part_as_gpt for ISOs that will support BIOS booting
Some hardware, like Lenovo Thinkpad T420, will not BIOS boot if the disk has a valid GPT.
See https://bbs.archlinux.org/viewtopic.php?id=264096 .

Instead of a valid GPT, change to a valid MBR and invalid GPT similar to what was used before 729d16b48c. That layout, despite having crazy partition tables, boots everywhere.
The difference is that -append_partition is still kept and specified before -isohybrid-gpt-basdat. Thus the appended partition will be listed as EFI system partition in MBR and as Microsoft basic partition in the invalid GPT.

Fixes #102.
2021-04-07 10:29:35 +03:00
David Runge
c8599788f0
Revert "configs/releng: improve UX"
This reverts commit 8b6f3545e3.
2021-03-30 22:40:37 +02:00
David Runge
495721a79c
Add changelog entry for v52
CHANGELOG.rst:
Add changelog entry for v52
2021-03-30 21:46:07 +02:00
nl6720
8b6f3545e3
configs/releng: improve UX
Implements #90
2021-03-30 21:09:44 +02:00
Giancarlo Razzolini
9875249e02 releng/packages: Add archinstall to the list of packages
Add archinstall to the list of packages thare are installed on the ISO.
2021-03-30 18:14:27 +00:00
David Runge
bc007ca5f3
Add releases section with PGP information
README.rst:
Add a "Releases" section that specifies who is creating releases and which PGP key ID is used to sign tags.
Additionally, information about how to retrieve the relevant public key and how to verify a tag in the repository is
added.

Fixes #114
2021-03-29 23:16:04 +02:00
David Runge
d178183c2e
Reduce amount of checked mirrors
configs/releng/airootfs/etc/xdg/reflector/reflector.conf:
Reduce the amount mirrors that reflector checks from 70 to 20.
This significantly reduces the time it takes to end up with an up-to-date mirrorlist during boot with the releng
profile.

Fixes #92
2021-03-29 21:00:06 +02:00
David Runge
0664efcf3c
Allow redirects for scripts
configs/releng/airootfs/root/.automated_script.sh:
Add the `--location` curl parameter (see `man 1 curl`) to allow for curl to retrieve a remote script even if the source
is being redirected (e.g. moved permanently) when using the `script=` kernel commandline parameter.

Fixes #113
2021-03-29 19:26:27 +02:00
nl6720
d86f8606dc
configs/releng: add and enable ModemManager
ModemManager's mmcli is the simplest way to connect with WWAN modems.

Mention mmcli in MOTD.

Implements #110.
2021-03-26 17:19:31 +02:00
nl6720
2c9eabd53f
configs/releng: start DHCP client for mobile broadband
Add /etc/systemd/network/20-wwan.network

Related to #110.
2021-03-26 12:28:07 +02:00
nl6720
edfb50ab6c
configs/releng: rename 20-wireless.network to 20-wlan.network
The file is limited to Wi-Fi (Type=wlan in networkd configuration).
2021-03-26 12:18:08 +02:00
nl6720
8a521d0bfa
Update configs/releng/airootfs/etc/systemd/network/20-{ethernet,wireless}.network
* Match the device type instead of the interface name.
* Replace DHCP section with DHCPv4/DHCPv6. systemd split the sections.
2021-03-26 11:35:20 +02:00
nl6720
8cbc548359
mkarchiso: do not set default mksquashfs options
Remove hardcoded '-comp xz', it prevents using mksquashfs defaults.

Fixes #112.
2021-03-26 08:37:38 +02:00
nl6720
8050fa4797
.gitlab/ci/build-host.sh: install erofs-utils
Allow building ISOs with EROFS airootfs images in CI.
2021-03-26 00:45:30 +02:00
nl6720
e847c4ad75
.gitlab/ci/build-host.sh: Do not upgrade packages with files in /usr/lib/modules/
Packages with files in /usr/lib/modules/ depend on the current kernel, thus they should not be updated.
2021-03-21 16:04:39 +02:00
Michael Gilchrist
6bb12552e4 Recursively change file permissions for folders listed in profiledef.sh
- if a folder listed in the associative array ends with a "/",
  recursively apply chmod and chown.
2021-03-21 14:00:13 +00:00
Alexander Epaneshnikov
e43017c955 add a sound card firmware 2021-03-14 20:34:43 +00:00
nl6720
702026e124
.gitlab-ci.yml: do not use build:secure on forks
Forks may not have access to secure runners. Restrict build:secure to https://gitlab.archlinux.org/archlinux/archiso/ only.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/106 .
2021-03-14 21:27:04 +02:00
nl6720
c241285c5f
.gitlab/ci/build-host.sh: increase mksquashfs timeout to 40 minutes
Some mksquashfs runs take a very long time and 1000 seconds might not be enough.
2021-03-14 20:20:46 +02:00
Jonathon Fernyhough
f502b56ec2 Create zsync control file for delta downloads 2021-03-14 17:42:54 +00:00
nl6720
bc67933af1
Support EROFS
EROFS, like Squashfs, is a read-only file system. It can be used to store airootfs in an image file.
Its advantage is the support for POSIX ACLs. EROFS downside is that currently it only supports LZ4 compression (LZMA support is not yet fully implemented).

A difference from Squashfs is that, EROFS stores change time (ctime) not modification time (mtime). The reverse is true for Squashfs.

Implements https://gitlab.archlinux.org/archlinux/archiso/-/issues/59
2021-03-09 16:25:45 +02:00
nl6720
711ab4cd1e
archiso/initcpio/hooks/archiso: remove redundant /sfs/ from airootfs mount point
Remove /run/archiso/bootmnt directory if nothing is mounted there. An empty directory is just confusing.
2021-03-09 16:25:45 +02:00
David Runge
652ad4deed releng: Add usbmuxd to list of packages
configs/releng/packages.x86_64:
Add usbmuxd to list of packages, so that users have the option to use iOS devices out-of-the-box for data connection
during installation.

Fixes #99
2021-02-16 21:43:46 +00:00
Kristian Klausen
03ac49f64e Remove haveged
haveged was added 8 years ago[1] to increase entropy and presumably to
prevent entropy starvation.

A few things has changed since, most notable:
* the kernel actively tries to add entropy (jitter entropy)[2][3][4][5]
* /dev/random no longer blocks after CRNG initialization[6][7]

[1] d7e790d ("Initialize pacman keyring on bootup")
[2] 3f2dc2798b
[3] 50ee7529ec
[4] https://lore.kernel.org/lkml/alpine.DEB.2.21.1909290010500.2636@nanos.tec.linutronix.de/T/
[5] https://lwn.net/Articles/800509/
[6] 30c08efec8
[7] https://lwn.net/Articles/808575/

Fix #98
2021-02-16 16:33:06 +00:00
nl6720
97f510df81
.gitlab-ci.yml: use correct units for image size description
`du --block-size=MiB` (and `du -m`) returns mebibytes not megabytes.

Additionally, shorten the du command. `du --block-size=MiB` is the same as `du -m`.
2021-02-06 15:35:55 +02:00
David Runge
f0ef2f3caf
Add changelog file
CHANGELOG.rst:
Add file to track changes (at least for v51).
2021-02-01 09:59:49 +01:00
David Runge
b588c52665
build-host.sh: Style and syntax fixes
.gitlab/ci/build-host.sh:
Set shebang to /usr/bin/env bash to be more portable/flexible.
Turn all posix statements ([]) to bash style statements ([[]]), as we are using bash.
Terminate the list of parameters to rm or cp with --.

Replace the implementation of finding a local ISO to use with one that relies on a sorted list of potential images.

Use virtio-net-pci for networking with qemu.
Set the cow_spacesize to 4G for the archiso environment.

Use --needed in the call to pacman to not re-install already up-to-date targets.
Attempt a full system upgrade (but ignore the kernel).
Increase the timeout for when installing packages to the archiso environment using pacman to 120s, as a system upgrade
is being done as well.

Use systemctl poweroff -i to shut down the virtual machine as it is more future proof and robust.
2021-01-31 23:46:51 +01:00
David Runge
428bf47370
Add build stage for continuous integration
.gitlab-ci.yml:
Add a build stage to the gitlab CI, that facilitates the scripts below .gitlab/ci/, building the baseline and releng
profiles in parallel.
Distinguish the use-case in which builds are done for master, schedules and tags in a secure environment and any other
where builds just have to be fast (for ensuring nothing is broken).

Use MiB as block size for the du call when generating data for the metrics file.
2021-01-31 23:25:22 +01:00
David Runge
2a24429404
Add continuous integration scripts to linting
Makefile:
Add scripts below .gitlab/ci/ to the lint target of the Makefile.
2021-01-31 16:27:41 +01:00
David Runge
07239499a6
Add scripts for continuous integration
.gitlab/ci/build-host.sh:
Add script to be run in a container with access to qemu.
It is a slight modification of arch-boxes' build-host.sh script to cater to the specific archiso requirements.

.gitlab/ci/build-inside-vm.sh:
Add script to be run in virtualized environment, established by build-host.sh.
This script builds the actual archiso profiles and creates checksum for the resulting image files.
2021-01-31 16:23:04 +01:00
nl6720
f9a7b206a5
configs/releng: move locale-gen from customize_airootfs.sh to a pacman hook
This finally removes customize_airootfs.sh from releng.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/21 .
2021-01-31 11:34:56 +02:00
nl6720
a46c74087f
config/releng: remove pacman hooks specific to ISO build process from airootfs after they run
This works around https://bugs.archlinux.org/task/49347 .
Leaving the hooks in the airootfs image will result in it being run when pacstrap is run in the live environment. This should not happen as they are intended for the ISO build process only.

Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/91 .
2021-01-31 09:55:25 +02:00
Sven-Hendrik Haase
767c09569d
Update my email in AUTHORS 2021-01-31 02:12:36 +01:00
Michael Vorburger
dd51457d09 rm un-used set_image() dupe of check_image() in run_archiso.sh 2021-01-31 01:02:53 +00:00
Michael Vorburger.ch
23c5d52bd8 add vorburger to AUTHORS.rst 2021-01-31 01:00:18 +00:00
nl6720
0f20a11bb7
Support setting more variables in profiledef.sh and rework the way overrides are applied
- Apply overrides before validating the options.
- Parse all paths with realpath. Fixes https://gitlab.archlinux.org/archlinux/archiso/-/issues/84 .
2021-01-29 00:01:24 +02:00
Michael Vorburger
e7306a309c add cloud-init to baseline 2021-01-27 14:16:43 +00:00
Michael Vorburger
744b8f13ed make baseline have working ethernet networking, like releng 2021-01-27 14:16:43 +00:00
Michael Vorburger
94dd194c22 add SSH server to baseline
inspired by https://wiki.archlinux.org/index.php/Archiso#Prepare_an_ISO_for_an_installation_via_SSH
2021-01-27 14:16:43 +00:00
Michael Vorburger
221a9211e5 use same airootfs_image_tool_options in baseline as in releng 2021-01-27 14:16:43 +00:00
nl6720
4f4047a3f8
configs/releng: move the mirror uncommenting sed command from customize_airootfs.sh to a pacman hook
After pacman-mirrorlist is installed, /etc/pacman.d/hooks/uncomment-mirrors.hook will run a sed command which uncomments all Server lines in /etc/pacman.d/mirrorlist.
This brings us another step closer to the complete removal of customize_airootfs.sh.

Related to https://gitlab.archlinux.org/archlinux/archiso/-/issues/21 .
2021-01-27 00:07:45 +02:00
Christian Hesse
a2c8dd3173 archiso_pxe_common: remove resolv.conf before copy
Booting via PXE we want to keep our DNS configuration. So remove
/etc/resolv.conf in new root before copying the current file.

Without this systemd-resolved fallback nameservers are used and we see an
error message when the root ships a symbolic link to systemd-resolved's
stub-resolv.conf:

cp: not writing through dangling symlink '/new_root/etc/resolv.conf'
2021-01-26 21:59:01 +00:00
Christian Hesse
b30d1cad9b mkarchiso: add version information
To date the iso version was used for iso volume information and iso file name.
In my custom builds I do use it a lot more:

* Inside the root fs: The system knows about its own version. I use this to:
  -> report the version to a server (poor man's inventory)
  -> let the system update itself

* On the iso fs: The files are served via rsync, running systems transfer
  version file first to check for available update.

* A grub environment file on the iso fs: Booting the iso from grub allows
  to create cow directory per version:

    loopback loop archlinux.iso
    load_env -f (loop)/arch/grubenv
    linux (loop)/arch/boot/x86_64/vmlinuz-linux ... \
        cow_directory=archlinux/${VERSION} ...

So let's just create these files.
2021-01-26 10:00:38 +01:00
Christian Hesse
d153b48613 mkarchiso: fix typos 2021-01-23 21:04:17 +01:00
Michael Vorburger
0ae5cd961e lint ALL (future) scripts/*.sh, not just run_archiso.sh 2021-01-23 19:23:38 +00:00
Michael Vorburger
608d1100c8 add -c => -cdrom cloud-init.iso support to run_archiso.sh
Co-authored-by: nl6720 <nl6720@gmail.com>
2021-01-23 16:43:55 +00:00
Michael Vorburger
88304719d0 add cloud-init files to .gitignore (and loosen out/ and work/) 2021-01-23 16:07:36 +00:00
Michael Vorburger
955343925b make run_archiso forward VM's port 22 to host 60022, for easy SSH testing 2021-01-23 16:02:50 +00:00