so having space issues, decided to reinstall Arch with right partitioning. It consists of three sections:
- backup needed files
- install Arch
- restore backup
I aim for the new Arch to mirror the old one, including the bash prompt, HISTFILESIZE, and more. I require my IRC client chat logs and samba configurations, among others.
# pacman -Qkk 2>/dev/null | grep -Ev '(0 missing files|0 altered files)'` or
# pacman -Qkk 2>/dev/null | grep -i 'backup file'This shows changed files of each package of your machine. Keeps track of files in backup=() array. Review each file individually and if you wish to retain them, document their absolute paths in a file, such as etc-conf.txt.
tar -T etc-conf.txt -czvf etc-conf.tar.gz
store it somewhere dry and away from sunlight - JK.
- it's not a bad idea to also backup whole /etc/ - doesn't require lots of space.
aryan@Archian:~$ sudo tar -czf etc-full.tar.gz /etc
tar: Removing leading `/' from member names
tar: /etc/pacman.d/gnupg/S.gpg-agent.browser: socket ignored
tar: /etc/pacman.d/gnupg/S.keyboxd: socket ignored
[...]
aryan@Archian:~$ du -sh etc-full.tar.gz
6.3M etc-full.tar.gz
aryan@Archian:~$ sudo install -Dt /media/info/Aryan/ArchBackup/ etc.tar.gz etc.conf etc-full.tar.gz
to restore, we need to diff it using utilities and add the changes. see https://wiki.archlinux.org/title/List_of_applications/Utilities#Comparison,_diff,_merge
I discovered a fantastic and user-friendly tool known as git! See https://wiki.archlinux.org/title/Dotfiles https://news.ycombinator.com/item?id=11071754
and then edit pkglist.txt to trim it.
And also perform a full backup (-Qq) just in case.
Here the core work begins,
- note this is my installation journal, you don't have to follow the steps with me. In fact, I recommend you not!
So I had two partitions on SSD: p1 (ESP) and p5. P5 is btrfs. I will have subvols on /, /boot, .local, .cache, .config due to various read/write operations on those directories. / On the HDD, I had about 90G of space at the beginning of the disk and 200G on the last sector. So I utilized LVM to create a large VG0. It is also btrfs and has subvols mounted on /home and /var since all VM images and Vagrant files are stored in /var. some notes about btrfs:
- when reinstalling, You don't need to format whole btrfs, you need to delete @root @var @boot subvols (or any system subvols) and its snapshots and create a new @root subvol and keep @home as it is. because subvols are just like directories.
- removing old snapshots with find:
find .snapshots/ -maxdepth3 -inum 256 -exec btrfs subvolume delete {} \;
| Device Name | Filesystem | Subvolume | Mount Point | size |
|---|---|---|---|---|
| /dev/nvme0n1p1 | vfat32 | - | /efi | 100M |
| /dev/nvme0n1p5 | btrfs | @root | / | 72G |
| /dev/nvme0n1p5 | btrfs | @boot | /boot | 72G |
| /dev/nvme0n1p5 | btrfs | @.config | ~/.config | 72G |
| /dev/nvme0n1p5 | btrfs | @.local | ~/.local | 72G |
| /dev/nvme0n1p5 | btrfs | @.cache | ~/.cache | 72G |
| Device Name | Partition Type | PV Size | lv path | lv size | Filesystem | Subvolume | Mount Point | size |
|---|---|---|---|---|---|---|---|---|
| /dev/sda1 | LVM2 | 92G | /dev/vg0/lv0 | 200G | btrfs | @home, @var | /home, /var | 200G |
| /dev/sda4 | LVM2 | 200G | /dev/vg0/lv0 | 200G | btrfs | @home, @var | /home, /var | 200G |
Let's go and make these partitions with me:
root@archiso ~ # cat /sys/firmware/efi/fw_platform_size
64
root@archiso ~ # ls /sys/firmware/efi/efivars/ - first we make sure we are booted with UEFI.
- then making required partitions with
cgdisk - create two lvm partitions on sda
root@archiso ~ # partprobe- update the kernel with the new partition information without the need to reboot.
root@archiso ~ # mkfs.btrfs -L root -fm single /dev/nvme0n1p5- make btrfs filesystem on nvme0n1p5 with label root and not metadata duplication.
root@archiso ~ # mount /dev/nvme0n1p5 /mnt
root@archiso ~ # btrfs subvolume create /mnt/@root
Create subvolume '/mnt/@root'
root@archiso ~ # btrfs subvolume create /mnt/@.config
Create subvolume '/mnt/@.config'
root@archiso ~ # btrfs subvolume create /mnt/@.local
Create subvolume '/mnt/@.local'
root@archiso ~ # btrfs subvolume create /mnt/@.cache
Create subvolume '/mnt/@.cache'
root@archiso ~ # btrfs subvolume create /mnt/@snapshots
Create subvolume '/mnt/@snapshots' - mount nvme0n1p5 and create subvols.
root@archiso ~ # pvcreate /dev/sda1
Physical volume "/dev/sda1" successfully created.
root@archiso ~ # pvcreate /dev/sda4
Physical volume "/dev/sda4" successfully created. - make HDD partitions ready to get on the vg0
root@archiso ~ # vgcreate vg0 /dev/sda1 /dev/sda4
Volume group "vg0" successfully created
root@archiso ~ # lvcreate -L 200G vg0 -n lv0
Logical volume "lvhome" created. - volume group and a logical volume with 200G from it created.
root@archiso ~ # mkfs.btrfs /dev/vg0/lv0 -L home-var -m single -f
Performing full device TRIM /dev/vg0/lv0 (200.00GiB) ...
NOTE: several default settings have changed in version 5.15, please make sure
this does not affect your deployments:
- DUP for metadata (-m dup)
- enabled no-holes (-O no-holes)
- enabled free-space-tree (-R free-space-tree)
Label: home-var
UUID: c2dee4c5-4946-4f6d-a6b2-2bb6eac02d99
Node size: 16384
Sector size: 4096 (CPU page size: 4096)
Filesystem size: 200.00GiB
Block group profiles:
Data: single 8.00MiB
Metadata: single 8.00MiB
System: single 4.00MiB
SSD detected: no
Zoned device: no
Features: extref, skinny-metadata, no-holes, free-space-tree
Checksum: crc32c
Number of devices: 1
Devices:
ID SIZE PATH
1 200.00GiB /dev/vg0/lv0 - make btrfs on lv0 with no metadata duplicity
root@archiso ~ # mount /dev/vg0/lv0 /mnt -v
mount: /dev/mapper/vg0-lv0 mounted on /mnt.
root@archiso ~ # btrfs subvolume create /mnt/@home
Create subvolume '/mnt/@home'
root@archiso ~ # btrfs subvolume create /mnt/@var
Create subvolume '/mnt/@var' - create two subvols for lv0
- mount / /boot /var /home /efi
- Even though /home is not required
root@archiso ~ # mount /dev/nvme0n1p1 /mnt/efi --mkdir # esp partition
root@archiso ~ # mount /dev/nvme0n1p5 -o ssd,compress=lzo,subvol=@root /mnt/
root@archiso ~ # mount /dev/nvme0n1p5 -o ssd,compress=lzo,subvol=@boot /mnt/boot --mkdir
root@archiso ~ # mount /dev/vg0/lv0 -o compress=lzo,subvol=@var /mnt/var --mkdir
root@archiso ~ # mount /dev/vg0/lv0 -o compress=lzo,subvol=@home /mnt/home --mkdir
root@archiso ~ # lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 776.8M 1 loop /run/archiso/airootfs
sda 8:0 0 1.8T 0 disk
├─sda1 8:1 0 92.6G 0 part
├─sda2 8:2 0 1.5T 0 part
├─sda3 8:3 0 50G 0 part
└─sda4 8:4 0 200.4G 0 part
└─vg0-lv0 254:0 0 200G 0 lvm /mnt/home
/mnt/var
sdb 8:16 1 14.8G 0 disk
├─sdb1 8:17 1 927M 0 part
└─sdb2 8:18 1 15M 0 part
nvme0n1 259:0 0 238.5G 0 disk
├─nvme0n1p1 259:1 0 100M 0 part /mnt/efi
├─nvme0n1p2 259:2 0 16M 0 part
├─nvme0n1p3 259:3 0 77.9G 0 part
├─nvme0n1p4 259:4 0 87.9G 0 part
└─nvme0n1p5 259:5 0 72.6G 0 part /mnt/boot
/mnt
- install packages
root@archiso ~ # pacstrap -K /mnt btrfs-progs lvm2 base base-devel linux linux-firmware linux-headers- create /etc/fstab
root@archiso ~ # genfstab -U /mnt >> /mnt/etc/fstab
root@archiso ~ # cat /mnt/etc/fstab
# Static information about the filesystems.
# See fstab(5) for details.
# <file system> <dir> <type> <options> <dump> <pass>
# /dev/nvme0n1p5 LABEL=root
UUID=215d9afb-2eda-477a-8aae-29ea3cd47b16 / btrfs rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=260,subvol=/@root 0 0
# /dev/nvme0n1p5 LABEL=root
UUID=215d9afb-2eda-477a-8aae-29ea3cd47b16 /boot btrfs rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=261,subvol=/@boot 0 0
# /dev/mapper/vg0-lv0 LABEL=home-var
UUID=c2dee4c5-4946-4f6d-a6b2-2bb6eac02d99 /var btrfs rw,relatime,compress=lzo,discard=async,space_cache=v2,subvolid=260,subvol=/@var 0 0
# /dev/mapper/vg0-lv0 LABEL=home-var
UUID=c2dee4c5-4946-4f6d-a6b2-2bb6eac02d99 /home btrfs rw,relatime,compress=lzo,discard=async,space_cache=v2,subvolid=256,subvol=/@home 0 0
# /dev/nvme0n1p1
UUID=04F3-BB5B /efi vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro 0 2
- chroot and make initramfs
- Before creating the initramfs, ensure that you add the
lvm2andbtrfshooks to the /etc/mkinitcpio.conf file:
HOOKS=(base udev autodetect microcode modconf keyboard keymap consolefont block lvm2 btrfs filesystems fsck)root@archiso ~ # arch-chroot /mnt
root@archiso ~ # mkinitcpio -P
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
==> Using default configuration file: '/etc/mkinitcpio.conf'
-> -k /boot/vmlinuz-linux -g /boot/initramfs-linux.img
==> Starting build: '6.7.9-arch1-1'
[...]
-> Running build hook: [lvm2]
-> Running build hook: [systemd]
-> Running build hook: [filesystems]
-> Running build hook: [fsck]
==> Generating module dependencies
==> Creating zstd-compressed initcpio image: '/boot/initramfs-linux-fallback.img'
==> Initcpio image generation successful- install systemd-boot:
[root@archiso /]# bootctl --esp-path=/efi/ install
Created "/efi/EFI/systemd".
Created "/efi/loader".
Created "/efi/loader/entries".
Created "/efi/EFI/Linux".
Copied "/usr/lib/systemd/boot/efi/systemd-bootx64.efi" to "/efi/EFI/systemd/systemd-bootx64.efi".
Copied "/usr/lib/systemd/boot/efi/systemd-bootx64.efi" to "/efi/EFI/BOOT/BOOTX64.EFI".
! Mount point '/efi' which backs the random seed file is world accessible, which is a security hole! !
! Random seed file '/efi/loader/.#bootctlrandom-seed5fd905e3637f6f52' is world accessible, which is a security hole! !
Random seed file /efi/loader/random-seed successfully written (32 bytes).
Created EFI boot entry "Linux Boot Manager".
[root@archiso /]# bootctl update
Skipping "/efi/EFI/systemd/systemd-bootx64.efi", same boot loader version in place already.
Skipping "/efi/EFI/BOOT/BOOTX64.EFI", same boot loader version in place already.- install kernel on esp
kernel-install add 6.7.9-arch1-1 /usr/lib/modules/6.7.9-arch1-1/vmlinuz- now the config file in
/efi/loaders/entries/path created - I also added
root=UUID=215d9afb-2eda-477a-8aae-29ea3cd47b16 rootflags=subvol=/@rootto the end ofoptions =line in the config file, because it didn't recognize my root partition.
*edit: I later also added nvidia_drm.modeset=1 module_blacklist=nouveau to that line because nouveau couldn't get unloaded even though I blacklisted it.
my fresh boot UEFI boot loader didn't persist and I replaced it with rEFInd. so I had EFI, 100MB size and it couldn't take long:
Filesystem Type Size Used Avail Use% Mounted on
/dev/nvme0n1p1 vfat 96M 88M 8.2M 92% /efithe kernel and initramfs files is huge for my ESP and I need ESP to be clean.
I had /boot on another subvol than / and systemd doesn't support btrfs. systemd let you have another partition with XBOOTLDR with partition type GUID of bc13c2ff-59e6-4262-a352-b275fd6f7172. but not on btrfs.
You can't, you need to use GRUB or refind for this if you want btrfs support. You could create another FAT partition and mount that to /boot if you don't want to reuse the ESP directly, but you need to make it adhere to the extended boot loader spec: https://wiki.archlinux.org/title/System … g_XBOOTLDR Again, You can't make systemd-boot boot from a btrfs partition. The "drivers" in this case would need to be provided by your actual motherboard. You can either substitute your ESP for /boot and put everything there or create an additional FAT partition.
resources: wiki.archlinux.org/systemd-boot#Installation_using_XBOOTLDR wiki.archlinux.org/REFInd#Manual_boot_stanza Forum
- first we need to change default subvolume to @boot:
aryan@archian:~$ sudo btrfs subvol set-default 261 /
aryan@archian:~$ sudo btrfs subvolume list -p /
ID 257 gen 2573 parent 5 top level 5 path @.local
ID 258 gen 2576 parent 5 top level 5 path @.cache
ID 259 gen 2577 parent 5 top level 5 path @.config
ID 260 gen 2554 parent 5 top level 5 path @root
ID 261 gen 2376 parent 5 top level 5 path @boot- it checks for /boot and /efi and finds entries, it recognizes my /boot filesystem, but if it doesn't we'd have to manually copy btrfs driver from
/usr/share/refind/drivers_x64/to/efi/EFI/refind/drivers_x64/. the config created in/efi/EFI/refind/refind.confis just a sample and the real config file is/boot/refind_linux.conf.
aryan@archian:~$ refind-install
Not running as root; attempting to elevate privileges via sudo....
[sudo] password for aryan:
ShimSource is none
Installing rEFInd on Linux....
ESP was found at /efi using vfat
Installing driver for btrfs (btrfs_x64.efi)
Copied rEFInd binary files
Copying sample configuration file as refind.conf; edit this file to configure
rEFInd.
Creating new NVRAM entry
rEFInd is set as the default boot manager.
Creating //boot/refind_linux.conf; edit it to adjust kernel options.
Installation has completed successfully.- delete initramfs and kernel files and the result:
aryan@archian:~$ df -hT /efi
Filesystem Type Size Used Avail Use% Mounted on
/dev/nvme0n1p1 vfat 96M 23M 74M 24% /efiBtrbk is a backup tool for btrfs subvolumes, taking advantage of btrfs specific capabilities to create atomic snapshots and transfer them incrementally to your backup locations.
Before running btrbk, you will need to create a configuration file. You might want to take a look at btrbk.conf.example provided with this package.
After a configuration change, it is highly recommended to check it by running btrbk with the -n,--dry-run option:
mounting subvolid=5 is recommended for snapshot/backup rather than snapshotting directly through filesystem (/home /).
/etc/btrbk/btrbk.conf:
snapshot_preserve_min 2d
snapshot_preserve 14d
timestamp_format short
volume /mnt/ssd
snapshot_dir @snapshots
subvolume @root
subvolume @boot
volume /mnt/hdd
snapshot_dir @snapshots
subvolume @var
subvolume @home- Btrbk does not create subdirs by default, the snapshot directory must first be created manually:
# btrfs subvolume create /mnt/ssd/@snapshots# btrfs subvolume create /mnt/hdd/@snapshotsThe "volume" section is merely used as a specifier for a base directory, and can be skipped if you prefer to configure everything using absolute paths. now run# btrbk run
create these hooks in the path: /etc/pacman.d/hooks/00-btrbk-pre.hook
[Trigger]
Operation = Upgrade
Operation = Install
Operation = Remove
Type = Package
Target=linux*
Target=systemd*
Target=glibc
[Action]
Depends = btrbk
Description = PreSnapshot btrbk
When = PreTransaction
Exec = /usr/bin/btrbk run --preserve --override=snapshot_create=always/etc/pacman.d/hooks/zx-btrbk-post.hook
[Trigger]
Operation = Upgrade
Operation = Install
Operation = Remove
Type = Package
Target=linux*
Target=systemd*
Target=glibc
[Action]
Depends = btrbk
Description = PostSnapshot btrbk
When = PostTransaction
Exec = /usr/bin/btrbk run --preserve --override=snapshot_create=alwaysthese creates a before/after btrfs snapshot every time linux* || systemd* || glibc update. I also have: pacdiff-pacman-hook-git, check-broken-packages-pacman-hook-git and this hook that checks orphaned packages:
/etc/pacman.d/hooks/99-orphans.hook
[Trigger]
Operation = Install
Operation = Upgrade
Operation = Remove
Type = Package
Target = *
[Action]
When = PostTransaction
Exec = /usr/bin/bash -c "/usr/bin/pacman -Qtd || /usr/bin/echo '==> No orphaned packages found.'"links: github.com/digint/btrbk#configuration, github.com/digint/btrbk#example-local-regular-snapshots-time-machine more hooks: reddit github.com/desbma/pacman-hooks/, forum.manjaro/t/howto-create-useful-pacman-hooks
lesspipe is a filter program for the less pager that allows you to view files in various formats directly within less. Here's how you can use lesspipe:
Install lesspipe if it's not already installed on your system. You can typically install it using your package manager. For example, on Debian-based systems, you can install it using:
sudo pacman -S lesspipeOnce lesspipe is installed, you can use it by setting the LESSOPEN environment variable to point to the lesspipe script. You can do this by running:
export LESSOPEN="| /usr/bin/lesspipe %s"
(/usr/bin/lesspipe.sh on ArchLinux)This command tells less to use lesspipe as the filter program.
Now, you can use less to view files of various formats. For example, you can view compressed files (e.g., gzip, bzip2), archives (e.g., tar, zip), PDF files, images, and more directly within less. For example, to view a compressed file example.tar.gz, you can use:
less example.tar.gzlesspipe will automatically decompress the file and show its contents within less.
Remember that the specific formats supported by lesspipe may vary depending on your system configuration and the available filters.
To enable:
1) Copy this file to somewhere (e.g. ~/.git-prompt.sh).
2) Add the following line to your .bashrc/.zshrc:
source ~/.git-prompt.sh
3a) Change your PS1 to call __git_ps1 as
command-substitution:
Bash: PS1='[\u@\h \W]$ '
ZSH: setopt PROMPT_SUBST ; PS1='[%n@%m %c]$ '
the optional argument will be used as format string.
3b) Alternatively, for a slightly faster prompt, __git_ps1 can
be used for PROMPT_COMMAND in Bash or for precmd() in Zsh
with two parameters, <pre> and <post>, which are strings
you would put in \[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]$(__git_ps1 " (%s)")\$ before and after the status string
generated by the git-prompt machinery. e.g.
Bash: PROMPT_COMMAND='__git_ps1 "\u@\h:\w" "\$ "'
will show username, at-sign, host, colon, cwd, then
various status string, followed by dollar and SP, as
your prompt.
ZSH: precmd () { __git_ps1 "%n" ":%~$ " "|%s" }
will show username, pipe, then various status string,
followed by colon, cwd, dollar and SP, as your prompt.
Optionally, you can supply a third argument with a printf
format string to finetune the output of the branch status- my .bashrc looks like this:
source .local/bin/git-prompt.sh
PS1='\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]$(__git_ps1 " (%s)")\$ 'Pull requests are welcome.