- Trigger an update with deltas enabled between release A and release B on a test device.
- Find the delta image that gets pulled from balena images -- it should be REGISTRY_URL:DELTA_TAG
- Alternatively, if running Supervisor v12.9.5 or greater (12.9.6 highly recommended), look in the Supervisor's journal logs.
- Alternatively, make a request the delta endpoint with
curl -H "Bearer: Authorization $(cat /mnt/boot/config.json | jq -r '.deviceApiKey)" "https://delta.balena-cloud.com/api/v3/delta?src=SOURCE_REGISTRY_URL&dest=TARGET_REGISTRY_URL"
. This should return the delta URL and is the best amongst these 3 alternatives, but from my own investigations and limited knowledge, this works sporadically so is not recommended.
- After you have the registry delta URL, on production devices:
balena login registry2.balena-cloud.com -u d_$(cat /mnt/boot/config.json | jq -r '.uuid') -p $(cat /mnt/boot/config.json | jq -r '.deviceApiKey')
balena pull DELTA_URL
- `balena image tag DELTA_URL
-- Logs begin at Wed 2021-06-23 07:55:37 UTC, end at Fri 2021-06-25 08:23:04 UTC. -- | |
Jun 24 09:04:12 localhost kernel: Booting Linux on physical CPU 0x0 | |
Jun 24 09:04:12 localhost kernel: Linux version 5.4.70+ (oe-user@oe-host) (gcc version 9.3.0 (GCC)) #1 SMP PREEMPT Tue Jan 5 03:15:22 UTC 2021 | |
Jun 24 09:04:12 localhost kernel: CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c5387d | |
Jun 24 09:04:12 localhost kernel: CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache | |
Jun 24 09:04:12 localhost kernel: OF: fdt: Machine model: TI AM335x BeagleBone Black | |
Jun 24 09:04:12 localhost kernel: Memory policy: Data cache writeback | |
Jun 24 09:04:12 localhost kernel: efi: Getting EFI parameters from FDT: | |
Jun 24 09:04:12 localhost kernel: efi: UEFI not found. | |
Jun 24 09:04:12 localhost kernel: cma: Reserved 48 MiB at 0x9c800000 |
root@2f9551d:~# journalctl --since="2021-09-10 18:10" --until="2021-09-10 18:11" --no-pager -xe | |
-- Logs begin at Fri 2021-09-10 16:31:28 UTC, end at Fri 2021-09-10 18:16:21 UTC. -- | |
Sep 10 18:10:41 2f9551d systemd[1]: Starting SSH Per-Connection Server (52.4.252.97:48986)... | |
Sep 10 18:10:41 2f9551d systemd[1]: Started SSH Per-Connection Server (52.4.252.97:48986). | |
Sep 10 18:10:41 2f9551d dropbear[5031]: Failed loading /etc/dropbear/dropbear_dss_host_key | |
Sep 10 18:10:41 2f9551d dropbear[5031]: Failed loading /etc/dropbear/dropbear_ecdsa_host_key | |
Sep 10 18:10:41 2f9551d dropbear[5031]: Child connection from ::ffff:52.4.252.97:48986 | |
Sep 10 18:10:42 2f9551d dropbear[5031]: Pubkey auth succeeded for 'root' with key md5 4b:a1:6d:4b:1b:a1:58:0b:0d:bb:82:06:ea:26:16:c8 from ::ffff:52.4.252.97:48986 | |
Sep 10 18:10:44 2f9551d balenad[1191]: [2021-09-10T18:10:44.686Z] Applying target state | |
Sep 10 18:10:44 2f9551d resin-supervisor[3877]: [2021-09-10T18:10:44.686Z] Applying target state |
Jan 04 20:21:12 pi4 echo[7733]: config.json changed | |
Jan 04 20:21:12 pi4 openvpn[3310]: Tue Jan 4 20:21:12 2022 event_wait : Interrupted system call (code=4) | |
Jan 04 20:21:12 pi4 openvpn[3310]: Tue Jan 4 20:21:12 2022 Closing TUN/TAP interface | |
Jan 04 20:21:12 pi4 openvpn[3310]: Tue Jan 4 20:21:12 2022 /sbin/ip addr del dev resin-vpn local 10.240.72.29 peer 52.4.252.97 | |
Jan 04 20:21:12 pi4 avahi-daemon[3307]: Got SIGTERM, quitting. | |
Jan 04 20:21:12 pi4 avahi-daemon[3307]: Leaving mDNS multicast group on interface veth9ca5cd5.IPv6 with address fe80::bc2f:bbff:fe42:b0e7. | |
Jan 04 20:21:12 pi4 avahi-daemon[3307]: Leaving mDNS multicast group on interface vethe7a5351.IPv6 with address fe80::4079:f5ff:fe08:b25e. | |
Jan 04 20:21:12 pi4 avahi-daemon[3307]: Leaving mDNS multicast group on interface veth604b734.IPv6 with address fe80::ac83:5bff:fe42:aaaf. | |
Jan 04 20:21:12 pi4 avahi-daemon[3307]: Leaving mDNS multicast group on interface veth8a63e57.IPv6 with address fe80::8801:90ff:fe4c:825a. | |
Jan 04 20:21:12 pi4 avahi-daemon[ |
For all commands below, if OS is v2.78.0 or newer, replace resin-supervisor
with balena-supervisor
. See: https://github.com/balena-os/meta-balena/blob/master/CHANGELOG.md#v2780
cat /etc/resin-supervisor/supervisor.conf
should show corrupted file.- Verify no other files are corrupt with
grep -v "/var/cache/ldconfig/aux-cache" /resinos.fingerprint | md5sum --quiet -c -
See: https://www.balena.io/docs/learn/more/masterclasses/device-debugging/#112-storage-media-corruption - Note last reboot time and when the device was observed to stop working (i.e. Supervisor could not start), notify support agents if possible
- From the dashboard page, open Chrome / Firefox developer's console with Ctrl/Cmd + Shift + I. Run the following:
Replace loopN with loop number listed with partx
command
balena os download raspberrypi4-64 -o rpi4.img
losetup -fP --show rpi4.img
mount /dev/loopNp6 $MOUNTPOINT
If the mount fails with wrong fs type
or similar, this is most likely because the image is a flasher image (i.e. for Intel NUC, for example), and you will need to do the following:
Jan 15 00:29:13 balena balena-supervisor[7724]: [event] Event: Update notification {} | |
Jan 15 00:29:13 balena balena-supervisor[7724]: [api] POST /v1/update 204 - 17.259 ms | |
Jan 15 00:29:14 balena balena-supervisor[7724]: [info] Applying target state | |
Jan 15 00:29:15 balena balena-supervisor[7724]: [event] Event: Service kill {"service":{"appId":1868586,"serviceId":1382522,"serviceName":"proxy","commit":"2f4203beb706ee4583cb13617d15f92d","releaseId":2039758}} | |
Jan 15 00:29 |
// This script may be run in a Chrome dev console, and will delete all the device config variables for each device in a fleet, | |
// which the exception of *_SUPERVISOR_DELTA or *_SUPERVISOR_DELTA_VERSION. | |
await (async () => { | |
// Make sure FLEET_ID is the ID of your fleet. | |
const FLEET_ID = 12345; | |
// Get devices in fleet | |
const devices = await sdk.models.device.getAllByApplication(FLEET_ID, { | |
$select: 'id', | |
}); |
-
Run update-balena-supervisor or update-resin-supervisor on the device. If that doesn't work...
-
PATCH the
should_be_managed_by__release
(orshould_be_managed_by__supervisor_release
if that doesn't work) field in the device resource (/v6/device
) to equal the release ID of the target Supervisor version. To find the release ID of the target Supervisor version: (Replace variables starting with $ with the fields that are relevant to your case, and run this in a dashboard dev console)
await sdk.pine.get({
resource: 'supervisor_release',
options: {
$filter: {
is_for__device_type: {
DISCLAIMER: This gist is up-to-date as of Supervisor v16.6.1 and OS 6.0.x and may not necessarily apply to future Supervisor or OS versions.
journalctl -u balena-supervisor -u resin-supervisor -xef
If you see any errors with the format (HTTP code XXX)
, this is not a Supervisor error format. The Supervisor forwards these error messages from Docker Engine.