Error: System service failed after update to 0.4.9

Updated to latest version today and services failed to start… debug logs below… not sure if any of the “segmentation fault” errors are a problem… or what they mean.

Any pointers would be welcomed ?

=====================
= Umbrel debug info =

Umbrel version

0.4.9

Flashed OS version

v0.3.10

Raspberry Pi Model

Revision : c03114
Serial : 100000001ac37bd5
Model : Raspberry Pi 4 Model B Rev 1.4

Firmware

Feb 25 2021 12:10:40
Copyright © 2012 Broadcom
version 564e5f9b852b23a330b1764bcf0b2d022a20afd0 (clean) (release) (start)

Temperature

temp=56.0’C

Throttling

throttled=0x0

Memory usage

          total        used        free      shared  buff/cache   available

Mem: 3.8G 379M 2.7G 8.0M 665M 3.3G
Swap: 4.1G 0B 4.1G

total: 10.0%
system: 10%
tor: 0%
thunderhub: 0%
sphinx-relay: 0%
specter-desktop: 0%
mempool: 0%
lnd: 0%
electrs: 0%
btcpay-server: 0%
bluewallet: 0%
bitcoin: 0%

Memory monitor logs

2022-05-18 19:42:25 Memory monitor running!
2022-05-18 19:42:36 Memory monitor running!
2022-05-18 19:42:47 Memory monitor running!
2022-05-18 19:42:58 Memory monitor running!
2022-05-18 19:43:09 Memory monitor running!
2022-05-18 19:43:21 Memory monitor running!
2022-05-18 19:43:32 Memory monitor running!
2022-05-18 19:43:43 Memory monitor running!
2022-05-18 19:43:54 Memory monitor running!
2022-05-18 19:44:05 Memory monitor running!

Filesystem information

Filesystem Size Used Avail Use% Mounted on
/dev/root 118G 3.1G 110G 3% /
/dev/sda1 916G 523G 347G 61% /home/umbrel/umbrel

Startup service logs

May 18 19:43:53 umbrel umbrel startup[19808]: Starting status monitors…
May 18 19:43:53 umbrel umbrel startup[19808]: Starting memory monitor…
May 18 19:43:53 umbrel umbrel startup[19808]: Starting backup monitor…
May 18 19:43:53 umbrel umbrel startup[19808]: Starting decoy backup trigger…
May 18 19:43:53 umbrel umbrel startup[19808]: Starting Docker services…
May 18 19:43:54 umbrel umbrel startup[19808]: /home/umbrel/umbrel/scripts/start: line 115: 19831 Segmentation fault docker-compose up --detach --build --remove-orphans
May 18 19:43:54 umbrel umbrel startup[19808]: Failed to start containers
May 18 19:43:54 umbrel systemd[1]: umbrel-startup.service: Control process exited, code=exited, status=1/FAILURE
May 18 19:43:54 umbrel systemd[1]: umbrel-startup.service: Failed with result ‘exit-code’.
May 18 19:43:54 umbrel systemd[1]: Failed to start Umbrel Startup Service.
May 18 19:44:04 umbrel systemd[1]: umbrel-startup.service: Service RestartSec=10s expired, scheduling restart.
May 18 19:44:04 umbrel systemd[1]: umbrel-startup.service: Scheduled restart job, restart counter is at 42.
May 18 19:44:04 umbrel systemd[1]: Stopped Umbrel Startup Service.
May 18 19:44:04 umbrel systemd[1]: Starting Umbrel Startup Service…
May 18 19:44:04 umbrel umbrel startup[20108]: ======================================
May 18 19:44:04 umbrel umbrel startup[20108]: ============= STARTING ===============
May 18 19:44:04 umbrel umbrel startup[20108]: ============== UMBREL ================
May 18 19:44:04 umbrel umbrel startup[20108]: ======================================
May 18 19:44:04 umbrel umbrel startup[20108]: Setting environment variables…
May 18 19:44:04 umbrel umbrel startup[20108]: Starting karen…
May 18 19:44:04 umbrel umbrel startup[20108]: Starting status monitors…
May 18 19:44:04 umbrel umbrel startup[20108]: Starting memory monitor…
May 18 19:44:04 umbrel umbrel startup[20108]: Starting backup monitor…
May 18 19:44:04 umbrel umbrel startup[20108]: Starting decoy backup trigger…
May 18 19:44:04 umbrel umbrel startup[20108]: Starting Docker services…
May 18 19:44:05 umbrel umbrel startup[20108]: /home/umbrel/umbrel/scripts/start: line 115: 20131 Segmentation fault docker-compose up --detach --build --remove-orphans
May 18 19:44:05 umbrel umbrel startup[20108]: Failed to start containers
May 18 19:44:05 umbrel systemd[1]: umbrel-startup.service: Control process exited, code=exited, status=1/FAILURE
May 18 19:44:05 umbrel systemd[1]: umbrel-startup.service: Failed with result ‘exit-code’.
May 18 19:44:05 umbrel systemd[1]: Failed to start Umbrel Startup Service.

External storage service logs

– Logs begin at Wed 2022-05-18 19:35:43 UTC, end at Wed 2022-05-18 19:44:05 UTC. –
May 18 19:35:48 umbrel systemd[1]: Starting External Storage Mounter…
May 18 19:35:48 umbrel external storage mounter[507]: Running external storage mount script…
May 18 19:35:48 umbrel external storage mounter[507]: Found device “External USB3.0”
May 18 19:35:48 umbrel external storage mounter[507]: Blacklisting USB device IDs against UAS driver…
May 18 19:35:48 umbrel external storage mounter[507]: Rebinding USB drivers…
May 18 19:35:48 umbrel external storage mounter[507]: Checking USB devices are back…
May 18 19:35:48 umbrel external storage mounter[507]: Waiting for USB devices…
May 18 19:35:49 umbrel external storage mounter[507]: Waiting for USB devices…
May 18 19:35:50 umbrel external storage mounter[507]: Checking if the device is ext4…
May 18 19:35:50 umbrel external storage mounter[507]: Yes, it is ext4
May 18 19:35:51 umbrel external storage mounter[507]: Checking if device contains an Umbrel install…
May 18 19:35:51 umbrel external storage mounter[507]: Yes, it contains an Umbrel install
May 18 19:35:51 umbrel external storage mounter[507]: Bind mounting external storage over local Umbrel installation…
May 18 19:35:51 umbrel external storage mounter[507]: Bind mounting external storage over local Docker data dir…
May 18 19:35:51 umbrel external storage mounter[507]: Bind mounting external storage to /swap
May 18 19:35:51 umbrel external storage mounter[507]: Bind mounting SD card root at /sd-card…
May 18 19:35:51 umbrel external storage mounter[507]: Checking Umbrel root is now on external storage…
May 18 19:35:52 umbrel external storage mounter[507]: Checking /var/lib/docker is now on external storage…
May 18 19:35:52 umbrel external storage mounter[507]: Checking /swap is now on external storage…
May 18 19:35:52 umbrel external storage mounter[507]: Setting up swapfile
May 18 19:35:52 umbrel external storage mounter[507]: Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
May 18 19:35:52 umbrel external storage mounter[507]: no label, UUID=513fbc7a-da4f-4389-82f2-21a75b3df04a
May 18 19:35:52 umbrel external storage mounter[507]: Checking SD Card root is bind mounted at /sd-root…
May 18 19:35:52 umbrel external storage mounter[507]: Starting external drive mount monitor…
May 18 19:35:52 umbrel external storage mounter[507]: Mount script completed successfully!
May 18 19:35:52 umbrel systemd[1]: Started External Storage Mounter.

External storage SD card update service logs

– Logs begin at Wed 2022-05-18 19:35:43 UTC, end at Wed 2022-05-18 19:44:05 UTC. –
May 18 19:36:04 umbrel systemd[1]: Starting External Storage SDcard Updater…
May 18 19:36:04 umbrel external storage updater[1715]: Checking if SD card Umbrel is newer than external storage…
May 18 19:36:05 umbrel external storage updater[1715]: No, SD version is not newer, exiting.
May 18 19:36:05 umbrel systemd[1]: Started External Storage SDcard Updater.

Karen logs

Got signal:
karen is getting triggered!
Got signal:
karen is getting triggered!
./karen: line 68: /home/umbrel/umbrel/events/triggers/: Is a directory
./karen: line 68: /home/umbrel/umbrel/events/triggers/: Is a directory
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events

Docker containers

NAMES STATUS
manager Up 8 minutes
bluewallet_lndhub_1 Restarting (3) 57 seconds ago
btcpay-server_web_1 Up 8 minutes
mempool_api_1 Up 8 minutes
sphinx-relay Up 8 minutes
lnd Restarting (1) 51 seconds ago

Umbrel logs

/home/umbrel/umbrel/scripts/debug: line 131: 20801 Segmentation fault docker-compose logs --tail=30 manager middleware

Bitcoin Core logs

/home/umbrel/umbrel/scripts/debug: line 137: 20805 Segmentation fault docker-compose logs --tail=30 bitcoin

LND logs

/home/umbrel/umbrel/scripts/debug: line 143: 20813 Segmentation fault docker-compose logs --tail=30 lnd

electrs logs

/home/umbrel/umbrel/scripts/debug: line 149: 20833 Segmentation fault docker-compose logs --tail=30 electrs

Tor logs

/home/umbrel/umbrel/scripts/debug: line 155: 21098 Segmentation fault docker-compose logs --tail=10 tor app_tor app_2_tor app_3_tor

App logs

bluewallet

./scripts/app: line 103: 21225 Segmentation fault docker-compose --env-file “${env_file}” --project-name “${app}” --file “${app_base_compose_file}” --file “${app_compose_file}” “${@}”

btcpay-server

./scripts/app: line 103: 21283 Segmentation fault docker-compose --env-file “${env_file}” --project-name “${app}” --file “${app_base_compose_file}” --file “${app_compose_file}” “${@}”

mempool

./scripts/app: line 103: 21356 Segmentation fault docker-compose --env-file “${env_file}” --project-name “${app}” --file “${app_base_compose_file}” --file “${app_compose_file}” “${@}”

specter-desktop

./scripts/app: line 103: 21410 Segmentation fault docker-compose --env-file “${env_file}” --project-name “${app}” --file “${app_base_compose_file}” --file “${app_compose_file}” “${@}”

sphinx-relay

./scripts/app: line 103: 21462 Segmentation fault docker-compose --env-file “${env_file}” --project-name “${app}” --file “${app_base_compose_file}” --file “${app_compose_file}” “${@}”

thunderhub

./scripts/app: line 103: 21550 Segmentation fault docker-compose --env-file “${env_file}” --project-name “${app}” --file “${app_base_compose_file}” --file “${app_compose_file}” “${@}”

==== Result ====

The debug script did not automatically detect any issues with your Umbrel.

Just noticed that for some reason… my update was to 0.4.9… not 0.4.18… trying that now…

That sounds like the OS didn’t update properly, so reflash the micro SD card, as it has the operating system only, so all of your user data is safe on the SSD/HDD. You’re doing a big jump in versions, from 0.3.10 to 0.4.18, so the updater on the dashboard might not get all the needed dependencies installed (primarily Docker) or the microSD card itself died, so you can try getting a new one if the reflash doesn’t work.

Thanks… yes I tried the upgrade again… it says it worked… but then debug logs still show the old version. Will refresh the SD and hopefully that solves it.

Flash to new version fixed.

1 Like

Awesome, I’m glad it worked. Like I said you were jumping from a 3.x to 4.x version, so that’s why the upgrade was giving us grief.

1 Like