tail: cannot open ‘/home/umbrel/umbrel/scripts/…/logs/memory-monitor.log’ for reading: No such file or directory
Filesystem information
Filesystem Size Used Avail Use% Mounted on
/dev/root 29G 3.2G 24G 12% /
/dev/root 29G 3.2G 24G 12% /
Startup service logs
– Logs begin at Thu 2022-05-19 09:46:30 UTC, end at Thu 2022-05-19 09:48:39 UTC. –
May 19 09:46:39 umbrel systemd[1]: Dependency failed for Umbrel Startup Service.
May 19 09:46:39 umbrel systemd[1]: umbrel-startup.service: Job umbrel-startup.service/start failed with result ‘dependency’.
External storage service logs
– Logs begin at Thu 2022-05-19 09:46:30 UTC, end at Thu 2022-05-19 09:48:39 UTC. –
May 19 09:46:34 umbrel systemd[1]: Starting External Storage Mounter…
May 19 09:46:34 umbrel external storage mounter[473]: Running external storage mount script…
May 19 09:46:35 umbrel external storage mounter[473]: Found device “Samsung PSSD T7”
May 19 09:46:35 umbrel external storage mounter[473]: Blacklisting USB device IDs against UAS driver…
May 19 09:46:35 umbrel external storage mounter[473]: Rebinding USB drivers…
May 19 09:46:35 umbrel external storage mounter[473]: Checking USB devices are back…
May 19 09:46:35 umbrel external storage mounter[473]: Waiting for USB devices…
May 19 09:46:36 umbrel external storage mounter[473]: Waiting for USB devices…
May 19 09:46:37 umbrel external storage mounter[473]: Waiting for USB devices…
May 19 09:46:38 umbrel external storage mounter[473]: Checking if the device is ext4…
May 19 09:46:38 umbrel external storage mounter[473]: Yes, it is ext4
May 19 09:46:39 umbrel external storage mounter[473]: mount: /mnt/data: can’t read superblock on /dev/sda1.
May 19 09:46:39 umbrel systemd[1]: umbrel-external-storage.service: Main process exited, code=exited, status=32/n/a
May 19 09:46:39 umbrel systemd[1]: umbrel-external-storage.service: Failed with result ‘exit-code’.
May 19 09:46:39 umbrel systemd[1]: Failed to start External Storage Mounter.
External storage SD card update service logs
– Logs begin at Thu 2022-05-19 09:46:30 UTC, end at Thu 2022-05-19 09:48:39 UTC. –
May 19 09:46:39 umbrel systemd[1]: Dependency failed for External Storage SDcard Updater.
May 19 09:46:39 umbrel systemd[1]: umbrel-external-storage-sdcard-update.service: Job umbrel-external-storage-sdcard-update.service/start failed with result ‘dependency’.
Karen logs
tail: cannot open ‘logs/karen.log’ for reading: No such file or directory
Docker containers
NAMES STATUS
Umbrel logs
ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor_proxy.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements
Bitcoin Core logs
ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor_proxy.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements
LND logs
ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor_proxy.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements
electrs logs
ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor_proxy.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements
Tor logs
ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor_proxy.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements
==== Result ====
The debug script did not automatically detect any issues with your Umbrel.
umbrel@umbrel:~/umbrel $
You have the answer right there.
Is very important for node operators to make the effort and read and understand these logs, not just posting them and wait for somebody to read them. Is part of the learning process.
Yes we can help you with reviewing them, but we need also to see that you understand these stuff. Otherwise is hard even for us to help you.
Your issue is more like due to those power cuts, your drive get hit and have some problems being accessed.
Could be also the power adapter (being first hit) that is failing and not giving anymore the correct voltage to the drive, through USB cable.
Could be even the USB cable itself, these also sometimes get fucked.
So start changing and testing your hardware. I know is a shity situation, but that is where you should look and concentrate to fix. Is not a software issue.
Try also re-flashing the mSD including changing it. These little memories get easily fucked with power cuts.
Your SSD is corrupted. Bad news!
As DarthCoin said, test all the hardware before trying to repair SSD file system, or you will only worst it, and loose all files.
The main files to preserve are the channel.backup and channels.db, copy them to another place first thing.
Cheers Darth thanks for the reply. Im learning lots just from directing me to the right place to take note of in the logs. Ive been posting and reading both here and telegram for answers and implementing crazy commands copy and paste and receiving more info to assess. The next day i forget the commands i learnt and have to reveiw every post again, just to consolidate my learning. Can be frustrating but enjoying the journey. I happened upon this post which details my own error and yet it seems Karens fault is totally different. Anyways thank you for your contribution to this community its settling to know that experienced people like you have made yrself availible to us noobs. Im a bitcoiner not a programmer but hey keen as mustarf and ill be paying too for all the noobs coming after me. Now…. time to get f_kn debit card out and spend more money on hardware. Buggar!! Lol
Hi, I noticed my note did not react (snowflake log showed last traffic about two weeks ago). When I logged in, the Blockchain wasn’t syncing. I ran the suggested updates and restarted, after that, I got this error message. The node ran fine until then. When I logged in, the only thing was that it told me it was running hot currently.
I made a debugging log per the guide, which can be found here.
I have the same error… Do I correctly understand the debug info that my MicroSD card is out of memory? Or is/are there other problems? Many many thanks!
=====================
= Umbrel debug info =
Umbrel version
0.5.3
Flashed OS version
v0.5.0
Raspberry Pi Model
Revision : d03114
Serial : 10000000e0c377ab
Model : Raspberry Pi 4 Model B Rev 1.4
Firmware
Dec 1 2021 15:01:54
Copyright (c) 2012 Broadcom
version 71bd3109023a0c8575585ba87cbb374d2eeb038f (clean) (release) (start)
Filesystem Size Used Avail Use% Mounted on
/dev/root 15G 15G 0 100% /
/dev/sda1 1.8T 593G 1.2T 35% /home/umbrel/umbrel
Startup service logs
Feb 14 10:30:06 umbrel umbrel startup[19520]: Starting status monitors…
Feb 14 10:30:06 umbrel umbrel startup[19520]: Starting memory monitor…
Feb 14 10:30:06 umbrel umbrel startup[19520]: Starting backup monitor…
Feb 14 10:30:06 umbrel umbrel startup[19520]: Starting decoy backup trigger…
Feb 14 10:30:06 umbrel umbrel startup[19520]: Starting Docker services…
Feb 14 10:30:08 umbrel umbrel startup[19520]: Starting manager …
Feb 14 10:30:08 umbrel umbrel startup[19520]: Starting auth …
Feb 14 10:30:08 umbrel umbrel startup[19520]: Starting tor_server …
Feb 14 10:30:08 umbrel umbrel startup[19520]: Starting tor_proxy …
Feb 14 10:30:08 umbrel umbrel startup[19520]: Starting dashboard …
Feb 14 10:30:09 umbrel umbrel startup[19520]: Starting manager … error
Feb 14 10:30:09 umbrel umbrel startup[19520]: ERROR: for manager Cannot start service manager: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/8c7a659f452a95b35ff75b61384e6d8bc2d7210ebf6b622e2e66b8d1d65a1bd9: no space left on device: unknown
Feb 14 10:30:09 umbrel umbrel startup[19520]: Starting dashboard … error
Feb 14 10:30:09 umbrel umbrel startup[19520]: ERROR: for dashboard Cannot start service dashboard: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/03375972e1e0d7a9bc27cd8fd6e094c8b373c83729a012148862593c2a62f2b9: no space left on device: unknown
Feb 14 10:30:09 umbrel umbrel startup[19520]: Starting tor_server … error
Feb 14 10:30:10 umbrel umbrel startup[19520]: ERROR: for tor_server Cannot start service tor_server: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/3eb8c0af2f79019078bf85515650a4ad84f9ba567609325ac8fdafc742a95d51: no space left on device: unknown
Feb 14 10:30:10 umbrel umbrel startup[19520]: Starting auth … error
Feb 14 10:30:10 umbrel umbrel startup[19520]: ERROR: for auth Cannot start service auth: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/cd923d52ec98ced8b61dd7899c594bd707ecf4eb4418f49de63dbc70ac8f93e7: no space left on device: unknown
Feb 14 10:30:10 umbrel umbrel startup[19520]: Starting tor_proxy … error
Feb 14 10:30:10 umbrel umbrel startup[19520]: ERROR: for tor_proxy Cannot start service tor_proxy: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/1c57a829db0b3fd8ca95315c667155510ba9ccc3398d3140c44888006cb1d85e: no space left on device: unknown
Feb 14 10:30:10 umbrel umbrel startup[19520]: ERROR: for manager Cannot start service manager: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/8c7a659f452a95b35ff75b61384e6d8bc2d7210ebf6b622e2e66b8d1d65a1bd9: no space left on device: unknown
Feb 14 10:30:10 umbrel umbrel startup[19520]: ERROR: for dashboard Cannot start service dashboard: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/03375972e1e0d7a9bc27cd8fd6e094c8b373c83729a012148862593c2a62f2b9: no space left on device: unknown
Feb 14 10:30:10 umbrel umbrel startup[19520]: ERROR: for tor_server Cannot start service tor_server: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/3eb8c0af2f79019078bf85515650a4ad84f9ba567609325ac8fdafc742a95d51: no space left on device: unknown
Feb 14 10:30:10 umbrel umbrel startup[19520]: ERROR: for auth Cannot start service auth: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/cd923d52ec98ced8b61dd7899c594bd707ecf4eb4418f49de63dbc70ac8f93e7: no space left on device: unknown
Feb 14 10:30:10 umbrel umbrel startup[19520]: ERROR: for tor_proxy Cannot start service tor_proxy: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/1c57a829db0b3fd8ca95315c667155510ba9ccc3398d3140c44888006cb1d85e: no space left on device: unknown
Feb 14 10:30:10 umbrel umbrel startup[19520]: Encountered errors while bringing up the project.
Feb 14 10:30:10 umbrel umbrel startup[19520]: Failed to start containers
Feb 14 10:30:10 umbrel systemd[1]: umbrel-startup.service: Control process exited, code=exited, status=1/FAILURE
Feb 14 10:30:10 umbrel systemd[1]: umbrel-startup.service: Failed with result ‘exit-code’.
Feb 14 10:30:10 umbrel systemd[1]: Failed to start Umbrel Startup Service.
External storage service logs
Feb 14 10:12:04 umbrel external storage mounter[499]: Checking USB devices are back…
Feb 14 10:12:04 umbrel external storage mounter[499]: Waiting for USB devices…
Feb 14 10:12:05 umbrel external storage mounter[499]: Waiting for USB devices…
Feb 14 10:12:06 umbrel external storage mounter[499]: Waiting for USB devices…
Feb 14 10:12:07 umbrel external storage mounter[499]: Checking if the device is ext4…
Feb 14 10:12:07 umbrel external storage mounter[499]: Yes, it is ext4
Feb 14 10:12:07 umbrel external storage mounter[499]: Checking filesystem for corruption…
Feb 14 10:12:07 umbrel external storage mounter[499]: e2fsck 1.44.5 (15-Dec-2018)
Feb 14 10:12:08 umbrel external storage mounter[499]: Superblock last mount time is in the future.
Feb 14 10:12:08 umbrel external storage mounter[499]: (by less than a day, probably due to the hardware clock being incorrectly set)
Feb 14 10:12:08 umbrel external storage mounter[499]: Superblock last write time is in the future.
Feb 14 10:12:08 umbrel external storage mounter[499]: (by less than a day, probably due to the hardware clock being incorrectly set)
Feb 14 10:12:08 umbrel external storage mounter[499]: umbrel: clean, 480967/122101760 files, 163230290/488378368 blocks
Feb 14 10:12:08 umbrel external storage mounter[499]: Mounting partition…
Feb 14 10:12:08 umbrel external storage mounter[499]: Checking if device contains an Umbrel install…
Feb 14 10:12:08 umbrel external storage mounter[499]: Yes, it contains an Umbrel install
Feb 14 10:12:08 umbrel external storage mounter[499]: Bind mounting external storage over local Umbrel installation…
Feb 14 10:12:08 umbrel external storage mounter[499]: Bind mounting external storage over local Docker data dir…
Feb 14 10:12:08 umbrel external storage mounter[499]: Bind mounting external storage to /swap
Feb 14 10:12:08 umbrel external storage mounter[499]: Bind mounting SD card root at /sd-card…
Feb 14 10:12:08 umbrel external storage mounter[499]: Checking Umbrel root is now on external storage…
Feb 14 10:12:09 umbrel external storage mounter[499]: Checking /var/lib/docker is now on external storage…
Feb 14 10:12:09 umbrel external storage mounter[499]: Checking /swap is now on external storage…
Feb 14 10:12:09 umbrel external storage mounter[499]: Setting up swapfile
Feb 14 10:12:10 umbrel external storage mounter[499]: Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
Feb 14 10:12:10 umbrel external storage mounter[499]: no label, UUID=cc9ffbb1-d2b2-447f-b6a6-57ea5e813001
Feb 14 10:12:10 umbrel external storage mounter[499]: Checking SD Card root is bind mounted at /sd-root…
Feb 14 10:12:10 umbrel external storage mounter[499]: Starting external drive mount monitor…
Feb 14 10:12:10 umbrel external storage mounter[499]: Mount script completed successfully!
Feb 14 10:12:10 umbrel systemd[1]: Started External Storage Mounter.
External storage SD card update service logs
Feb 14 10:12:17 umbrel external storage updater[901]: == Begin Update Script 01-run.sh ==
Feb 14 10:12:17 umbrel external storage updater[901]: =======================================
Feb 14 10:12:17 umbrel external storage updater[901]: =============== UPDATE ================
Feb 14 10:12:17 umbrel external storage updater[901]: =======================================
Feb 14 10:12:17 umbrel external storage updater[901]: =========== Stage: Install ============
Feb 14 10:12:17 umbrel external storage updater[901]: =======================================
Feb 14 10:12:17 umbrel external storage updater[901]: =============================================
Feb 14 10:12:17 umbrel external storage updater[901]: Installing on Umbrel OS v0.5.0
Feb 14 10:12:17 umbrel external storage updater[901]: =============================================
Feb 14 10:12:17 umbrel external storage updater[901]: ./01-run.sh: line 25: cannot create temp file for here-document: No space left on device
Feb 14 10:12:17 umbrel external storage updater[901]: == End Update Script 01-run.sh ==
Feb 14 10:12:17 umbrel external storage updater[901]: == Begin Update Script 02-run.sh ==
Feb 14 10:12:17 umbrel external storage updater[901]: =======================================
Feb 14 10:12:17 umbrel external storage updater[901]: =============== UPDATE ================
Feb 14 10:12:17 umbrel external storage updater[901]: =======================================
Feb 14 10:12:17 umbrel external storage updater[901]: ========= Stage: Post-update ==========
Feb 14 10:12:17 umbrel external storage updater[901]: =======================================
Feb 14 10:12:17 umbrel external storage updater[901]: == End Update Script 02-run.sh ==
Feb 14 10:12:17 umbrel external storage updater[901]: == Begin Update Script 03-run.sh ==
Feb 14 10:12:17 umbrel external storage updater[901]: =======================================
Feb 14 10:12:17 umbrel external storage updater[901]: =============== UPDATE ================
Feb 14 10:12:17 umbrel external storage updater[901]: =======================================
Feb 14 10:12:17 umbrel external storage updater[901]: =========== Stage: Success ============
Feb 14 10:12:17 umbrel external storage updater[901]: =======================================
Feb 14 10:12:17 umbrel external storage updater[901]: Removing backup
Feb 14 10:12:17 umbrel external storage updater[901]: ./03-run.sh: line 19: cannot create temp file for here-document: No space left on device
Feb 14 10:12:17 umbrel external storage updater[901]: == End Update Script 03-run.sh ==
Feb 14 10:12:17 umbrel external storage updater[901]: Deleting cloned repository
Feb 14 10:12:17 umbrel external storage updater[901]: Removing lock
Feb 14 10:12:17 umbrel systemd[1]: Started External Storage SDcard Updater.
Karen logs
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
Docker containers
NAMES STATUS
Umbrel logs
Attaching to manager
Tor Proxy logs
Attaching to tor_proxy
App logs
bitcoin
Attaching to
bitfeed
Attaching to
bluewallet
Attaching to
electrs
Attaching to
lightning
Attaching to
lightning-terminal
Attaching to
ln-visualizer
Attaching to
lnbits
Attaching to
mempool
Attaching to
ride-the-lightning
Attaching to
tailscale
Attaching to
thunderhub
Attaching to
==== Result ====
The debug script did not automatically detect any issues with your Umbrel.
Hello Darth, my name Javier, my English is not good. Apologies
After 1 year of optimal operation I started having problems with umbrel. At first I detected when I entered that bitcoin node was at 99.53% and very slow, I also remember having downloaded the umbre 26.0 update…
After restarting, resetting, I started directly without being able to enter
Error message… Error: System service failed
when I enter umbre.local…
Checking the HW…, the SSD card I confirmed that it works well. The specific question is, if I try formatting and/or reflashing my SSD card…do I lose all the user information, Umbrel passwords? Because I had Bitcoin node down, LN with open channels even with satoshis.
What would you recommend me to do? Thank you
