Umbrel Troubleshooting Guide

TABLE OF CONTENTS

Basic rules / steps before going down digging
  • PATIENCE IS THE KEY. If something bad happen with your node, do not panic, do not freak out, relax and analyze your options, write down your steps, what you did, what errors you get, run the debug logs and READ them (not just pass them to others).

image

Patience you must have my young padawan…

  • re-flashing the mSD card is a simple and first step. It doesn’t hurt in anyway user data (that is on the disk drive). These tiny cards can usually get broken or files corrupted on it. so a re-flash will re-format and put the right version of Umbrel OS.
  • learn/know how to connect using SSH to your node. Even if you are a Windows user, it’s easy through a command prompt window or using Putty
  • when you login using SSH you will not see typing the password. Is by design like that. Just type and press ENTER.
  • for RaspPi users, default user to login is umbrel. The default password is moneyprintergobrrr (in case you change it in the first login, it will be your chosen password).
  • most of the errors/issues come from DEFECTIVE HARDWARE, in special with Rasp Pis. Do not ignore this even if you really want NOT to be this the cause. IT WILL BE. So before doing any software changes or crazy stuff, check your hardware (USB cables, drive case, drives, mSD cards, power adapters).
  • do not just restart your node several times, without knowing what is causing. Restarting the node when is in a sync process can break more things than fix them. So until your are sure that you are not interrupting a process, you better just wait a while. Starting a node is a slow process that takes some time until is catching up. Restart only if is really necesary.
  • do not just unplug your node from power source. That can do more damage than good. Try first to access it through SSH and from command line use sudo reboot. Unplug from power should be the last resort.
  • Learn how to read the logs. Reading the logs is very important and understand what are saying. It will help you to debug yourself your node. There are 3 tabs on top: Logs - specific logs for Umbrel software, DMESG - specific logs about hardware, Apps - specific logs about apps running

image

  • You can share those logs in Umbrel Telegram group, there’s nothing private inside of them, only software / hardware debug.
How to login using SSH into my Umbrel node?

Open a terminal on your computer and enter:

ssh -t umbrel@umbrel.local

in Windows 7 you can use Putty, a tiny program to use SSH

in Windows 10 you can use PowerShell and the command is

ssh umbrel@umbrel

Instead of umbrel.local also you can use the local IP of your node.

The default password is moneyprintergobrrr.

On version 0.3.3 or later, the password is your personal user password instead.

Is my Raspberry Pi or other device compatible?

If you have a Raspberry Pi 4 with at least 4GB of RAM, you can run Umbrel on it. Is recommended to use 8GB RAM for better performance.
If you have a laptop, desktop, barebone also are compatible, with a a Linux OS (Debian or Ubuntu).

My Umbrel node keeps crashing, how can I generate the debug logs?

What can I do to fix the issue?

If you’re not using the official power supply, it’s probably the power supply. To detect undervoltage, connect to your RPi via SSH and run this command:

vcgencmd get_throttled

If it doesn’t output throttled=0x0, then it’s either the power supply or your SSD is using too much power (this can only be the case if you’re not using the recommended hardware).

Check if your router detects your node. If it doesn’t, either you ethernet cable isn’t plugged in correctly or the node doesn’t boot. If you think the ethernet cable isn’t the issue, follow the answer of the previous question.

If it does detect the node, try to access it with the IP address directly. If you can’t access the dashboard via the IP address either, try to run an automatic issue finding tool over SSH command:

cd umbrel

~/umbrel/scripts/debug --upload

It’ll automatically create a log and upload it to a server to be visible. Nothing personal / private is shared there. At the end of the log you will find a link to be shared with other members / devs in Umbrel Telegram group. The format of the link is like this:

https://umbrel-paste.vercel.app/ea2

You can contact, Umbrel community on Telegram or post on Umbrel Community Forum your issue.

My Umbrel node doesn’t boot. What can I do?

Do you have connected anything to the GPIO pins? If yes, try to unplug it and reboot the RPi by unplugging the power supply and then plugging it back in.

I can’t access the dashboard at umbrel.local. What can I do?

Please follow this dedicated guide about how to assign a static IP for your node.

Setting a fixed address on the Raspberry Pi

Please follow this dedicated guide about how to assign a static IP for your node.

Manually accessing `bitcon-cli` and `lncli`

On Umbrel, these binaries are always available in UMBREL_ROOT_DIR/bin/. On Umbrel OS, you can access them over SSH and run
cd ~/umbrel/bin
sudo docker exec lnd lncli

and

sudo docker exec bitcoin bitcoin-cli

Reset your Umbrel user data (if you lost your password)

Do this only if you do not have any funds on your LND wallet! If you have funds, then save your seed + backup file so you are able to restore it later if needed.

You are going to loose your seed, settings, data and applications!

Enter using SSH and run the following:

sudo systemctl stop umbrel-startup && sudo rm -rf ~/umbrel/lnd/!(lnd.conf) && sudo rm ~/umbrel/db/user.json && sudo rm ~/umbrel/db/umbrel-seed/seed && sudo systemctl start umbrel-startup

Manually updating Umbrel node

To manually update your node using SSH access.

Make a channels backup. Go to Umbrel dashboard - Lightning and on the right top side, 3 dots, save backup on your local PC.

A. For Rasp Pi users:

Enter using SSH into your Umbrel node, then run:

cd ~/umbrel && sudo ./scripts/update/update --repo getumbrel/umbrel#v0.4.8

Replace v0.4.8 with the version you want to update to.

If the update was stuck in update, run this before the above command:

sudo rm statuses/update-in-progress

Restart your node if is done OK:

sudo reboot

B. For normal Linux OS users:

  • Make a channels backup. Go to Umbrel dashboard - Lightning and on the right top side, 3 dots, save backup on your local PC.
  • Stop Umbrel node: sudo ./scripts/stop
  • Save your lnd.conf file if you edited (add color, name or specific features to your node). How to edit lnd.conf in Umbrel. More details about what you can add to a lnd.conf file here. After version 0.4.0 this file is not overwritten anymore with a default one, but just in case, make a backup.
  • Update system: sudo apt update and then sudo apt upgrade
  • Restart system
  • Update Umbrel (replace version with the latest release):
    cd ~/umbrel && sudo ./scripts/update/update --repo getumbrel/umbrel#v0.4.8
  • Replace lnd.conf file with the one you save it before update (After version 0.4.0 this file is not overwritten anymore with a default one)
  • Start Umbrel: sudo ./scripts/start
  • Leave the node to catch up with blocks and logs, usually takes several minutes, be patient. Now you can enter into dashboard Umbrel

C. Just re-flash the mSD card with the latest Umbrel OS version, like you did when you install it first time.

Recovering from a channels.backup (Umbrel OS)

Once you’ve restored from the 24 words, it might take a few minutes to a few hours for it to scan all of your previous Bitcoin (on-chain) transactions and balances. Meanwhile, here’s how you can restore the funds in your Lightning channels.

STEP 1: COPY OVER THE CHANNEL BACKUP FILE FROM YOUR COMPUTER TO YOUR UMBREL.

Enter using SSH and run this:

scp <path/to/your/channel/backup/file> umbrel@umbrel.local:/home/umbrel/umbrel/lnd/channel.backup

Replace <path/to/your/channel/backup/file> with the exact path to channel backup file on your computer

The password is *moneyprintergobrrr*, except on version 0.3.3 or later where the password is your personal user password instead.

STEP 2: RECOVER FUNDS

cd ~/umbrel && ./bin/lncli restorechanbackup --multi_file /data/.lnd/channel.backup

After you run this, wait for 1 minute. You should now be able to see your channels being closed on http://umbrel.local/lightning.

A more detailed guide about 6 ways to recover your funds stuck in an unresponsive node here

How can I edit the alias of my Umbrel node?

Please keep the following security disclaimer in mind (by @mayankchhabra, the founder of Umbrel):

Aliases can do more harm than good by leaking your private info (h/t @lukechilds for bringing this up when we were considering setting default aliases as ’s Umbrel). Imagine you name your alias “Lounès’s Umbrel”. I can then go to 1ml.com, instantly find your node, see your balance, open channels, etc. There isn’t much of an upside of setting a custom alias for private use as aliases aren’t unique and you can’t directly open channels by just using them as you still need the public key (and the onion address if it’s a Tor node). They have more value for bigger nodes (usually businesses) like Bitrefill, Bitfinex, etc so you can instantly find them and open a channel.

By this you can rename your LN node via SSH, so you do not have random name.

sudo nano ~/umbrel/lnd/lnd.conf

Add alias=My amazing node just after [Application Options]


[Application Options]

alias=My amazing node

listen=0.0.0.0:9735

rpclisten=0.0.0.0:10009

Save the file using Ctrl + X and y
Restart your node:

sudo systemctl restart umbrel-startup

NOTICE:

  • If you need more advanced information about configuring the lnd.conf file here you have a full example explained of that config file.
  • For all previous versions of Umbrel v0.4.0, keep a copy of this lnd.conf file. After Umbrel update this file will be overwritten. So after a software update put the lnd.conf file back from your copy.
  • After the version 0.4.0 the lnd.conf file is not overwritten anymore, so no need to make backups.
What is the procedure to rescan wallet UTXOs?

I sent funds to my node wallet and wasn’t 100% synced, now I can’t see them.

Step 1 - Enter using SSH into your node:

Step 2 - execute the following command to rescan wallet’s UTXOs:

sed -i "s/\[Application Options\]/\[Application Options\]\nreset-wallet-transactions=true/g;" ~/umbrel/lnd/lnd.conf && sudo reboot

Step 3 - Wait some time so LND can rescan all wallet’s UTXOs. If it solve the issue, run this to stop it from rescanning on a future restart:

sed -i "s/reset-wallet-transactions=true//g;" ~/umbrel/lnd/lnd.conf && sudo reboot

Wait, it can take quite a while.

One Umbrel App do not start or it can’t be installed

If you have error that a specific Umbrel App do not start or it keeps staying in “installing”, maybe the docker container for that app it wasn’t installed correctly or it hangs.

Step 1 - Enter using SSH into your Umbrel node:

Step 2 - stop Umbrel node with:

cd umbrel

Normal Linux OS use sudo ./scripts/stop
UmbrelOS (RPi) use sudo systemctl stop umbrel-startup

Step 3 - clean / reconfigure the broken container:

sudo rm .env && sudo ./scripts/configure

Step 4 - apply latest Umbrel update (replace the version number with the latest):

sudo ./scripts/update/update --repo getumbrel/umbrel#v0.4.8

Restart your node:

Normal Linux OS use sudo ./scripts/start
UmbrelOS (RPi) use sudo systemctl start umbrel-startup

You can open Umbrel dashboard, but wait until latest blocks are catching up and also electrs is synced. All txs and channels will appear after a while.

Using previously downloaded blockchain to save time when setting up my Umbrel node

This is not native supported by Umbrel for different reasons but you can make it at your own risk following these instructions:

  1. Flash Umbrel OS to an SD card and boot it in a Pi with your SSD attached
  2. Run through the setup process
  3. Go to the settings page and shutdown Umbrel (Your SSD is now formatted correctly for Umbrel)
  4. Remove the SSD and attach it to a host Linux machine with access to the pre-downloaded blockchain
  5. Copy blocks and chainstate directories from your Bitcoin Core data directory to /umbrel/bitcoin/ on the SSD
  6. Remove the SSD from the host, plug it back into your Umbrel, and power it back on.

Notes/Options:

  • Download blockchain using torrent, here is the torrent file (from bitcoin.org) and alternative source here.
  • Open Bittorrent and add the torrent file.
  • Start downloading
  • Copy the downloaded files into bitcoin/data folder
  • To read/attach the node disk to a Windows machine, you will need a software to read ext4 File System. For this you can use this simple driver or Linux File System from Paragon Software (trial)

Another way could to sync from another running Bitcoin node from same LAN or trusted connection, without the need to shut down the node to copy the data disk to disk.
In your new node, use the following command to connect to existing running node and sync the blockchain data directly, on the fly.
Enter SSH into your new Umbrel node, then:
cd ~/umbrel/bin
bitcoin-cli addnode "192.168.0.6:8333" "onetry"
replace “192.168.0.6” with your existing running node IP.
This will make your new node to sync directly from your old running node, without need to be connected to internet, if you have a slow connection or Tor is making problems.
Once is finished, you can remove the IP from peers and go back to normal connection.
More details here

How can I see logs for each node module?

Enter using SSH into your Umbrel node, then do this command:

cd /home/umbrel/umbrel && sudo docker-compose logs bitcoin

replace “bitcoin” with any other module (docker repository) you want to check:

tor, lnd, electrs etc.

My external disk drive is not mounting, what can I do?

If you detect that your disk drive is not recognized, is blacklisted, is not accessible, enter SSH and run de debug logs:

~/umbrel/scripts/debug --upload

At the end of the log you will obtain a link where the log was uploaded. Open it and see the lines about your USB drive (" External storage service logs" section).

If you see:

Blacklisting USB device IDs against UAS driver...

USB devices weren't registered after 10 tries...

then, try this command using SSH access:

sed -i 's/ blacklist_uas/ #blacklist_uas/g' /home/umbrel/umbrel/scripts/umbrel-os/external-storage/mount && sudo reboot

After that, check again your USB cabling connection and restart normally your node. Wait until the node is catching up with the blocks, it will take a while, be patient.

After a bad shutdown, the node is not syncing

After it was 100% synced, I shutdown the node. Now can access the UI dashboard but its not syncing anymore. Did a reflash for the mSD. Troubleshoot log says:

Bitcoin Core logs

Attaching to bitcoin

bitcoin | Error: Failed loading settings file:

bitcoin | - Unable to parse settings file /data/.bitcoin/settings.json

How to fix this issue:

Enter using SSH into your Umbrel node and run:

sudo rm ~/umbrel/bitcoin/settings.json

Restart the node machine

sudo reboot

After update Umbrel user data is not accessible

Some Rasp Pi users are claiming this issue, but is not for all. After updating with latest Umbrel version the node is starting but in dashboard the txs and LN channels are not functional.

How to fix it:

Enter using SSH into your node and run:

sudo nano ~/umbrel/lnd/lnd.conf

find the line

protocol.anchors=true

and delete it
Restart your node

sudo reboot

Some docker component fail to start

I can’t access umbrel.local on browser or ip address. Did ssh and ran debug script. First suspect line is:

stat /var/lib/docker/overlay2/....... no such file or directory

How to fix this issue:

  • just in case, re-flash the mSD card with the latest version of UmbrelOS (exactly the steps you did first time installing your node using the instructions from getumbrel.com
  • If still don’t do nothing, use this command (enter using SSH into your node):

sudo systemctl stop umbrel-startup.service && docker system prune --force --all && sudo systemctl start umbrel-startup.service

Restart your node

sudo reboot

Optional another command to clear the docker containers is:

sudo docker kill $(sudo docker ps -aq) && sudo docker rm $(sudo docker ps -aq)

then restart your node

How to add a WatchTower to my node?

Enter using SSH into your node and run:

cd ~/umbrel

nano lnd/lnd.conf

Add these 4 lines at the bottom:

[wtclient]
wtclient.active=true

[watchtower]
watchtower.active=true

save & close the lnd.conf file (CTRL+X), then run
docker-compose restart lnd

At this point, your node is running the client & server.

To get your WTServer address, run this:

docker exec -it lnd lncli tower info

To add some else’s node as WT to your node, have them run that command above (on an umbrel) and get the “uris” value, and you run this on your Umbrel (in umbrel folder):

docker exec -it lnd lncli wtclient add

(and before you press return, add their “uris” value. it should look like this:

e25b281e05320d83b58........c@qxyhqlor3wqnys7zajalxy5smfapdi7qd...........onion:9911

To check if your node is connected to a tower, run this:
docker exec -it lnd lncli wtclient towers

Here you have a list of altruistic nodes that offers WT services for your node.

I have a closed channel stuck, what can I do to release the funds?

I initiated a channel close on Umbrel 2 weeks ago but it remains stuck on “closing”." If I select “view info” on the channel in RTL (Ride The Lightning), I can see it under “Waiting Close.” I have tried restarting my node several times.

What to do:

Enter using SSH into your node.

Then run this command to check pending channels:

~/umbrel/bin/lncli pendingchannels

Look in the array “waiting_close_channels” and identify your pending channel with a string like

851ddc0900b775ff4afbd12ed4c8c003456eaea862ba96664344b06a04eec7c5:1

Also you can see when the funds will be released checking the “maturity_height”

"waiting_close_channels: [ ...."maturity_height": 700998,"

That means you will have to wait until that block 700998 is mined and your funds will be released. So just wait.

If you are very impatient, then you can force close that channel. Run this command (put the right id of the channel, this is just an example):

~/umbrel/bin/lncli closechannel --force 851ddc0900b775ff4afbd12ed4c8c003456eaea862ba96664344b06a04eec7c5 1

Again you can run the command to see the maturity:

~/umbrel/bin/lncli pendingchannels

and you will see the new “maturity_height” block number.

Watch Umbrel logs in real time

Enter using SSH into your node.
Then run this command to watch logs in real time:
cd umbrel
docker-compose logs -f bitcoin
Replace with lnd / electr / tor for each node module

How to change Thunderhub hardcoded password

To change thunderhub password:

  • ssh to umbrel
  • open file /home/umbrel/umbrel/apps/thunderhub/data/thubConfig.yaml
  • replace password value on first line. Keep the single quotes ', e.g. masterPassword: my new password
  • Uninstall Thunderhub and re-install it again (will take that your new password)
Electrum Server is not running correctly or cannot connect

So, some users report that cannot connect the wallets/services to their Electrum Server (electrs service in Umbrel).
Step 1: check status of electrs service, enter SSH into your node and run:
cd ~/umbrel
docker-compose logs --tail=300 electrs

See if there’s some file corruption or the index is broken. If there is such thing, run this command to repair / reindex electrs database:

sudo systemctl stop umbrel-startup && sudo rm -rf ~/umbrel/electrs/!(electrs.toml) && sudo systemctl start umbrel-startup

Then wait for one day (to allow electrs to reindex) and try connecting your BlueWallet to your Umbrel (or any wallet that supports Electrum) and see if that makes any difference.

How can I change the user name for my Umbrel node?
ssh -t umbrel@umbrel.local 

(The password is your dashboard password)

Then run:

echo "$( jq '.name = "NEWNAME"' ~/umbrel/db/user.json )" > ~/umbrel/db/user.json

Don’t forget to change NEWNAME by your new username.

Be careful running this command. This file contains your hashed password and seed. Copy paste it and change only NEWNAME.

Login screen it just freeze, can't login

I just setup the node and see the login screen and the button is not active, can’t login or continue.

ssh -t umbrel@umbrel.local 

(The password is moneyprintergobrrr. Be aware that you will not see what you type, just type it and press enter)
Then run:

cd umbrel
sudo systemctl stop umbrel-startup && sudo rm -rf ~/umbrel/lnd/!(lnd.conf) && sudo rm -f ~/umbrel/db/user.json && sudo rm -f  ~/umbrel/db/umbrel-seed/seed && sudo systemctl start umbrel-startup
How can I restart a service of the node without restarting the machine?

To restart only a single service without restarting the whole machine run this:

ssh -t umbrel@umbrel.local 

Use your umbrel user password.
Replace “lnd” with other services of your node: bitcoin, lnd, tor, electrs etc

cd ~/umbrel && docker-compose restart lnd
Wallet is locked. How to unlock it in command line
ssh -t umbrel@umbrel.local 

The prompted password is your dashboard password.
Then execute:

cd ~/umbrel && docker-compose exec lnd /bin/bash

Then:

lncli unlock

When prompted, enter the password moneyprintergobrrr
Done. Now open Umbrel dashboard page and login.

Blockchain data is corrupted, how can I fix it?

You have two options:

  • delete the blocks data (only) and re-sync all blocks again
  • run re-index command (takes time)

To run re-index, SSH into umbrel and:
cd umbrel/bitcoin
nano bitcoin.conf

add this line
reindex=1
CTRL+X - save it - ENTER to exit

restart the umbrel node:
sudo ./scripts/stop
sudo ./scripts/start

Have patience, it will take long time, maybe days.
You can check the status in logs:
SSH
cd umbrel
docker-compose logs -f bitcoin

After finishes, stop the node and remove the reindex=1 line, then restart the node in normal mode.
It will take also some time for electrs to reindex the blocks.

Several docker services could not start - Failed to start containers

If you see in your node debug logs, that error message about docker “failed to start containers”, SSH into your node and run this command:

cd umbrel
sudo systemctl stop docker.service && sudo systemctl stop docker.socket && sudo rm -rf /var/lib/docker/* && sudo umount /var/lib/docker || true && sudo rm -rf /mnt/data/docker/* && sudo reboot

I updated my Umbrel but it stuck in finish it

I run the Umbrel update but it gets stuck and in debug logs I see this error:
umbrel external storage updater

That means the updating process was not correctly finished and you need to restart it.
SSH into node and run:
rm -f ~/umbrel/statuses/update-in-progress && sudo reboot
The node will reboot and all will be back.

RTL stuck in starting after update Umbrel

I updated my Umbrel to latest version and now RTL is not starting anymore. I tried restarting, re-install but nothing fix it.
This is a known issue on amd64 based Umbrels. It will be solved with the next release. For now, here is a hot fix to be executed from your Umbrel root dir:
sed -i "s/bash/sh/g;" app-data/ride-the-lightning/rtl/entrypoint.sh && scripts/app stop ride-the-lightning && scripts/app start ride-the-lightning

But soon this fix will be included in a new release. This is only temporary fix.

9 Likes
Error: system service failed
Umbrel is still synchronising after update V0.4.2
Umbrel BTC and Lightning Wallets status stuck at "Synchronizing"
LND doesn't work
Lost password - reset node
How to restore your node wallet
How to create a Lightning Node Alias?
Problem: Umbrel disaster recovery
Introducing the plug-and-play Umbrel server from The Bitcoin Machines
Getting Started with your Umbrel
How to activate Wifi?
Please help with Thunderhub
Docker hub getting worse and worse
Umbrel is unavailable after few hours
Need help initial startup getting "failed to start docker application container engine"
Thread 'main' panicked at 'reader failed:
0.4.2, core 22.0, and the new LND update
LND `lncli unlock` command 'succeeds' but wallet isn't unlocked
Rebalancing command LNCLI?
Can't SSH into umbrel
Umbrel on Ubuntu Bare Metal Provider
Electrs on Umbrel isn’t responding to requests
Umbrel Password set up
Problem with 0.4.4 upgrade
Error to create address
Some Issue Installing Apps
Umbrel is unavailable after few hours
Stuck at "deleting previous images" updating 0.4.5
SSH - Unable to SSH into Node / Unable to unload wallets
Error: System service failed (RPi 8gb)
Blockchain won't fully sync after running latest update
Stuck at "deleting previous images" updating 0.4.5
Starting Umbrel forever
Error: System service failed (RPi 8gb)
Restored wallet - two channels won't close
Apps stuck on "Starting..."
Umbrel is not in the sudoers file
Apps stuck on "Starting..."
Booting problem: System service failed
Docker containers failing to start
Can't resolve hostname, Tor doesnt start
Umbrel not recognizing SSD
Receiving "Error: System service failed"
Cooperative closed channel pending for wk
Multiple nodes for school /educational purposes
Node stuck on loading screen
Problem with 0.4.4 upgrade
Umbrel UI stuck at loading
Make Bitcoin Core listen to LAN peer only
Node stuck on loading screen
Update SD card to latest umbrel version

Thanks! This guide saved me an heart attack. :grinning:

1 Like

when I input this in the terminal, it asked about a fingerprint and if i was sure i wanted to connect, I said yes and now I am no longer able to input any further commands, is there something I am missing or a different way to achieve this?

Yes, follow the instructions in the screen. It says what you have to do.