Use previously downloaded blockchain

Hi, and thanks for a great forum.

I decided to try to copy an already synced block chain from another node. However I ran into a little problem I hope you can help me with.

I did:

  1. Closed down the Pi4 via Umbrel webinterface after 15% sync.
  2. Moved disk to laptop and deleted “blocks” and “chainstate” folders and replaced them with “blocks” and “chainstate” folders from a freshly updated bitcoin core node.
  3. Mounted the HDD back into the Pi4 and booted up.

I looked in the logs and got this:

...
2021-05-10T18:57:00Z init message: Loading block index...
2021-05-10T18:57:00Z Switching active chainstate to Chainstate [ibd] @ height -1 (null)
2021-05-10T18:57:00Z boost::filesystem::status: Permission denied: "/data/.bitcoin/blocks/index"
2021-05-10T18:57:00Z : Error opening block database.
Please restart with -reindex or -reindex-chainstate to recover.
2021-05-10T18:57:00Z Aborted block database rebuild. Exiting.
2021-05-10T18:57:00Z Shutdown: In progress...
2021-05-10T18:57:00Z scheduler thread exit
2021-05-10T18:57:00Z Shutdown: done

I then added “reindex=1” to the bitcoin.conf file. It changed the log slightly to:

...
2021-05-11T06:50:04Z init message: Loading block index...
2021-05-11T06:50:04Z Switching active chainstate to Chainstate [ibd] @ height -1 (null)
2021-05-11T06:50:04Z Wiping LevelDB in /data/.bitcoin/blocks/index
2021-05-11T06:50:04Z boost::filesystem::status: Permission denied: "/data/.bitcoin/blocks/index"
2021-05-11T06:50:04Z Error: Error opening block database
2021-05-11T06:50:04Z Shutdown: In progress...
2021-05-11T06:50:04Z scheduler thread exit
2021-05-11T06:50:04Z Shutdown: done

I then ran: “sudo chown -R 1000:1000 bitcoin” standing in the umbrel directory.

Now logs look like this:

...
2021-05-11T08:11:33Z init message: Loading block index...
2021-05-11T08:11:33Z Switching active chainstate to Chainstate [ibd] @ height -1 (null)
2021-05-11T08:11:33Z Wiping LevelDB in /data/.bitcoin/blocks/index
2021-05-11T08:11:33Z boost::filesystem::create_directory: No error information: "/data/.bitcoin/blocks/index"
2021-05-11T08:11:33Z Error: Error opening block database
2021-05-11T08:11:33Z Shutdown: In progress...
2021-05-11T08:11:33Z scheduler thread exit
2021-05-11T08:11:33Z Shutdown: done

Anyone who knows how to go from here?

Thank - but still looks like it doesn’t work…

Docker Containers

NAMES                       STATUS
sphinx-relay                Up 11 seconds
specter-desktop_web_1       Up 22 seconds
btcpay-server_web_1         Up 37 seconds
btcpay-server_postgres_1    Up 47 seconds
btcpay-server_nbxplorer_1   Up 48 seconds
btc-rpc-explorer_web_1      Up About a minute
bluewallet_lndhub_1         Up About a minute
bluewallet_redis_1          Up About a minute
middleware                  Up About a minute
neutrino-switcher           Up About a minute
bitcoin                     Restarting (1) 22 seconds ago
nginx                       Up About a minute
lnd                         Up About a minute
manager                     Up About a minute
dashboard                   Up About a minute
electrs                     Up About a minute
tor                         Up About a minute

Bitcoin Core logs
-----------------

Attaching to bitcoin
bitcoin              | 2021-05-11T09:36:25Z Using at most 125 automatic connections (1048576 file descriptors available)
bitcoin              | 2021-05-11T09:36:25Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
bitcoin              | 2021-05-11T09:36:25Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
bitcoin              | 2021-05-11T09:36:25Z Script verification uses 3 additional threads
bitcoin              | 2021-05-11T09:36:25Z scheduler thread start
bitcoin              | 2021-05-11T09:36:25Z HTTP: creating work queue of depth 16
bitcoin              | 2021-05-11T09:36:25Z Using random cookie authentication.
bitcoin              | 2021-05-11T09:36:25Z Generated RPC authentication cookie /data/.bitcoin/.cookie
bitcoin              | 2021-05-11T09:36:25Z Using rpcauth authentication.
bitcoin              | 2021-05-11T09:36:25Z HTTP: starting 4 worker threads
bitcoin              | 2021-05-11T09:36:25Z Using wallet directory /data/.bitcoin
bitcoin              | 2021-05-11T09:36:25Z init message: Verifying wallet(s)...
bitcoin              | 2021-05-11T09:36:25Z init message: Loading banlist...
bitcoin              | 2021-05-11T09:36:25Z SetNetworkActive: true
bitcoin              | 2021-05-11T09:36:25Z Using /16 prefix for IP bucketing
bitcoin              | 2021-05-11T09:36:25Z Cache configuration:
bitcoin              | 2021-05-11T09:36:25Z * Using 2.0 MiB for block index database
bitcoin              | 2021-05-11T09:36:25Z * Using 24.8 MiB for transaction index database
bitcoin              | 2021-05-11T09:36:25Z * Using 21.7 MiB for basic block filter index database
bitcoin              | 2021-05-11T09:36:25Z * Using 8.0 MiB for chain state database
bitcoin              | 2021-05-11T09:36:25Z * Using 143.6 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
bitcoin              | 2021-05-11T09:36:25Z init message: Loading block index...
bitcoin              | 2021-05-11T09:36:25Z Switching active chainstate to Chainstate [ibd] @ height -1 (null)
bitcoin              | 2021-05-11T09:36:25Z Wiping LevelDB in /data/.bitcoin/blocks/index
bitcoin              | 2021-05-11T09:36:26Z boost::filesystem::create_directory: No error information: "/data/.bitcoin/blocks/index"
bitcoin              | 2021-05-11T09:36:26Z Error: Error opening block database
bitcoin              | Error: Error opening block database
bitcoin              | 2021-05-11T09:36:26Z Shutdown: In progress...
bitcoin              | 2021-05-11T09:36:26Z scheduler thread exit
bitcoin              | 2021-05-11T09:36:26Z Shutdown: done

I just did a copy of my fully synched blockchain and it worked perfectly. So documenting the steps for others.

  1. Stop umbrel using script
  2. go to bitcoin folder and delete folders (indexes, blocks, chainstate)
  3. Stop fully synched blockchain
  4. Copy the 3 folders (indexes, blocks, chainstate) from a fully synced blockchain
  5. Edit bitcoin.conf and add reindex=1 in the #indexes section
  6. Start umbrel using script
  7. Wait for reindex to complete… it took me around 3 hours.
  8. Confirm the UI is showing 100% synchornized and bitcoin wallet “Active”
  9. edit bitcoin.conf to comment reindex=1 line.

That looks like a brilliant solution as far as I can understand it. Being a totally Pi newbie if you are still there visybl can I ask a really basic question: How do you copy the set of indexes, blocks, chainstate over from one SSD drive to the other, can you plug both into the same Pi and if so what instructions do you use to copy the files over? Sorry I realise this is a shockingly basic question to ask. The reason I want to understand fully is that the entire family want to have their own nodes, which would take a very long time if we had to download new blockchains for each.

read the OP, it says right there how.

Thanks DarthCoin. I guess ‘Moved disk to laptop’ covers it. I’ll try a PC and if that can’t read the SSD I’ll try a Mac.

In the troubleshooting manual I wrote very specific steps and how to do it, if you use a windows or other OS.

i’m not using a rasberrypi. I’m running umbrel on linux mint and trying to do this. I’m a very novice user of linux as well as umbrel, so I don’t know the commands to enter into the terminal window. I’ve tried to just launch umbrel and have it sync the full blockchain, but it constantly crashes mid-way through or stalls on “synching”

I’m able to run bitcoin core software no problem on this same linux machine and have a fully synched chainstate, blocks, and index file for that. I just want to understand the proper way to bring this over to umbrel and have umbrel run the node on that same machine. Would appreciate some more detailed instructions on this that are specific to running umbrel on linux. Thanks!

Hi @jackietreehorn
Just to confirm, are you trying to do a fresh install or transfer your existing downloaded chainstate to your Umbrel bitcoin node?

1 Like

I’m actually trying to do both (fresh umbrel install AND xfr existing downloaded chainstate to the new/freshly installed Umbrel node). But I managed to get this figured out. I have a downloaded blocks/chainstate from a Bitcoin node I was running bitcoin core software natively in linux mint previously. And now wanted to install Umbrel on that same linux mint computer and just migrate the blocks/chainstate folder to Umbrel and save all the hassle, time, stalling that was happening in re-downloading the whole thing again.

In case anyone else is trying to do the same thing, the way I did this was to:

  1. Install Umbrel fresh on Linux
  2. Login to new Umbrel node, launch browser umbrel interface, install Bitcoin app from app store, launch app. It will start to sync
  3. Log out of umbrel node and close the browser window (don’t shut linux machine down)
  4. Open Terminal from home/umbrel$
  5. Stop umbrel from command line, sudo ./scripts/stop
  6. Navigate to Umbrel/app-data/bitcoin/data/bitcoin and delete the “blocks” and “chainstate” folders
  7. Replace with fully-sync’d blocks and chainstate folders from bitcoincore software folder that was running natively before Umbrel
  8. Edit bitcoin.conf file in app-data/bitcoin/data/bitcoin/umbrel-bitcoin.conf to add reindex=1 in the #indexes section
  9. Open Terminal from home/umbrel$ sudo ./scripts/start to restart the node
  10. Open browser and re-login to Umbrel and open Bitcoin app. It should start the reindexing process and should not have to sync but only a few blocks to get up to date.
1 Like