LND: Tor & Clearnet - How to setup hybrid-mode

This is a copy of the article we published freely available on Github & LN+.
If this is helping you, we appreciate a small gift via LNURL in the footer below - it’s 50:50 shared between Osito and myself.
We’ve also added Y’all links in case this is more for you

Now, let’s get into it :rocket:

Prelude and Objective

The lightning network functions in rapid growing speed as infrastructure for payments across the globe between merchants, creators, consumers, institutions and investors alike. Hence the key pillars of sustained growth are their nodes, by providing reliable, liquid, discoverable and trustless connection points between those parties.

While a growing number of nodes come online every day, a significant share of those are using Tor, which allows them to remain anonymous in a sense that they don’t need to reveal their real IP address (aka Clearnet IP). However, this methodology paired with the increased demand for Bitcoin payments will continue to stretch Tor’s capacity to cater for continued need of supply. It also hampers existing and new node’s metric of success being a reliable and fast peering partner.

To mitigate some of ongoing Tor capacity constraints, a node runner may choose to reconsider (see Chapter Caution: Clearnet! offering both, a Tor as well as a Clearnet IP connection option. Next to the drawbacks outlined in the first section below, it has three main net benefits

  1. allows for alternative discovery, routing and peering in case your own Tor cluster is affected by capacity constraints. Even though mostly temporarily, it cuts into your reliability
  2. provides quicker routing of HTLCs, both for payment and probing. Quite nervous waiting 8 seconds for your transfer confirmation at the supermarket or bar, isn’t it?
  3. offers other clearnet-only nodes to connect directly to you. Otherwise it would be required for you as Tor-only to peer-connect to them first, before they can open a channel

With those considerations in mind, have a careful read through the words of caution below, make an educated decision by yourself, and then use our guide below on how to enable a hybrid Tor & Clearnet Node.

Table of Content

  • Prelude and Objective
  • Caution clearnet!
  • Preconditions
  • Configuring hybrid-mode
    • Static IP
    • Dynamic IP: Solution 1 - NAT/UPnP
    • Dynamic IP: Solution 2 - Dynamic DNS (DDNS)
  • Wrap-Up
  • Bonus Special Case: Clearnet through VPN

Caution: Clearnet!

A word of caution: Running a node behind the Tor network offers many advantages (anonymity, security and usability) and therefore it is currently the most recommended way. For nodes maintaining a high number of connected channels and/or have high availability requirements, Tor can be a hindrance. Tor’s organic network is prone to censorship of a country’s law regulation and internal failures of circuits and relays. LND also allows running clearnet nodes that do not make use of the Tor network but directly connect to peers. This requires node administrators to take care of the underlying system’s security policies. At least one port (default: 9735) needs to be forwarded and exposed for remote peers to connect to. Setting up a firewall is highly recommended. Not only security is a topic to be thought about, also the risk of being localized by clearnet IP. Only use hybrid-mode if privacy is not of concern!

Preconditions:

For this guide the following is required:

  • You are tech-savvy and know what you do
  • A fully installed and synchronized node (Umbrel / custom)
  • For RaspiBlitz these features will be implemented and available in Release v1.8.
  • lnd-0.14.1-beta or later
  • tor.streamisolation=false must be turned off when using hybrid-mode :warning:

Hybrid-mode was brought to life in LND by Lightning Labs in version lnd-0.14.0-beta. A new option was introduced to split connectivity and to separately address Tor-only peers via Tor and clearnet peers via clearnet:

[tor]

; Allow the node to connect to non-onion services directly via clearnet. This
; allows the node operator to use direct connections to peers not running behind
; Tor, thus allowing lower latency and better connection stability.
; WARNING: This option will reveal the source IP address of the node, and should
; be used only if privacy is not a concern.

tor.skip-proxy-for-clearnet-targets=true

Configuring hybrid-mode:

Advertising clearnet connectivity LND needs to know the external IP of a node. There are two different cases to investigate: static and dynamic IP connections.

A static IP is rather easy to set in LND. An obvious pre-requisite is, your ISP provides an IPv4, alternatively an IPv6 address to your connection. This external IP address has to be applied to LND’s option externalip. That’s almost it. But most internet providers change IPs on a regular basis or at least on reconnection. Therefore externalip in lnd.conf would have to be changed accordingly each time a new IP was assigned, followed by a restart of lnd.service to reload lnd.conf. This is unsustainable for continuous node running. Two possible solutions to prevent re-editing and restarting LND:

  • Solution 1: NAT/UPnP
  • Solution 2: Dynamic DNS (DDNS)

Static IP:

Static IPs are rarely provided for home use internet connections. It is a feature mostly offered to cable or business connections. Having a static IP makes configuring of lnd.conf much easier. In this case option externalip needs a closer look.

; Adding an external IP will advertise your node to the network. This signals
; that your node is available to accept incoming channels. If you don't wish to
; advertise your node, this value doesn't need to be set. Unless specified
; (with host:port notation), the default port (9735) will be added to the
; address.
; externalip=

Dynamic IP: Solution 1 - NAT/UPnP:

Dealing with dynamic IPs tends to be a bit more complex. LND provides an integrated approach to this: NAT. NAT tries to resolve dynamic IPs utilising built-in techniques in order to fetch a node’s external IP address. Notable that LND doesn’t handle the setting of externalip and nat at the same time well. Choose only one of them, based on your router’s UPnP capabilities (nat traversal).

; Instead of explicitly stating your external IP address, you can also enable
; UPnP or NAT-PMP support on the daemon. Both techniques will be tried and
; require proper hardware support. In order to detect this hardware support,
; `lnd` uses a dependency that retrieves the router's gateway address by using
; different built-in binaries in each platform. Therefore, it is possible that
; we are unable to detect the hardware and `lnd` will exit with an error
; indicating this. This option will automatically retrieve your external IP
; address, even after it has changed in the case of dynamic IPs, and advertise
; it to the network using the ports the daemon is listening on. This does not
; support devices behind multiple NATs.
; nat=true

Dynamic IP: Solution 2 - Dynamic DNS (DDNS):

Dynamic DNS (DDNS) is a method of automatically updating a name server in the Domain Name System (DNS), often in real time, with the active DDNS configuration of its configured hostnames, addresses or other information. (src)

A script or an app regularly fetches the client’s current IP address which is saved for later requests. LND is able to resolve a given domain / DDNS to the actual IP address as well. Log output of HostAnnouncer listed below:

[DBG] NANN: HostAnnouncer checking for any IP changes...
[DBG] NANN: No IP changes detected for hosts: [ln.example.com]
...
[DBG] NANN: HostAnnouncer checking for any IP changes...
[DBG] NANN: IP change detected! ln.example.com:9735: 111.11.11.11:9735 -> 222.22.22.22:9735

In this case lnd.conf needs to know a reserved DNS domain instead of an external IP. Option externalhosts has to be set:

[Application Options]
# specify DDNS domain (port optional)
externalhosts=ln.example.com:9735

Lightning explorers like 1ml.com and amboss.space show and use IP addresses only. The node itself also only makes use of the resolved IP addresses (see lncli getinfo). Domains can be some fancy giveaway for peering invitations on chat groups or printed on business cards … who knows what it might be good for in the future.

Wrap-Up:

Summing up the introduced LND options in this article, here are some examples of complete configurations:

Static IP:

[Application Options]
# specify an external IP address e.g. 222.22.22.22:9735 / [2002::de16:1616]:9736
externalip=222.22.22.22:9735
# externalip=[2002::de16:1616]:9736
# specify an interface (IPv4/IPv6) and port (default 9735) to listen on
# listen on IPv4 interface or listen=[::1]:9736 on IPv6 interface
listen=0.0.0.0:9735
# listen=[::1]:9736

[tor]
tor.active=true
tor.v3=true
# deactivate streamisolation for hybrid-mode
tor.streamisolation=false
# activate split connectivity
tor.skip-proxy-for-clearnet-targets=true

Dynamic IP - NAT:

[Application Options]
# specify an interface (IPv4/IPv6) and port (default 9735) to listen on
# listen on IPv4 interface or listen=[::1]:9736 on IPv6 interface
listen=0.0.0.0:9735 
# listen=[::1]:9736
nat=true

[tor]
tor.active=true
tor.v3=true
# deactivate streamisolation for hybrid-mode
tor.streamisolation=false
# activate split connectivity
tor.skip-proxy-for-clearnet-targets=true

Dynamic IP - DDNS:

[Application Options]
# specify an interface (IPv4/IPv6) and port (default 9735) to listen on
# listen on IPv4 interface or listen=[::1]:9736 on IPv6 interface
listen=0.0.0.0:9735
# listen=[::1]:9736
externalhosts=ln.example.com:9735

[tor]
tor.active=true
tor.v3=true
# deactivate streamisolation for hybrid-mode
tor.streamisolation=false
# activate split connectivity
tor.skip-proxy-for-clearnet-targets=true

After restarting LND, it is now offering two (or three with IPv6) addresses (URIs). These can be verified by typing lncli getinfo:

"uris": [
        "<pubkey>@<onion-address>.onion:9735",
        "<pubkey>@222.22.22.22:9735",
        "<pubkey>@[2002::de16:1616]:9736"
    ],
Special Case: VPN Setup

Clearnet over VPN

To prevent exposure of a node’s real IP address connecting through VPN is an approach if anonymity is crucial. To achieve this, some preconditions must be checked and met:

  • :white_check_mark: VPN server or provider is able to forward ports.
  • :white_check_mark: VPN setup is able to split-tunnel processes (killswitch).
  • :white_check_mark: Home setup is able to forward specific ports (router/modem).
  • :white_check_mark: Home setup is able to allow incoming traffic (firewall).

Check? Let’s go!

  1. Declarations
internal_port = Internal LND listening port (for easy setup: internal_port = port-forwarded-VPN_port, but does not necessarily have to be)
port_forwarded_VPN_port = VPN assigned forwarding port
static_VPN_IP = IP of your VPN service/provider
ddns_domain = DDNS (DNS domain) for IP resolution
  1. Firewall: allowing incoming port
sudo ufw allow <internal_port> comment 'lnd-vpn-port'
sudo ufw reload
  1. Router/Modem: forwarding / mapping internal port to VPN assigned port (check first if it is necessary)

This step is managed very individually due to high amount of routers and modems out there. Usually GUI-based webinterfaces let define ports to be forwarded to specific devices within a local network.

  1. LND: configuring lnd.conf for VPN setup:
  • If VPN provides a static IP:
...
[Application Options]
externalip=<static_VPN_IP>[:<port_forwarded_VPN_port>]
# listen on IPv4 interface
listen=0.0.0.0:<internal_port>
# listen on IPv6 interface, if used
# listen=[::1]:<internal_port2> 

[tor]
tor.streamisolation=false
tor.skip-proxy-for-clearnet-targets=true
...
  • If VPN provides dynamic IPs and a DDNS was claimed:
...
[Application Options]
externalhosts=<ddns_domain>[:<port_forwarded_VPN_port>]
# listen on IPv4 interface
listen=0.0.0.0:<internal_port>
# listen on IPv6 interface, if used
#listen=[::1]:<internal_port2>

[tor]
tor.streamisolation=false
tor.skip-proxy-for-clearnet-targets=true
...

Note: Internal port and assigned VPN port are not necessarily the same. A router/modem may be configured to map any internal to any external port.

  1. VPN: Configure VPN connection and check port reachability

Set up a VPN connection with whatever your VPN provider recommends (individual step). Check if the opened port is reachable from the outside by running nc (on Linux) and ping from the internet e.g. with dnstools.ch.

1. run: nc -l -p 9999 (9999 is port_forwarded_VPN_port)
2. ping port 9999 from the internet
  1. Killswitch (depends on VPN client): Exclude Tor process from VPN traffic by VPN client or UFW

Most VPNs route all traffic through their network to protect against data leakage. In this case Tor traffic should be excluded from the VPN network because it is anonymized per se plus we want to add redundancy of connectivity and make use of lower clearnet responding times for faster htlc processing. Killswitch can be applied using UFW as well. To do so, please follow this guide.If your VPN client supports command line input, excluding the Tor process could be handled like this:

pgrep -x tor // returns pid of tor process
<vpn cli split-tunnel command> pid add $(pgrep -x tor) // optional step: if VPN provider supports CLI this step can be automated in a script, e.g. after Tor or node restart
  1. Restart LND and watch logs for errors (adjust to your setup)
tail -f ~/.lnd/logs/bitcoin/mainnet/lnd.log
  1. Lookup node addresses:

If everything is set, two URI addresses will be displayed.

$ lncli getinfo

"uris": [
        "<pubkey>@<onion-address>.onion:9735",
        "<pubkey>@222.22.22.22:9999"
    ],

Alternatively check listening ports with netstat:

netstat -tulpen | grep lnd

Result:

tcp6       0      0 :::9999                :::*                    LISTEN      1000       11111111   1111111/lnd
  1. Check connectivity with clearnet peers

To test clearnet connectivity find and ask other clearnet peers to connect to your node, e.g.: lncli connect <pubkey>@222.22.22.22:9999 Successful connection:

lncli connect <pubkey>@222.22.22.22:9999
{

}

Written by osito, Co-Authored & Reviewed by Hakuna.

If this guide was of help and you want to share some :heart: and contribution, please feel free to send a :zap: tip to our :zap: addresses: 0x382f9cf667447bb8@ln.tips (osito) | hakuna@btcadresse.de (Hakuna / HODLmeTight) or send some sats via LNURL


Also we are always grateful for incoming channels to our nodes: osito: HodL⚡NodL & Hakuna: HODLmeTight

6 Likes

Thank you for the guide. I’ve setup hybrid mode but had to use DDNS as NAT wouldn’t work for some reason. I haven’t been able to get IPv6 working for hybrid mode, one small mistake is that [::1]:port is the loopback address, it should be [::]:port. I tried using ports 9735 and 9736 but it only shows IPv4 and onion address in lncli getinfo. Port 9735 on IPv4 and IPv6 broke connectivity with lnd logs complaining [::]:9735 couldn’t connect as the port is in use. [::]:9736 works and lnd doesn’t continuously restart, however, no IPv6. I have a native dual-stack IPv4 & IPv6 service from my ISP and all my devices work fine in dual-stack, including the Pi 4 host. What else could be causing this? Anything else I should try? I enabled IPv6 in docker daemon based on docker documentation.

Did you forward the port 9735 in your router to point to your node local IP?

Both ports 9735 & 9736 are forwarded

Did you also open them in ufw?
Check on https://ping.eu if respond those ports

Even when not in ufw, it should not prevent Lnd from starting I think.
I find IPv6 very cumbersome, and barely any nodes use it. Hence I removed it completely from mine. Parked for another day maybe

I noticed that when setting extrernalip to a domain name (mynode.myext:9735), the Umbrel dashboard still uses the ip of mynode.myext. How can I setup a domain name instead of the IP?

As far as I know, Lnd will always resolve to the IP

But just to be sure, for domain name, you need to follow step 2, by setting
externalhosts:9735=ln.example.com
and not externalip

I managed to go hybrid via Dynamic-IP - DDNS using this tutorial. Thanks @Hakuna!

One question remains:

Can I use both addresses (IPv4 on port 9735 & IPv6 on port 9736) at the same time?

No, all addresses must listen on same port 9735

Only when you have another LN node in the same LAN you have to separate on a different port.

It’s not possible to set 0.0.0.0:9735 and [::]:9735 at the same time. LND goes into a restart loop.

lnd | unable to create server: listen tcp [::]:9735: bind: address already in use
lnd | [INF] LTND: Version: 0.14.2-beta commit=v0.14.2-beta, build=production, logging=default, debuglevel=info
lnd | [INF] LTND: Active chain: Bitcoin (network=mainnet)
lnd | [INF] RPCS: RPC server listening on 0.0.0.0:10009
lnd | [INF] RPCS: gRPC proxy started at 0.0.0.0:8080
lnd | [INF] LTND: Opening the main database, this might take a few minutes…
lnd | [INF] LTND: Opening bbolt database, sync_freelist=false, auto_compact=false
lnd | [INF] LTND: Creating local graph and channel state DB instances
lnd | [INF] CHDB: Checking for schema update: latest_version=24, db_version=24
lnd | [INF] LTND: Database(s) now open (time_to_open=4.056325251s)!
lnd | [INF] LTND: We’re not running within systemd
lnd | [INF] LTND: Waiting for wallet encryption password. Use lncli create to create a wallet, lncli unlock to unlock an existing wallet, or lncli changepassword to change the password of an existing wallet and unlock it.
lnd | [ERR] RPCS: [/lnrpc.Lightning/SubscribeInvoices]: wallet locked, unlock it to enable full RPC access
lnd | [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
lnd | [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
lnd | [ERR] RPCS: [/lnrpc.Lightning/ListChannels]: wallet locked, unlock it to enable full RPC access
lnd | [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
lnd | [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
lnd | [INF] LNWL: Opened wallet
lnd | [INF] CHRE: Primary chain is set to: bitcoin
lnd | [INF] CHRE: Initializing bitcoind backed fee estimator in CONSERVATIVE mode
lnd | [INF] LNWL: Started listening for bitcoind transaction notifications via ZMQ on 10.21.21.8:28333
lnd | [INF] LNWL: Started listening for bitcoind block notifications via ZMQ on 10.21.21.8:28332
lnd | [INF] LNWL: The wallet has been unlocked without a time limit
lnd | [INF] CHRE: LightningWallet opened
lnd | [INF] SRVR: Onion services are accessible via Tor! NOTE: Traffic to clearnet services is not routed via Tor.
lnd | [INF] TORC: Starting tor controller
lnd | [ERR] LTND: Shutting down because error in main method: unable to create server: listen tcp [::]:9735: bind: address already in use
lnd | [INF] TORC: Stopping tor controller
lnd | [ERR] TORC: DEL_ONION got error: invalid arguments: unexpected code
lnd | [ERR] LTND: error stopping tor controller: invalid arguments: unexpected code
lnd | [INF] LTND: Shutdown complete

But you are doing it wrong!
0.0.0.0 means ALL IPs !
You must put your node IP not ALL in existence!
Guys, learn the basics of networking.

I was referring to the listen= address not externalip= address based on your answer about listening address. It would be pointless to have lnd listen to your own IP address, you should know this with your networking genius.

You seriously need to stop with your insults.

Try to specify a separate port:

listen=0.0.0.0:9735
listen=[::]:9736

But what do I do here?

externalhosts:9735=ln.example.com

and/or

externalhosts:9736=ln.example.com

I am still not shure the hybrid fully works for me. Reaction time from amboss.space is still 2-3 days…

it should be:
externalhosts=ln.example.com:9735
It appears that setting port 9736 is unnecessary and doesn’t do anything. I just tried

listen=0.0.0.0:9735
listen=[::]:9736
externalhosts=lnd.mydomain.com:9735
externalhosts=lnd.mydomain.com:9736

Output of lndcli getinfo is

“uris”: [ “029be4825c37225ac1404d2a02a8341c389552f9de2e658762ba1ded582d78bfa6@onion_address:9735”,
“029be4825c37225ac1404d2a02a8341c389552f9de2e658762ba1ded582d78bfa6@IPv4:9735”,
“029be4825c37225ac1404d2a02a8341c389552f9de2e658762ba1ded582d78bfa6@IPv4:9736”
],

It’s using my IPv4 address to listen on ports 9735 and 9736, but still not picking up IPv6 even though I have a AAAA DNS record for lnd.mydomain.com

Hi @blckbx,

Please see my reply to @mr.cook above

Sorry to ask, but I get this error when I try lndcli getinfo:

image

That’s because lncli is not in your PATH, it’s in ~/umbrel/bin. If you type echo $PATH you will see paths separated by colons that bash will search within, left to right until it finds the command and executes that binary.

You can permanently add lncli and bitcoin-cli to your path by adding the following line to ~/.profile signing out and back into your Umbrel.
PATH="$PATH:$HOME/umbrel/bin"

or to run the command once, type out the full path to the bin file:
~/umbrel/bin/lncli getinfo

I hope that made sense.

1 Like

Thanks @Aydo, that makes perfectly sense. :slightly_smiling_face:

1 Like