Private WAN to connect multiple sites using WireGuard

Hello all!

Today I wanted to share how I made my network, connecting together my base LAN at my flat together with other devices outside it and other sites, or virtual locations.

• First part (green) is my base network at my flat, using 192.168.1.0/24 addressation:
  • Central point of everything is my main Netgear R6220 router, running OpenWRT. Except for basing routing and switching it runs also few other services, like SMB Samba server sharing attached USB drive, mjpeg server allowing for streaming video from attached webcam for survilance when I’m out and need coverage of blind spots of my main security camera, and, most importantly WireGuard “server”. Technically it’s peer-to-peer type connection without any “servers”, and “clients”, but due to it’s purpose as a central point to which all other WireGuard “clients” connect I’ll be calling it that for simplicity. I decided on 10.0.0.0/24 network for WireGuard network to clearly distinguish it from other local networks.
    It serves as a central hub for everything, all WireGuard traffic comes here and is routed to it’s destination – if it’s inside this LAN then directly to it, if it’s targeting any other network then it’s routed back to WireGuard to this network’s VPN gate address.
  • D-Link NAS server, serving files using SMB over local network, and with some limited FTP shares over internet if needed,
  • Multiple Windows PCs, including my main workstation, that can also be used as gaming server through Nvidia GameStream and Moonlight clients, all other PCs and laptops, private and work one when working form home, etc. all within 192.168.1.0/24 network,
  • Windows Server 2019 machine serving as IIS web server (with, for example this website), MySQL server for WordPress sites, MS SQL Server for learning and developing and private projects, hMailserver for my emails, PLEX server for some media access stored on my NAS server etc.,
  • Some other stuff like IP Security camera, Andorid TV, Printer, small unmanaged Switch as I ran out of ports in main router, etc.
  • In future Rover’s back-end server, doing 3d mapping from stereo images, object detection etc., will also be here, although it probably will be ran just on my Workstation, as it has most computing power out of mine computers right now,
• Second part (light blue) consists technically of two networks, 192.168.111.0/24 as main part, and additional 192.168.8.0/24 network made by LTE router. It’s main purpose is being onboard rover network, but if needed I can easily take it with me to serve as “mobile site” when traveling etc., could be used with for example hotel/train WiFi as a way of securing connection, routing all traffic through WireGuard to my secure network at home instead of local one only etc.
  • Central device here is TP-Link TL-WR902AC mobile router. It serves as router for 192.168.111.0/24 network, connecting all devices onboard my Rover, it connects as a client to LTE router for internet connectivity and has WireGuard working as a “client”, serving as a gateway for all local traffic between this and other LANs. Traffic from/to 192.168.111.0/24 and 192.168.8.0/24 networks is routed through this tunnel, allowing secure communication between all parts of my network. It’s powered over USB so can be easily powered from Rover’s onboard 5V rail.
  • Huwawei mobile LTE router, serving as Internet gateway for this network. It could of course be done using just one device with inbuild modem, or USB LTE modem, but that’s just what I had on stock from few years ago, before I had phone with two SIM slots and this was my main mobile internet source. And it works fine, even if it adds a bit more complexity, it’s powered over USB and also hass inbuild battery, so can also be easily powered from Rover 5V rail. All other devices connect using 192.168.111.0/24 to main TP-Link router and it is in turn connecting to this router over 192.168.8.0/24 network as a client for internet connectivity.
  • All other devices that need to be connected to my network in this site. In base scenario onboard my Rover it’ll be main Meadow F7 board, maybe later some secondary boards too, and multiple ESP-32CAM modules, sharing video streams. Maybe some other wireless enabled modules later. In other scenarios, like traveling router – other devices as needed.
  • WireGuard here, like in all other devices, is configured to use split-tunneling – only local traffic is routed to it and then it’s routed to LTE Router and out to internet and to WireGuard “server” for further routing. All other, direct-to-internet traffic is routed straight to LTE router and there to internet, omitting WG tunnel. In basic, Rover scenario, only device connecting to non-local resources will be Meadow board connecting to Azure IoT Hub, and that connection is secured on it’s own, so there’s no need to add another layer of encryption and route it through my home network. In traveling-router scenario, connecting to unsecured network, it can be easily configured to run all traffic through WG tunnel to my flat, so it’s secured locally from any potential eavesdropping on unsecured local network, like hotel or train one, and enters internet only over my trusted connection at my flat.
• Third part (not on map, as it’s not concerned with Rover project) is/will be at my family home. Right now only parents’ Laptop has WireGuard client, again configured with split tunneling, so they can easily access internet “traditional” way, at the same allowing me to connect to it when they need some help, or share them some files securely over VPN. In the future I’m considering adding there another OpenWRT-with-Wireguard router, so on one hand I could get easy control over whole network, on the other it would allow other devices to connect to my resources, for example Android TV to connect to PLEX server at my flat.
• Fourth part, pink one, is the one I’m using right now, writing this text on my Surface Pro siting on a train, using hot-spot from my phone to connect to internet. This technically is not and will not be “site”, as each device has it’s own WireGuard “client” and it’s pure point-to-site connection from both laptop, and phone, and both are configured with split-tunneling to allow easy “typical” internet access and secure access to resources in my flat and other parts of my network. Each device having it’s own WG “client” makes everything much easier, no need for any additional routing (I’m not even sure if I’d be able to configure WireGuard on Android to route traffic from other devices…). If I’m on my laptop I’m just enabling tunnel here and I can access my local files, play games remotely on my main Workstation (Moonlight is great!) etc., same on phone, or when I need to manage servers at my home, print something quickly so I have it ready when I’ll get back and not forget etc.
  And, of course, as main point of this whole architecture, I can easily use my Surface as controller for my Rover over the internet!
• Other parts on the map are still in planning stage:
  • As mentioned earlier Azure IoT Hub, that will be used for managing settings of the Rover, but not direct control over it. That way I won’t need to clutter control app with many settings that won’t be needed on daily basis, ranging from changing some delays in code, to for example gimbal probing frequency etc. It will be connected directly by Meadow F7 board, ommiting WireGuard tunnel, as those connections are encrypted by default.
  • Azure back-end servers. My i7-7700K workstation, even tough still quite powerful, is quite limited with it’s Kaby Lake architecture and only 4 cores, and fast image processing if of course computing power hungry. Compared to that on Azure I can run even 96 vcore monsters for around 1 euro per hour (or, if I will learn how to use CUDA in my apps one day, some GPU-based monsters for similarly low prices). So when developing and testing I can use just my workstation, but when I’m planning longer sessions, or “going on adventure”, I can then quickly spin up such powerful server, connect it to my network using WireGuard, and, from Rover’s perspective only difference will be using different IP address and much faster responses!

Just to test if all of this can even work I made simple test yesterday. On Meadow board I put simple test program that connects board to TP-Link router and awaits connection. My Surface was connected using my phone’s hot spot. This simulates scenario where I’m in the field and Rover is beyond WiFi range from me, so I can control it only over internet.

• 

So what’s going on on that screen?

• First, Meadow connects to it’s dedicated WiFi network. We see “Connection request completed” in Visual Studio console output.
• In test application (left console windows) I provided IP of Meadow. As you can see on the right, Surface is connected to phone hot-spot and also WireGuard client and it’s different network that Meadow one (and different than my home LAN, as mentioned above it’s 192.168.1.0/24).
• In test application UDPClient class instance is connecting to provided client (Meadow) end-point, then it’s sending message to it.
• Because target IP is local one, message is sent to WireGuard interface. There it gets encrypted and sent through internet to public IP of my home router.
• At my home router message gets decrypted. Router checks target IP, then check there is static route defined pointing to WireGuard address, so it sends it there. As the second router already connected do main router it already knows is public IP and can send encrypted again packets to it over internet.
• Packets arrive through LTE router and are sent to second router. Here they get decrypted again. Router checks target IP, it’s in its network so sends the message to Meadow.
• On Meadow UDPClient is already awaiting connection. When it receives first message it checks sender IP and connects to it, so from now on it will accept only data from that source. It also prints sender endpoint to console, as we can see on bottom. Because my Surface’s WireGuard works now in point-to-site mode, it’s IP is shown as WG IP – 10.0.0.2. Then Meadow sends back message “Connected!” as we can see in left console and enters recive-and-respond loop.
• Upon receiving next message Meadow uses HttpClient class to request web page from my webserver at my flat. This simulates communication with back-end server. After receiving data back it prints first 100 chars to console, to show it worked and also sends them back to test app, which prints received message too.

Whole code for this can be found here, although it’s just test repo and work in progress so it’s not pretty, but working: https://github.com/SebaWnek/WiFi-test

That way we see everything works correctly, I was able to communicate 3 devices each in different physical location. In theory each of them could be in any part of the world, as they are interconnected using VPN tunnels over the internet. That way I can control my rover literary anywhere, as long as there is cellular network there!

Is it slow? Let’s check:

• 

• 

First test was made as above. Second one – with both Meadow and Surface connected to the same router. 150ms difference is not that bad in my opinion, taking into account how long path was.

Is this hard to configure? Not really!

It definietely requires some basic networking knowledge and also at least basic knowledge of OpenWRT. Fortunately everything can be even done using just GUI on OpenWRT, there are WireGuard applications for Windows, Android and probably any other system and WG itself is really easy to configure, especially compared to things like OpenVPN. Only extra bits of work is creating correct firewall rules and static routes on routers working in site-to-site mode so the traffic will be correctly routed between different networks, especially in hub router, but that requires only basic networking knowledge.

Thanks for reading and if you have any suggestions or questions feel free to leave a comment!