Full graph →

Networking

In system design, networking answers three questions. Can service A reach service B — reachability. Is B actually the service A meant to talk to — identity. Did the bytes arrive complete and unmodified — integrity. IP addressing and CIDR arithmetic are documented everywhere; what matters at the design level is how names, keys, and tunnels compose into those three guarantees.

This article grounds the concepts in the SFI internal network: a private .lofi domain, an internal certificate authority, and a WireGuard mesh that production servers use to pull deployment assets from internal services like Gitea — without any of it being exposed to the public internet.

Names Are Authority

A domain you control gives you two things: autonomy — services can move hosts, change addresses, or gain replicas without their consumers noticing, because the name is the stable contract and the address is an implementation detail — and authority — you operate the resolver, so you decide what every name means, with no external dependency and no leakage of your internal topology to public DNS.

SFI's internal services live under .lofi: gitea.zen.lofi, grafana.lofi, step-ca.lofi. The zone is served by a dnsmasq instance on the internal network. A private top-level domain never resolves on the public internet — that is both the feature (queries for internal names cannot leak meaningfully) and the cost (no public CA will ever issue a certificate for one, which is why the next section exists). If you are choosing a private domain today, .internal (reserved by ICANN for exactly this) and home.arpa (RFC 8375) are the collision-safe options.

The mechanism that makes a private domain coexist with the public internet is split DNS — the resolver is chosen by name suffix, not configured globally:

# systemd-resolved drop-in — installed by the wireguard playbook on cloud
# hosts that can't reach the resolver on the LAN; LAN hosts get it from DHCP
[Resolve]
DNS=10.1.0.1
Domains=~lofi

The ~lofi marks a routing domain: only queries under .lofi go to the internal resolver (whose address is on the mesh — more below); everything else follows the host's normal DNS. The Kubernetes equivalent forwards the zone from the cluster's own resolver:

# coredns-custom ConfigMap: cluster workloads resolve .lofi like any other name
data:
  lofi.server: |
    lofi:53 {
      cache 30
      forward . 10.0.0.2   # the internal dnsmasq
    }

Every consumer — laptops, VMs, pods — resolves gitea.zen.lofi the same way, and none of them knows or cares which host currently serves it.

Trust Is a Certificate Chain

Reachability without identity is a liability. The guarantee you actually need is: the service that answered is the one the name refers to, and the conversation was neither truncated nor tampered with. TLS provides authentication, integrity, and confidentiality in one handshake — but the authentication is only as good as the chain from the server's certificate up to a root you trust. Public CAs cannot anchor that chain for .lofi names, so the internal network runs its own root: a smallstep step-ca instance.

The design decision that keeps an internal CA from becoming a manual certificate bureaucracy: step-ca speaks ACME, the same protocol Let's Encrypt uses. Issuance and renewal are exactly as automated as they are on the public internet — the reverse proxy just points at a different directory URL:

{
    acme_ca      https://step-ca.lofi:9000/acme/acme/directory
    acme_ca_root /step-ca-data/certs/root_ca.crt
}

grafana.lofi {
    reverse_proxy 10.0.0.2:3000
}

Caddy obtains and renews a certificate for every .lofi site block automatically; no human ever handles a key or an expiry date.

What remains is trust distribution — the internal root certificate must land in every OS trust store, which is a first-boot provisioning concern rather than a networking one (see the automation-seams section of shell for the two-line step ca bootstrap step that fleet VMs run). Interactive devices — laptops, phones — get the root installed once per platform trust store.

Reachability Without Exposure

The remaining problem is the one VPNs solve, restated for infrastructure: production servers in the cloud must reach internal services — a deploy pulls assets from Gitea — but those services must not be exposed publicly. The SFI answer is a WireGuard mesh: every participating host holds a keypair and a mesh address (10.1.0.0/24), with the peer list rendered from a single source of truth (peers.json) by Ansible on VMs and Nix on workstations.

WireGuard's identity model is deliberately SSH-like rather than PKI-like: peers authenticate each other by static public keys exchanged out-of-band, and everything crossing the tunnel is encrypted and integrity-protected — identity and integrity at layer 3, before any application protocol speaks. Its distinctive idea is cryptokey routing: each peer entry's AllowedIPs is simultaneously a routing table entry and a firewall rule —

[Peer]
# soundship — internal DNS and reverse proxy
PublicKey = Epvbuaw...RD4=
AllowedIPs = 10.1.0.1/32
PersistentKeepalive = 25

Outbound, a packet destined for 10.1.0.1 is encrypted to that peer's key. Inbound, a decrypted packet whose source address is not in the sender's AllowedIPs is dropped. Route and ACL collapse into one declaration; there is no separate firewall state to drift out of sync.

Two operational conventions from the mesh worth stealing: only peers with a stable, reachable address declare an Endpoint (public DNS for cloud VMs, LAN IPs for always-on hosts); peers behind dynamic NAT declare none — their location is learned from the handshakes they initiate, and PersistentKeepalive = 25 keeps the NAT mapping pinned so the session survives silence. See wireguard for mesh operations and key management.

How a Production Server Reaches .lofi

The pieces above compose into a routing walk. A deploy step on db2.streetfortress.cloud — a VM in a public cloud — fetches from https://gitea.zen.lofi:

  1. systemd-resolved matches the ~lofi routing domain and sends the DNS query to 10.1.0.1 — and that query is itself the first packet to ride the mesh: the kernel finds 10.1.0.1 in wg0's AllowedIPs, encrypts to soundship's key, and ships it as UDP to soundship's endpoint.
  2. dnsmasq answers with every address it knows for the host — for the Gitea host, both the LAN and the mesh address (10.0.0.6 and 10.1.0.6). Which one gets dialed is decided by the client's source-address selection (RFC 6724): the kernel prefers the candidate sharing the longest prefix with one of its own addresses, and on a mesh-attached cloud VM that is the 10.1.0.x answer. Know this rule, because the failure mode when it misfires is ugly — a host that dials the LAN address it has no route to sends packets out its default gateway into a black hole and waits out the full connect timeout instead of failing fast.
  3. The TCP connection follows the same cryptokey route as the DNS query did. Gitea is listening on a private address; from the public internet it does not exist.
  4. TLS completes the picture: Gitea presents a certificate for gitea.zen.lofi chaining to the internal root that landed in the server's trust store at first boot.

The result is defense in depth with no operational overhead per connection: the mesh authenticates hosts by key at layer 3, TLS authenticates the service by name at layer 7, and a deploy running in a public cloud is indistinguishable from one running on the LAN.

Segmentation and Zero Trust

The mesh is a boundary, not a substitute for authorization. Segmentation still applies — a database should not share a segment with a public-facing web server, and AllowedIPs gives you per-peer granularity to enforce it — and the zero-trust posture still applies inside the tunnel: being on the mesh proves key possession, not permission. Every service continues to authenticate and authorize each request. For enforcement mechanics — firewall rules, security groups, Kubernetes network policies — see network-isolation; for the request-level identity layer, see authentication-authorization.

References