• Web Infrastructure: Uxtly runs on two servers with three jails each. Jails communicate privately, and reverse proxy jails listen on the internet. Each server has two "virtual switches" (if_bridge) - ibridge for database replication and xbridge for passing traffic. It also has two network interfaces - an internal inic connected to ibridge and an external xnic redirected through xbridge. There are seven virtual cables with VNICs at either end (epair). Additionally, each server has four encrypted tunnels (spiped).
• Load Balancing: In the DNS provider, there's an 'A' record per server for load balancing by a round-robin algorithm. To shut down a server, remove its 'A' record and wait for the TTL to expire (default 5 minutes in Cloudflare). For targeting a particular server by IP, see the health-checking post.
• Configurations: Configuration files (*.conf) are the same on both servers. They read server-specific settings from other files. /etc/rc.conf specifies how to create and interconnect bridges and epairs on boot. /etc/jail.conf defines jails and their VNICs. Jails rc.conf's configure VNICs, gateway, tunnels, and services within the jails.
• Firewall (pf): The firewall configuration denies all traffic by default and allows only necessary traffic. Allowed incoming traffic is limited to the Nginx jail and SSH from IPs in the <xpeers> table. Rate limits block IPs that connect 100 times within 10 seconds and add them to the <ratelimit> table. Allowed outgoing traffic is limited to the Node jail for specific connections. Deploying, generating TLS certificates, and extracting backups are done via rsync from an external server.
• References: Shawn Webb (2012) Virtually Networked FreeBSD Jails.
• Open Source: Check out ops-utils/location-server for automated installation and setup.
• FAQs: A bare-metal server can handle ~6 million daily requests with an 8-core server (Hacker News 2021) and serve over 800 Gb/s per server (Netflix 2022). Bare-metal is more cost-effective than cloud once the cloud bill exceeds $240/month. In case of hardware failure, the company takes care of it. For microservices, nested jails or multiple top-level application jails can be created. The infrastructure can be simulated in VirtualBox with a guide. Hard disk failures can be handled with hot spares in ZFS.