Network Design
It's tempting to start a network design by picking out hardware. Resist that. The gear and protocols are the last decision, not the first. Before any of that, I want to know what the network actually has to do, what happens when parts of it fail, and who's going to be running it. A good design should make the normal path obvious and make failures easy to see.
Questions I ask first
- How many users, devices, racks, sites, or tenants?
- What's sensitive to latency, and what eats bandwidth?
- How much downtime is actually acceptable? (Get a real answer, not "none.")
- What needs to be kept apart from everything else?
- What Internet, WAN, and cloud connections are needed?
- How fast is it going to grow?
- Who's going to operate it, and with what tools?
- What are the limits on budget, support contracts, licensing, and hardware lifecycle?
Think in failure domains
A failure domain is everything that goes down when one thing breaks. Redundancy only helps if the redundant parts don't secretly depend on the same thing. Some classic examples:
- Two switches plugged into the same power circuit.
- Two "diverse" carriers that come into the building through the same conduit.
- Two virtual routers on the same hypervisor.
- Two firewalls that both got the same bad config pushed to them.
Whenever I see "redundant," I try to ask: redundant against what?
Keep it understandable
Every layer, protocol, and box should be there to solve a specific problem. Simple networks are easier to run, secure, and fix. Complexity is fine when it buys you real scale, resilience, security, or automation. But I'm suspicious of cleverness only one person can explain, especially if that person is me from three years ago.
Address planning
Give out addresses in contiguous blocks by site, rack, tenant, or function, so you can summarize later and have room to grow. Grabbing the next free subnet every time works fine until it really doesn't. Here's the kind of layout I like:
| What | Example block | Why |
|---|---|---|
| Loopbacks | 10.255.0.0/24 | Stable router IDs and management targets |
| Point-to-point links | 10.254.0.0/16 | Same-size subnets everywhere |
| Site A users | 10.10.0.0/16 | Summarizes cleanly by site |
| Site B users | 10.20.0.0/16 | Room for lots of VLANs |
| IPv6 per site | 2001:db8:1000::/48 | A /64 for each LAN |
Keep layer 2 small
Stretching VLANs over long distances makes loops, broadcast storms, and control-plane problems spread a lot further. Route between zones whenever you can. Only stretch layer 2 when an application or a migration genuinely needs it, and ideally only for as long as it needs it.
Kinds of redundancy
- Devices: more than one router, switch, firewall, and power supply.
- Paths: uplinks and carrier routes that are actually diverse.
- Protocols: dynamic routing, first-hop redundancy, fast failure detection.
- Services: more than one DNS, DHCP, authentication, and monitoring server.
- Operations: backups, out-of-band access, spare hardware, and written-down recovery steps. People forget this one.
Capacity and oversubscription
Not every port is going to run at line rate at the same time, so some oversubscription is normal. Just make it a decision, not an accident. Think about server-to-server traffic, backups, replication, storage, VM migrations, and Internet traffic. Then watch real usage and leave enough room for a failure and for growth.
Security from the start
Decide on trust boundaries while you're designing, not after it's built. Figure out your management, server, user, guest, backup, public, and tenant zones. Then decide what has to go through a stateful firewall and what's fine with a routed ACL.
Thinking about day two
- Standardize configs and naming.
- Buy hardware with a support lifecycle you understand.
- Make sure you can keep paying for the licenses.
- Collect logs, metrics, and flows from day one.
- Back up configs automatically.
- Write down how normal failover and maintenance work.
- Have an out-of-band way in.
Documentation that helps during an outage
The test for documentation: is it useful when things are on fire? The stuff I want on hand:
- Logical topology diagram
- Physical cabling and rack diagram
- IP address and VLAN plan
- A summary of routing and security policy
- Carrier circuit IDs and contacts
- Device inventory and software versions
- Change, rollback, and disaster recovery steps
Planning a change
For anything non-trivial, I like to write this out before touching anything:
Goal: what problem does this fix?
Prechecks: what should I confirm before starting?
Steps: exactly what I'll do, in order.
Validation: how I'll know it worked.
Rollback: how to undo it, and the point where I decide to.
Communication: who needs to know before, during, and after.
Questions for reviewing a design
- What happens when each link, device, power feed, and provider fails?
- Can I still get in if the main network is down?
- Can the routes and addresses be summarized?
- Are the trust boundaries clear?
- Can it be monitored and backed up?
- Could someone who didn't build it understand it?
Last Modified: October 12, 2026, 11:46 am