WSUS in the Real World (Under construction)

I have been managing WSUS for about ten years. It hasn't always been a happy relationship. But I've finally found a setup I'm happy with, and I'd love to share it with you. This article is meant to be a mind-dump on tons of subjects within WSUS. I'm writing it to teach my successor everything I've learned.

This article will detail how to configure an internal WSUS server hosting approved updates, and an external WSUS replica server that syncs approvals with the the internal server, but tells clients to download update files from Microsoft. The external server then rolls-up all client status to the internal server. This is great for a roaming laptop workforce without constant VPN access.

This configuration is used at a company with over 60 offices in multiple AD domains. However, all machines pointed to the same WSUS infrastructure. WSUS does not use Active Directory or Kerberos and functions solely over HTTP and HTTPS.

Normally this would not work totally reliably because of a known issue with WSUS not replicating WSUS ID cookies between master and replica servers. However, with a scheduled task on your clients you can just kick them to refresh the cookie if needed. This will be detailed in the client configuration section.

Annoying voice: You should follow my advice and rebuild WSUS

WSUS is notorious for degrading over time and increasing in storage costs without extreme administrative intervention. Additionally, it is disposable infrastructure as it contains no business nor historical data. For this guide you should prepare to completely wipe out your entire WSUS configuration and data, and I'll help you. I've done this several times in my career and it should not be a big deal. However, this guide can be retrofitted to an existing installation if you see absolutely no need.

Strip down WSUS server for reinstallation

This assumes you have a dedicated machine for WSUS that may already have components installed. If the machine has other things installed, some instructions will irrevocably delete applications and websites.

Some of these steps seem redundant or pointless but they each prevent/address a problem I've encountered.

1. Cleanup with features and roles uninstall

  1. Server Manager > Manage > Remove Roles and Features
  2. Under Roles on the left, uncheck Web Server (IIS) and Windows Server Update Services.
  3. Under Features on the left, uncheck the following, which we want a fresh install for later:
    1. Remote Server Administration Tools > Role > Windows Server Update Services Tools
      (~~Do not skip manually unchecking this! It stores files in a location we will be clearing out.~~)
    2. Windows Internal Database
    3. Windows Process Activation Service
  4. Reboot when finished

2. Server hygiene and orphan configuration cleanup

These steps are to ensure a long stable life for your server and remove any past orphaned configurations. Uninstalling some roles doesn't uninstall their configuration settings. You have to purge some role configurations manually if they're corrupted, and this fact trips up many administrators trying to fix issues.

  1. First, completely update the server directly against Microsoft Update and reboot. Recommend you wait until this is finished. These updates contain critical compatibility fixes to allow WSUS to update new versions of Windows and should be resident before you start configuring WSUS. Reboot when finished updating.
  2. Command to baseline Windows SxS to save disk space:
    dism.exe /online /Cleanup-Image /StartComponentCleanup /ResetBase
  3. Windows Server doesn't come with cleanmgr.exe unless you have the Desktop Experience installed, but there are hacks to install it, so run it now if you have it.
  4. Remove/rename contents of the Windows Internal Database folder where WSUS stores update settings, do not delete the folder itself:
    C:\Windows\WID
  5. Remove/rename leftover web server settings folders:
    1. C:\inetpub
    2. C:\Program Files\Update Services
  6. If extant, delete the contents of your old WSUS update storage directory. Do not delete the folder itself.
  7. Reboot, it may take awhile.

Building a blank-slate WSUS server

This will prepare either a master or replica server, whose individual configuration diverges later. The default settings of installing WSUS is unfortunately not the best way to do it, please follow these instructions very carefully because there are some tricks.

1. Storage folder

  1. Decide where you want to keep your WSUS files, and create the folder if needed. This is required for both master and replica servers.
    I have my folder on the root of the storage drive, so it's E:\WSUS.
  2. On your WSUS content folder, grant the "NETWORK SERVICE" security principal "FULL CONTROL" rights to the root folder, applying to all files and subfolders.
    Normally these kinds of broad permissions-granting instructions are nonsense, but this is literally in Microsoft guidance. Just do it. It's not a security risk.

2. Roles install

  1. Server Manager > Manage > Add Roles and Features.
  2. Under Roles on the left, check Windows Server Update Services.
  3. Do not add anything under roles. It will automatically add what it needs.
  4. Under WSUS > Role Services, use the default WID Database and WSUS Services. WID is actually SQL Server code. Using full SQL Server introduces a lot of configuration and maintenance complexity. Only very large companies in certain scenarios would need to run a full SQL Server instance. I've tried it but don't recommend it.
  5. Under WSUS > Content, put in the path of your storage folder.
  6. Under Web Server Role, just use defaults.
  7. Under Confirmation, select "Restart the destination server" and click Install.
  8. Wait for it to complete
  9. Run the following command from an Administrator command prompt:
    sfc.exe /scannow
    This is required to fix a known issue with reinstalling WSUS components after their removal. DISM fixes the SxS, SFC fixes the OS.
  10. Server Manager > click the flag > Perform post-install tasks. Wait for this to finish.

Master internal server - specialization

You need a 2012R2, preferably 2016, Windows server. You can try 2008R2/2012 and it will sort-of work with hacks you can find on Google. I use 2012R2, but if you have the choice use Server 2016 because I do not trust Microsoft to backport all future WSUS fixes.

For WSUS I use a 60GB OS disk and a 150GB storage disk. Your requirements will vary drastically depending on the breadth of software in your organization. It's best to over-provision storage as much as possible to give yourself slack space as WSUS naturally expands and contracts by tens of gigabytes during normal use.

Regarding RAM I don't know the realistic minimum, but I'd say 4GB RAM+. Remember, Windows caches commonly-used files in RAM, so the more RAM you have the better you can serve clients at wire speed. On CPU, I do know how it feels running WSUS on a two-core VM and a 1.8GHz E5-2403 server, and it's painful. But it still works.

1. Windows Server Updates Services Configuration Wizard - Master

  1. Launch Windows Server Update Services console.
  2. Select Synchronize from Microsoft Update
  3. Click "Start Connecting"
  4. Click "Next" when complete.
  5. Download updates only in these Languages: [required languages].
  6. Under Chose Products, unselect everything then only choose only these products in addition to other random software you need to update.
    1. Windows 10 Dynamic Update
    2. Windows 10
    3. Windows 7
    4. Windows 8.1
    5. Windows Defender
    6. Windows Dictionary Updates
    7. [Selected needed server versions]
    8. [Select needed Office versions]
  7. Under Choose Classifications, select everything 'EXCEPT' DRIVERS. DO NOT SELECT DRIVERS. EVER.
  8. Under Configure Sync Schedule, select Synchronize Automatically, 24 times a day. This update cadence is required to ensure you distribute virus definition updates as quickly as possible. Minutes matter. It doesn't cost you anything.
  9. Under Finished, do not begin initial synchronization.
  10. Click "Finish"

2. Update Services console configuration - Master

  1. On the left, Update Services > [Server name] > Options > "Update Files and Languages"
  2. Select Download express installation files.
  3. Under the Update Languages tab, re-verify you're only syncing your required languages.
  4. On the left, Update Services > [Server name]
  5. Click Synchronize Now.
  6. This process may take hours. 

3. Configure IIS - Master

Now we need our Master server to respond to WSUS.EXAMPLE.COM.

  • Open the Internet Information Services (IIS) Manager and go to Sites.
  • First, you'll see the "Default Web Site." Do not delete the Default Web Site on a WSUS server. Several WSUS processes rely on it existing for their sanity checks. It's innocuous and won't hurt anything.
  • Right-click "Default Web Site" > Bindings
  • Edit the following binding so we can later have WSUS become the default http server on port 80. This will be important for the external replica working blindly later.
    • [EDIT]
      Type: http - Port: 80
      change this to:
      Type: http - Port 8080
  • Second, you'll see the "WSUS Administration" site. Despite its name, this is the entire WSUS web application.
  • Right-click "WSUS Administration" > Bindings
  • Add the following bindings. Do not edit existing bindings.
    • [ADD]
      Type: http - IP address: All Unassigned - Port 80
      Host name: [blank]
      This is important for the replica server to be able to blindly connect to an IP address on port 80 through the DMZ firewall without presenting a destination host name.
    • [ADD]
      Type: https - IP address: All Unassigned - Port 443
      Host name: WSUS.EXAMPLE.COM
      SSL Certificate: WSUS.EXAMPLE.COM
  • Click [OK].

Preparation to roam your clients with DNS, DMZ, TLS, and IIS

If you want to enable roaming clients to securely use WSUS, it may appear you only have two options. You could:

  • Put a server in the DMZ, have it host updates locally, and have all clients talk to it. Unfortunately, this would crush your outbound bandwidth from laptops at home downloading massive Windows 10 updates from your office Internet connection.
  • Put a server in the DMZ, tell it to not store updates locally, and have all clients download updates from Microsoft. Unfortunately, this would crush your inbound bandwidth from desktops in the office downloading massive Windows 10 updates over your office Internet connection.

You can't win! Or can you? In fact, there is another option. You can combine the particularities of split-brain DNS and WSUS replica servers to provide what your clients need, in the most appropriate way, from anywhere in the world, all rolling up to a single central console.

This is possible because the clients who request the IP address of WSUS.EXAMPLE.COM inside your network can be given your internal WSUS server, and external clients who request the IP address of WSUS.EXAMPLE.COM will be given your publicly-accessible server in the DMZ. Both servers will have a TLS certificate for WSUS.EXAMPLE.COM. There's no reason a TLS certificate can only be on one server.

The replica server will clone all the patch approvals on your master server, then report-back to your master server everything it does with external clients. How to configure the servers themselves will be later in the guide.

1. Internal DNS configuration

  • Create a CNAME record for WSUS.EXAMPLE.COM and point it to the network name of your internal DNS server.

2. External DNS configuration

  • Create an A record and point it to your external IP address.

3. DMZ network configuration

For the external server in DMZ, on the network layer allow inbound port 80 and 443 from external. Allow the server unlimited outbound access to the Internet, or restrict it to just the addresses in this article.

4. TLS certificate

Communications with your WSUS server over untrusted networks should be protected with HTTPS, so you'll need a TLS (formerly SSL) certificate for WSUS.EXAMPLE.COM. You'll need to acquire that and install it yourself for the internal server, but we can get an automated certificate for the external server.

Replica external server - specialization

There are several fundamental differences in configuring a replica server, please follow instructions very carefully.

1. Windows Server Updates Services Configuration Wizard - Replica

  1. Launch Windows Server Update Services console.
  2. Select Synchronize from another Windows Server Update Services server
  3. Server name: [IP address of Master server]
    Port number: 80
    (You can't use SSL to connect because encryption is based on the host name, which, if your server is in the DMZ, won't point to the right place. It's okay to use port 80 here. You could use the HOSTS file to get around this, but undocumented changes to DNS resolution per-server is best to avoid. WSUS is resistant to HTTP MitM and will not install non-Microsoft files unless you enable that on the client. However, TLS provides critical integrity assurance and manipulation resistance and makes it work better, which is why we use it for client communications.)
    Check: This is a replica of the upstream server.
  4. Click "Next"
  5. Click "Start Connecting"
  6. Click "Next" when complete.
  7. Download updates only in these Languages: [required languages].
  8. Under Configure Sync Schedule, select Synchronize Automatically, 24 times a day.
  9. Under Finished, do not begin initial synchronization.
  10. Click "Finish"

2. Update Services console configuration - Replica

  1. On the left, Update Services > [Server name] > Options > "Update Files and Languages"
  2. Select Do not store update files locally; computer install from Microsoft Update
  3. Click [Ok]
  4. On the left, Update Services > [Server name]
  5. Click Synchronize Now.
  6. This process may take hours. 

3. Configure IIS for HTTP - Replica

First we need the Replica to respond to WSUS.EXAMPLE.COM on HTTP.

  • Open the Internet Information Services (IIS) Manager and go to Sites.
  • You'll see the "WSUS Administration" site. Despite its name, this is the entire WSUS web application.
  • Right-click "WSUS Administration" > Bindings
  • Add the following bindings. Do not edit existing bindings. HTTPS will be in another step.
    • [ADD]
      Type: http - IP address: All Unassigned - Port 80
      Host name: WSUS.EXAMPLE.COM
  • Click [OK].

4. Get self-renewing TLS certificate with Let'sEncrypt

Configuring IIS

Client configuration

todo

Client configuration - Scheduled tasks

todo

References

https://omgdebugging.com/2017/10/09/command-line-equivalent-of-wuauclt-in-windows-10-windows-server-2016/