Originally posted to https://www.mjshellenberger.com on 1/17/2023. Cross-posting here as well.
The shift to endpoint management (Microsoft Intune) and Windows Updates (Windows Update for Business) in the cloud is an absolute success in terms of supporting modern Windows endpoints. The ability to manage devices and ensure updates are flowing regardless of a device’s current location is absolutely essential to ensuring devices stay up to date and compliant. It can present a unique network challenge though as all content destined for the device must now be pulled down across an Internet connection. For a single endpoint on a lone connection, this isn’t much of a concern. However, let’s consider the enterprise network where hundreds or perhaps thousands of endpoints are now sourcing content from the Internet and a problem arises. Inefficient network use and bottlenecks on the Internet connection become a very real concern. Fortunately, Microsoft accounted for this shift in content location very early on by engineering a solution built into every Windows 10/11 client. In this article, I’ll be sharing what I found while reading up on delivery optimization and Microsoft Connected Cache.
What is Delivery Optimization in Windows 10/11?
To address the inefficiency of every device reaching out to Internet-based content distribution networks (CDN) everytime content is required, Microsoft engineered a technology known as delivery optimization in the Windows platform. Delivery optimization allows a Windows 10/11 device to pull cached, trusted content from other Windows 10/11 device peers on the same network. By sourcing some or all of the content from cache on peer endpoints, the reliance on Internet bandwidth is greatly reduced. Delivery Optimization capabilities are enabled by default. For most small to mid-sized networks, there probably isn’t a lot of tweaking you really need to do to this, either. It just works. You can use Delivery Optimization with Windows Update, Windows Server Update Services (WSUS), Windows Update for Business, or Microsoft Configuration Manager (when installation of Express Updates is enabled).
How does Delivery Optimization work?
Any Windows 10 (1511) or later client has delivery optimization enabled, by default. So, for this example let’s assume we are on a network with two Windows endpoints. One of the endpoints (Client1) determines it needs a Windows Update. Client1 will start by retrieving the updates content metadata. The content metadata file is a hash file that contains the SHA-256 block-level hashes of each piece in the file. These chunks are approximately 1MB in size. Once the authenticity of the content metadata file is proven, Client1 retrieves a list of local peer cache sources. In the case of this particular update, there are no local peer cache locations, so the client is referred to the CDN for content delivery. A checksum is performed for each chunk of the download to verify authenticity before the client accepts the data. After download, the content is cached on Client1 for potential future delivery to other clients. Client1 also registers itself as a cache peer for the local network (using the public NAT of the Internet connection it retrieved the updates through). Unless you specifically make changes to the Delivery Optimization settings of your Windows endpoints, the cache will continue to hold the content for three days before it is purged.
Now let’s assume another Windows endpoint on the network decides it also needs the same Windows update file as Client1. The content metadata file is retrieved and verified by Client2 but this time additional peer cache locations are communicated to Client2 letting it know that Client1 is a potential optimized location to source the content. The same scenario of downloading the chunks and verifying checksum is run to ensure the content on the peer has not been tampered with. Any invalid chunks will be discarded. If a peer sends multiple bad chunks, the peer is banned and will no longer be a source for the client performing the download.
Once Client2 has retrieved all the pieces of the file and reassembled, the source (Windows Update in this case) will verify the validity of the file and then continue on with the installation. Client2 will now notify the Delivery Optimization service that it’s a peer cache source for this update and will maintain the cache for three days before purging the data.
When should I consider manual configuration of Delivery Optimization Settings?
For most small to medium sized environments, the default settings for delivery optimization should work just fine and there is little need to tweak. There are a few scenarios that might warrant minor changes. Microsoft highlights a few here in their Recommended Delivery Optimization Settings docs article. Examples given are:
- Networks with > 30 Windows endpoints – If you have more than 30 devices in your local network or group, change file size to cache from the default 50 MB to 10 MB. If you have more than 100 devices (and are running Windows 10, version 1803 or later), set this value to 1 MB. This will help ensure endpoints leverage peer-to-peer capability in more downloads.
- Large Number of Laptops – Set the Allow Uploads on battery power to 60% to increase the number of devices that can upload while limiting battery drain.
- Large Number of Stationary Endpoints – Say for instance a computer lab or bank of desktop computers that are always on. In this scenario, it might make sense to change the Content Expiration value to 7 days or more. (30 days is the max). With this configuration, these devices will end up uploading more for a longer period of time maximizing their more ideal network circumstance.
Delivery Optimization is also VPN aware in that if a VPN is active on the client, uploads will be suspended. It does this by monitoring the network adapter type and details, looking for VPN connection types or adapter descriptions that contain keywords such as “VPN”. You can modify this behavior using the delivery optimization settings if you’d like.
How do I modify Delivery Optimization settings?
If you choose to make changes to the default delivery optimization settings, you have a few options for administration. If your clients are joined to a Windows Server Active Directory, group policy can be leveraged. You’ll find the settings under Computer Configuration\Administrative Templates\Windows Components\Delivery Optimization. If your clients are managed by an MDM solution, such as Microsoft Intune, you can leverage the more modern configuration service provider (CSP) settings via a configuration profile. A Delivery optimization profile can be created for Windows clients managed by Intune by navigating to: Configuration profiles > Create profile > Windows 10 and later > Templates > Delivery Optimization
How can I tell if Delivery Optimization is working properly?
If you are curious to see how delivery optimization (DO) has been working on a specific client, or you are troubleshooting a client delivery optimization issue, you have several Powershell commands that can report out on DO activity. The first command, Get-DeliveryOptimizationPerfSnap shows a historical view of how DO has been performing on a client.
adding the -verbose flag will display the following additional information:
- Bytes from peers (per type)
- Bytes from CDN (the number of bytes received over HTTP)
- Average number of peer connections per download
This additional info can be helpful in identifying where bits are being sourced in the event that you believe clients might be going to the CDN more than they should and are not leveraging peers.
To demonstrate delivery optimization in action and the associated PowerShell commands to see activity, I ran a test on two different clients on my network. Client1 was my corporate-owned Windows 11 device joined to Azure Active Directory and managed by Microsoft Intune. Client2 was my personal Windows 10 device.
Beginning on Client1, I browsed to the Microsoft Journal app in the Windows Store, then downloaded and installed the app. Running the PowerShell command on Client1: Get-DeliveryOptimizationStatus you can see delivery optimization in action with the downloaded file from the CDN.
On Client2, I then proceeded to download and install the Microsoft Journal app from the Windows Store and while the download was occurring, ran the PowerShell command: Get-DeliveryOptimizationStatus -PeerInfo.
As you can see, Client2 was actively sourcing the file from Client1. If you are interested in how delivery optimization is working across all systems on your network, you can leverage Windows Update for Business reports. More information about deploying and configuring Windows Updates for Business reports can be found here: https://learn.microsoft.com/en-us/windows/deployment/update/wufb-reports-overview Valuable reports can be run such as observed bandwidth savings across all devices that used peer-to-peer distribution over the past 28 days.
If you wish to delete the entire delivery optimization cache on a client, you can use the command: Delete-DeliveryOptimizationCache. The command will clear files from the cache and remove all persisted, related data.
Delivery Optimization listens on port 7680 for requests from other peers by using TCP/IP. This means the local Windows firewall instance will need to be opened for DO to work properly. By default, the proper exceptions already exist but if you manage the Windows Firewall via enterprise management, you will need to review your configuration to ensure the proper ports are open. You can easily test whether a particular client is listening on TCP port 7680 by running the command: telnet [Target Client IP] 7680 from another client on the same network. Note: If the telnet client is not already installed, you can run the command: dism /online /Enable-Feature /FeatureName:TelnetClient to quickly install it. Another important note is that delivery optimization does not use multicast, so no need to have that configured on your network for DO to work properly.
Conclusion
Since delivery optimization is built-into Windows 10/11 by default and enabled with a configuration that suits most situations, there is likely little you will need to further configure the solution. If, however, you manage an enterprise network, there are some additional tweaks and monitoring that can be done to ensure DO is achieving maximum efficiency for your network. In the next blog article, we will talk about two additional solutions for enterprise networks that can also be leveraged for managing client bits distribution: Microsoft Connected Cache (Standalone) and Microsoft Connected Cache for Configuration Manager.
Originally posted to https://www.mjshellenberger.com on 1/17/2023. Cross-posting here as well.