Not yet. The WinRM cannot complete the operation error isn't an authentication failure. It's the client telling you it never got a WS-Management answer from that name on that port: the name resolved somewhere else, the path or a network firewall dropped TCP 5985, the WinRM service or its listener isn't there, the Windows Firewall rule on the target is off or scoped to the local subnet, the target's network profile flipped to Public, or a proxy got in the way. Microsoft's own copy of the message, from Server Manager's troubleshooting guide, says as much: "Windows Remote Management (WinRM) cannot complete the operation. Verify that the specified computer name is valid, that the computer is accessible over the network, and that a firewall exception for the Windows Remote Management service is enabled and allows access from this computer. By default, the WinRM firewall exception for public profiles limits access to remote computers within the same local subnet." TrustedHosts doesn't touch any of that. Microsoft describes it as an authentication setting: "you should consider the TrustedHosts setting to be the list of hosts for which you wish to suppress the error generated by being unable to verify the server's identity." Work through it as a chain: which command, which name, from where, and what else works? → what does the whole error say? → does the name resolve to the right address, and does TCP 5985 answer? → does WinRM itself answer (Test-WSMan)? → on the target: is the service running and is there a listener on the right addresses? → is the firewall rule on, in which profile, scoped to which addresses, and what profile is the network in? → is a proxy in the path? → once it connects, does the error change to an authentication one? → change one link → prove it from the same client. Resolve-DnsName, Test-NetConnection, Test-WSMan, Get-Service, winrm enumerate winrm/config/listener, Get-WSManInstance, Get-NetConnectionProfile, Get-NetFirewallRule, Get-NetFirewallAddressFilter, netsh winhttp show advproxy, and setspn -Q are the sources for that chain. Set TrustedHosts to * just in case and you've told every PowerShell session on that box to skip server identity checks for every name, while the firewall rule that dropped the packet is still dropping it.

If the same server also fails Kerberos to file shares or SQL with a target name error, The Target Principal Name Is Incorrect covers the DNS, password, and SPN side. If the remoting failure shows up during Hyper-V live migration or remote host management, Hyper-V Live Migration Failed (Event 21502) covers the migration side. This post covers **"a Windows client or server that fails Enter-PSSession, Invoke-Command, New-PSSession, Test-WSMan, Windows Admin Center, Server Manager, or Hyper-V Manager to a Windows Server target with 'WinRM cannot complete the operation', with or without the related 'connection to the remote host was refused', 'client cannot connect to the destination', 'cannot process the request', or OperationTimeout errors, and you need a read-first path before you set TrustedHosts to , open the Public firewall rule to Any, or re-run Enable-PSRemoting everywhere."*

TL;DR

• Read the whole error, not the first five words. WinRM cannot complete the operation means no answer came back; "The WinRM client cannot process the request" and "WinRM cannot process the request" mean an answer came back and authentication failed. They have different fixes (Step 2).
• Prove the path before the service. Test-NetConnection -ComputerName FS02 -Port 5985 tells you whether the name resolves and TCP 5985 answers from this client. A failure there is DNS, routing, or a firewall, not PowerShell (Step 3).
• Test-WSMan is the cheapest WinRM probe. Microsoft: it "submits an identification request that determines whether the WinRM service is running on a local or remote computer." Without -Authentication it's sent anonymously, so it separates a missing listener from missing rights (Step 4).
• A running service isn't a listener. Microsoft: "By default, no WinRM listener is configured. Even if the WinRM service is running, WS-Management protocol messages that request data can't be received or sent." A policy-made listener with an empty filter listens on nothing (Step 5).
• The Public profile is the classic cause. Microsoft: Enable-PSRemoting "also creates a firewall rule for public networks that allows remote access only from computers in the same local subnet." A server whose NIC fell to Public answers its own subnet and nobody else (Step 6).
• TrustedHosts is for authentication, not reachability. Microsoft: "The computers in the trusted hosts list aren't authenticated." Use it only for the IP-address or workgroup case it was made for, with a specific entry (Steps 8 and 9).

Where in the chain did it break?

Step 1: Capture the symptom (read-only)

Write these on the ticket first:

Then branch on what you see:

Step 2: Read the whole error (read-only)

Copy the full text, including the part after the first sentence. In PowerShell, the last error holds it:

# Read-only: the full text of the last remoting error, and its error ID
$Error[0].Exception.Message
$Error[0].FullyQualifiedErrorId

Then match it against Microsoft's messages. They look alike and they point at different links:

Two details matter when you compare with forum posts. First, the text PowerShell prints usually says WinRM where Microsoft's Server Manager copy says Windows Remote Management (WinRM), and the WinRM service where it says the Windows Remote Management service; read it as the same no-answer case (our reading). Second, Microsoft's Server Manager table lists the same no-answer text for workgroup targets where TrustedHosts was already set, and gives the cause as "a subnet timeout error occurs if the user has not changed the default subnet restrictions for computers in workgroups". That's the firewall scope (Step 6), not authentication: adding the server to TrustedHosts a second time won't change it.

Step 3: Does the name resolve, and does the port answer? (read-only)

Run these from the client that fails, by the same name the failing command used:

# Read-only, on the client: what the name resolves to from here
Resolve-DnsName -Name FS02.contoso.com -Type A_AAAA

# Read-only, on the client: name resolution, route, and TCP 5985 (WinRM over HTTP) in one test
Test-NetConnection -ComputerName FS02.contoso.com -Port 5985 -InformationLevel Detailed

# Read-only, on the client: TCP 5986 (WinRM over HTTPS), if the command uses -UseSSL
Test-NetConnection -ComputerName FS02.contoso.com -Port 5986

Then, on the target's console or an RDP session (remoting to it is the thing that's broken), read the addresses it actually has:

# Read-only, on the target: its IPv4 addresses, to compare with what DNS returned on the client
Get-NetIPAddress -AddressFamily IPv4 | Select-Object -Property InterfaceAlias, IPAddress, PrefixLength

The ports are Microsoft's defaults: "PowerShell Remoting with WinRM listens on the following ports" HTTP 5985 and HTTPS 5986, and the Invoke-Command reference adds "The default ports are 5985 (WinRM port for HTTP) and 5986 (WinRM port for HTTPS)." Older guidance with port 80 and 443 is from before WinRM 2.0; Microsoft's HTTPS article: "On earlier versions of Windows, WinRM HTTP uses port 80 and WinRM HTTPS uses port 443."

If the path fails only from one site or VPN, check the network firewall between them for TCP 5985 before you touch the server. Microsoft's Server Manager guide gives the same first suspect for this message: "The network connecting the client and server is offline, or there is a problem with the target server's connection to the network."

Step 4: Does WinRM itself answer? (read-only)

An open port isn't proof that WinRM is behind it. Test-WSMan asks WinRM directly. Microsoft: it "submits an identification request that determines whether the WinRM service is running on a local or remote computer."

# Read-only, on the client: anonymous identify request to the default port
Test-WSMan -ComputerName FS02.contoso.com

# Read-only, on the client: the same request with authentication (also returns the OS version)
Test-WSMan -ComputerName FS02.contoso.com -Authentication Default

# Read-only, on the client: over HTTPS, if the target has an HTTPS listener
Test-WSMan -ComputerName FS02.contoso.com -UseSSL

Microsoft explains why the first one is useful: "If you do not specify the Authentication parameter, the Test-WSMan request is sent to the remote computer anonymously, without using authentication." So the two forms split the problem:

For Windows Admin Center, run the same tests from the gateway, not from your browser's machine: it's the gateway that opens the WinRM session. Microsoft's WAC page says a failed Enter-PSSession from the gateway "might relate to WinRM issues, network connectivity problems, or authorization issues."

Step 5: On the target, is the service running and is there a listener? (read-only)

Read these on the target, from its console or RDP, in an elevated PowerShell window. Microsoft: "You must have administrative rights to view or change settings for the local computer in the WSMan: drive."

# Read-only, on the target: the WinRM service and its startup type
Get-Service -Name WinRM | Select-Object -Property Name, Status, StartType

# Read-only, on the target: the listeners, in Microsoft's form
winrm enumerate winrm/config/listener

# Read-only, on the target: the same listeners through PowerShell, with Source and ListeningOn
Get-WSManInstance winrm/config/listener -Enumerate

# Read-only, on the target: the service settings (auth methods, IPv4Filter, IPv6Filter, ports)
winrm get winrm/config

# Read-only, on the target: does WinRM answer locally, before the network is involved?
Test-WSMan -ComputerName localhost

# Read-only, on the target: the session endpoints and who may use them
Get-PSSessionConfiguration | Format-Table -Property Name, Permission -AutoSize

Microsoft's defaults to compare with: "On server versions of Windows, the WinRM service startup type is Automatic. However, on client versions of Windows, the WinRM service is disabled by default." And remoting is on by default on servers: "Windows Server 2012 and newer releases of Windows Server are enabled for PowerShell remoting by default." So a Windows Server target without a listener had it removed or never got the default, usually by policy, an image, or a hardening script (our reading).

Read the listener output against Microsoft's listener properties:

The empty ListeningOn case is Microsoft's own example. When the listener policy is misconfigured: "A configuration error in the policy results in an empty value for the ListeningOn property." The fix is in the policy: Microsoft's remoting help says "Enable the policy and specify the IPv4 and IPv6 filters." Its remoting help names the policy Allow automatic configuration of listeners, under Computer Configuration\Administrative Templates\Windows Components\Windows Remote Management (WinRM)\WinRM service; Microsoft's Server Manager page names the setting in that folder "Allow remote server management through WinRM". Read both names as the same listener policy when you look for it in your GPOs (our reading).

Step 6: Is the firewall rule on, for the right profile and scope? (read-only)

This is the step that matches the second half of the message, "By default, the WinRM firewall exception for public profiles limits access to remote computers within the same local subnet." Read the profile first, then the rules, on the target:

# Read-only, on the target: the network category of each connected adapter
Get-NetConnectionProfile | Select-Object -Property InterfaceAlias, Name, NetworkCategory

# Read-only, on the target: the WinRM inbound rules, whether they're on, for which profile, and the action
Get-NetFirewallRule -Name 'WINRM*' | Select-Object -Property Name, DisplayName, Enabled, Profile, Action

# Read-only, on the target: the remote address scope of each WinRM rule
Get-NetFirewallRule -Name 'WINRM*' | ForEach-Object -Process {
    $r = $_
    $r | Get-NetFirewallAddressFilter | Select-Object @{n='Rule';e={$r.Name}}, RemoteAddress
}

# Read-only, on the target: the local port on each WinRM rule
Get-NetFirewallRule -Name 'WINRM*' | Get-NetFirewallPortFilter | Select-Object -Property Protocol, LocalPort

# Read-only, on the target: the rules group policy delivers (the RSOP store), which local edits can't override
Get-NetFirewallRule -PolicyStore RSOP | Where-Object -FilterScript { $_.DisplayName -like '*Remote Management*' } |
    Select-Object -Property Name, DisplayName, Enabled, Profile, Action

Microsoft's rule names come from the Enable-PSRemoting and remoting help pages: WINRM-HTTP-In-TCP and WINRM-HTTP-In-TCP-NoScope in its client example, and WINRM-HTTP-In-TCP-PUBLIC for the Public rule on servers. The display name you'll see in the console is "Windows Remote Management (HTTP-In)". Microsoft also warns: "The name of the firewall rule can be different for different versions of Windows." That's why the read above uses a wildcard.

What Microsoft says the defaults are:

On the policy side, Microsoft names two settings. The Windows Firewall: Allow local port exceptions policy decides whether local rules apply at all: "This policy allows members of the Administrators group to create a firewall exception for the Windows Remote Management (WinRM) service." And for Server Manager's version of this message, Microsoft's advice is the same read: "Try verifying that Windows Firewall exceptions have been enabled on the target server for Windows Remote Management (WinRM) inbound rules."

Step 7: Is a proxy in the path? (read-only)

Microsoft: "Because PowerShell remoting uses the HTTP protocol, it is affected by HTTP proxy settings." Read the machine-wide WinHTTP setting and any session option on the client:

# Read-only, on the client: the WinHTTP advanced proxy setting (Microsoft's current form)
netsh winhttp show advproxy

# Read-only, on the client: the older form, deprecated but still present on many builds
netsh winhttp show proxy

# Read-only, on the client: a session option set in this session or in a profile script
$PSSessionOption

Microsoft's proxy notes for WinRM: "By default, WinHTTP is not configured to use a proxy server." and, for WinRM 1.1 and earlier, "the WinRM client relies on the Windows HTTP services (WinHTTP) proxy configuration." On the session side, New-PSSessionOption's ProxyAccessType accepts IEConfig, WinHttpConfig, AutoDetect, NoProxyServer, and None, and "The default value is None."

Microsoft's proxy page is also clear about HTTP through a proxy: "If the proxy server requires authentication, the WinRM clients and servers must use HTTPS." A design that sends WinRM to internal servers through an authenticating proxy over HTTP is the problem, not a setting (our reading).

Step 8: When it connects, does the error change to authentication? (read-only)

Often, fixing the profile or the rule turns "cannot complete the operation" into one of the "cannot process the request" messages. That's progress: the connection works and the authentication path is now the question. Microsoft's rules for which path you get:

On the IP and workgroup rows, Microsoft adds: "Use the Credential parameter in all remote commands. This is required even when you connect as the current user." The Invoke-Command reference says the same: "To use an IP address in the value of ComputerName, the command must include the Credential parameter."

Read what the client already trusts, and the SPNs behind the name you typed:

# Read-only, on the client: the current TrustedHosts list
Get-Item WSMan:\localhost\Client\TrustedHosts

# Read-only, any domain-joined machine: which account holds the HTTP SPN for the target's names
setspn -Q HTTP/FS02.contoso.com
setspn -Q HTTP/FS02

# Read-only: the SPNs registered on the target's computer account
setspn -L FS02

# Read-only, on the client: request a ticket for the target's HTTP SPN, to see the Kerberos error directly
klist get HTTP/FS02.contoso.com

WinRM's Kerberos SPN is HTTP/ plus the name. Microsoft's New-PSSessionOption reference explains the classic conflict: the IncludePortInSPN option "Includes the port number in the Service Principal Name (SPN) used for Kerberos authentication", and it exists because "For example, an IIS application that allows for Kerberos authentication can require the default SPN to be registered to a user account that differs from the computer account." If setspn -Q HTTP/FS02.contoso.com returns a service account instead of the FS02 computer account, that's your Kerberos failure (our reading of Microsoft's text).

Read the authentication errors against Microsoft's Server Manager table:

Two cases look like remoting failures and aren't this one. First, Access is denied after the session opens: the account isn't in Administrators or Remote Management Users on the target, or the endpoint's Permission excludes it (Step 5). Second, the second hop: "Access to the resource on ServerC is denied, because the credentials you used to create the PowerShell Remoting session aren't passed from ServerB to ServerC." That's delegation, and Microsoft's second-hop page lists the options in order of preference, starting with CredSSP and resource-based Kerberos constrained delegation.

Step 9: Change one link (a change)

Fix the link the reads pointed at, on the side it's on, and nothing else. Write down the before value for each.

Service stopped or disabled (target). Set it back to Microsoft's server default and start it:

# Change, on the target: WinRM to Automatic and running
Set-Service -Name WinRM -StartupType Automatic
Start-Service -Name WinRM

If a policy or baseline set it to Disabled, fix it there, or the next refresh puts it back (our reading).

No listener, missing rules, or default endpoints removed (target). Enable-PSRemoting puts back Microsoft's defaults. What it does, per Microsoft: it runs Set-WSManQuickConfig, which "Starts the WinRM service.", "Sets the startup type on the WinRM service to Automatic.", "Creates a listener to accept requests on any IP address.", and "Enables a firewall exception for WS-Management communications.", then sets up and enables the session configurations and restarts WinRM.

# Change, on the target only, in an elevated Windows PowerShell window: restore the remoting defaults
Enable-PSRemoting -Force

Run it only on the target that needs it. Microsoft: "Because the configuration starts listeners, it's prudent to run it only where it's needed." Run it in Windows PowerShell if you manage Windows PowerShell endpoints; Microsoft warns that in PowerShell 7 it "only affects endpoints created with PowerShell version 6 and greater". -SkipNetworkProfileCheck does nothing on a server: Microsoft says it "has no effect on server versions of Windows, that by default, have a local subnet firewall rule for public networks."

Listener functionality broken (target, error 0x80338012 on local winrm commands). Microsoft's two steps, in order:

winrm invoke Restore winrm/Config
winrm quickconfig

Policy listener with an empty filter (GPO). Edit the listener policy in the GPO that applies to the target and fill in the IPv4 and IPv6 filters (Microsoft: "Wildcards ( *) are permitted."), or narrow them to the address ranges the servers use. Then read Get-WSManInstance winrm/config/listener -Enumerate again after the next policy refresh: ListeningOn should list the server's addresses.

Adapter in the Public profile on a domain server (target). Fix why domain detection failed: DNS servers on that adapter, a DC reachable from that adapter's network, a second adapter that shouldn't be registering. Microsoft's Set-NetConnectionProfile can change Public to Private but not to Domain: "You cannot set the DomainAuthenticated type by using this cmdlet. The server automatically sets the value of DomainAuthenticated when the network is authenticated to a domain controller." For an adapter that truly isn't on a domain network (a backup or storage VLAN with no DC), Microsoft's form is:

# Change, on the target: mark one non-domain adapter Private (not a fix for a domain adapter that should be DomainAuthenticated)
Set-NetConnectionProfile -InterfaceAlias 'Ethernet 2' -NetworkCategory Private

Rule scoped to the local subnet, and the clients really live elsewhere (target or GPO). Microsoft's example opens the Public rule to everywhere: Set-NetFirewallRule -Name 'WINRM-HTTP-In-TCP-PUBLIC' -RemoteAddress Any. Read its warning first: "The firewall rule for public networks is meant to protect the computer from potentially malicious external connection attempts. Use caution when removing this rule." We'd add only the management subnet instead (our form):

# Change, on the target: keep the local subnet and add only the admin or jump-box subnet to the Public WinRM rule
Set-NetFirewallRule -Name 'WINRM-HTTP-In-TCP-PUBLIC' -RemoteAddress LocalSubnet, 10.50.0.0/24

If the rule comes from a GPO, change it in the GPO; a local edit won't win.

Proxy in the path (client). If the WinHTTP proxy isn't needed on that machine, Microsoft's reset is netsh winhttp reset proxy, which "Resets the WinHTTP proxy setting to DIRECT." If something else on the box needs it, leave it and bypass the proxy for remoting only (our form):

# Change, for this session only, on the client: no proxy for this remoting connection
$so = New-PSSessionOption -ProxyAccessType NoProxyServer
Enter-PSSession -ComputerName FS02.contoso.com -SessionOption $so

IP address or workgroup target (client). Prefer connecting by name with Kerberos. If you must use NTLM, prefer HTTPS on the target; otherwise add the one host, not *. Microsoft: "The value that you set for the TrustedHosts item affects all users of the computer." Save the current value, then add to it:

# Read first, on the client: keep the current value somewhere you can restore it from
$curValue = (Get-Item WSMan:\localhost\Client\TrustedHosts).Value
$curValue

# Change, on the client: append one host to the existing list (Microsoft's -Concatenate form)
Set-Item WSMan:\localhost\Client\TrustedHosts -Value '10.20.30.40' -Concatenate

For Microsoft Entra-only joined machines, Microsoft's documented fix is a TrustedHosts value plus the client SPN prefix change: "To resolve this issue, set the default SPN prefix to HOST by running the following command". Apply that only to the Entra-only case it was written for.

HTTPS instead of TrustedHosts (target). Microsoft: "WinRM HTTPS requires a local computer Server Authentication certificate with a CN matching the hostname to be installed. The certificate mustn't be expired, revoked, or self-signed." With the certificate in place, run winrm quickconfig -transport:https on the target, open TCP 5986 ("Open port 5986 for HTTPS transport to work."), and connect with -UseSSL.

SPN on the wrong account (domain). If HTTP/FS02 is registered to a service account for a web app, either move that app to its own name, or use New-PSSessionOption -IncludePortInSPN for remoting so the client asks for HTTP/FS02:5985. Moving SPNs breaks the other service if you get it wrong; The Target Principal Name Is Incorrect covers reading SPNs before changing them. Microsoft's setspn note: "you should use setspn -S instead because it verifies that there are no duplicate SPNs."

Step 10: Prove it from the same client (read-only, and again after a reboot)

Run the same reads from the client that failed, by the name users type, then again after the target reboots and after the next policy refresh. A profile that drops to Public at boot, or a GPO that turns the rule back off, only shows up then:

# Read-only, on the client: the path, the WinRM answer, and a real remote command
Test-NetConnection -ComputerName FS02.contoso.com -Port 5985
Test-WSMan -ComputerName FS02.contoso.com -Authentication Default
Invoke-Command -ComputerName FS02.contoso.com -ScriptBlock { $PSVersionTable.PSVersion }

# Read-only, on the client: TrustedHosts holds only what you meant to add
Get-Item WSMan:\localhost\Client\TrustedHosts

# Read-only, on the target after a reboot: still the right profile, rule, and listener
Get-NetConnectionProfile | Select-Object -Property InterfaceAlias, NetworkCategory
Get-NetFirewallRule -Name 'WINRM*' | Select-Object -Property Name, Enabled, Profile, Action
Get-WSManInstance winrm/config/listener -Enumerate

If the tool was Windows Admin Center, run the Invoke-Command from the gateway; if it was Server Manager, refresh the server in the pool and read its Manageability column.

Pass criteria (our reading): TcpTestSucceeded True on the port you use from every client subnet that needs it; Test-WSMan -Authentication Default returns the product version; Invoke-Command by the FQDN users type returns output without -Credential inside the domain; the adapter is DomainAuthenticated (or deliberately Private for a non-domain network) after a reboot; the WinRM rule for that profile is enabled with a scope you chose; ListeningOn lists the server's addresses; TrustedHosts is empty or holds specific entries with a written reason; and the original command works from the original client.

Common false starts

When a change is justified

FAQ

What does "WinRM cannot complete the operation" mean? The client sent a WS-Management request to that name and port and got no usable answer. Microsoft's text names the checks: "Verify that the specified computer name is valid, that the computer is accessible over the network, and that a firewall exception for the Windows Remote Management service is enabled and allows access from this computer." It's reachability, not authentication; the authentication errors say "cannot process the request" instead (Steps 2 to 6).

What port does WinRM use? TCP 5985 for HTTP and TCP 5986 for HTTPS. Microsoft: "Starting in WinRM 2.0, the default listener ports configured by Winrm quickconfig are port 5985 for HTTP transport, and port 5986 for HTTPS." Older Windows used 80 and 443 (Step 3).

How do I fix "the WinRM firewall exception for public profiles limits access to remote computers within the same local subnet"? Read the target's network profile first. On a domain server, Public usually means domain detection failed on that adapter; fix that and the Domain rule applies. If clients in another subnet really need access on a Public network, add their subnet to the Public WinRM rule rather than opening it to Any (Steps 6 and 9).

Should I set TrustedHosts to * to fix WinRM? No. TrustedHosts is for NTLM connections by IP address or to workgroup machines, and Microsoft describes it as the list of hosts "for which you wish to suppress the error generated by being unable to verify the server's identity." It doesn't fix a firewall, profile, or listener problem. When you need it, add specific hosts with -Concatenate, or use HTTPS (Steps 8 and 9).

Why does Enter-PSSession work by name but not by IP address? By name inside a domain, remoting uses Kerberos. Microsoft: "Kerberos authentication doesn't support IP addresses. When you specify an IP address, NTLM authentication is used." NTLM needs HTTPS or a TrustedHosts entry on the client, plus -Credential (Step 8).

Related posts

• The Target Principal Name Is Incorrect (when the error turns into a Kerberos or SPN failure once the connection works)
• Hyper-V Live Migration Failed (Event 21502) (when remoting fails between Hyper-V hosts during migration or remote host management)
• Active Directory Replication Error 1722 (when the same firewall, profile, or NIC also blocks RPC between the same servers)

Free checklist

Setting TrustedHosts to * on every admin box, or opening the Public WinRM rule to Any on every server, before anyone read which link dropped the request, is a change made everywhere at once. So is a Conditional Access policy enforced before anyone read the report-only results. The free Conditional Access pre-enforcement checklist keeps break-glass accounts, pilot groups, and report-only review on one page:

👉 https://cashflow4375.gumroad.com/l/ujkqxb?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=winrm-cannot-complete-the-operation

If you want the rest of the Intune runway written down for the team (enrollment hygiene, compliance phasing, break/fix cards and read-only PowerShell snapshots), the Intune & M365 Admin Starter Pack pairs with this triage: https://cashflow4375.gumroad.com/l/joonf?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=winrm-cannot-complete-the-operation

Admin Pack Studio; not affiliated with Microsoft. Steps 1-8 and 10 are read-only checks: Resolve-DnsName, Test-NetConnection, Get-NetIPAddress, Test-WSMan, Get-Service, winrm enumerate, winrm get, Get-WSManInstance, Get-PSSessionConfiguration, Get-NetConnectionProfile, Get-NetFirewallRule, Get-NetFirewallAddressFilter, Get-NetFirewallPortFilter, netsh winhttp show advproxy, netsh winhttp show proxy, Get-Item on the WSMan: drive, setspn -Q, setspn -L, klist get, and Invoke-Command with a read-only script block. Setting the WinRM service startup type, starting it, running Enable-PSRemoting, winrm invoke Restore, or winrm quickconfig, changing listener policy, network profiles, firewall rule scope, the WinHTTP proxy, TrustedHosts, HTTPS listeners and certificates, the SPN prefix, or SPNs are deliberate changes. Change one link at a time, prove it from the same client after a reboot, and check current Microsoft Learn for version-specific behavior.

Not yet. Event 1135 isn't a diagnosis. It's the cluster telling you the other nodes stopped hearing a node's heartbeats on every route for longer than the threshold allows, or that its Cluster service stopped. The cause is usually below the cluster: a NIC, a driver, a team or virtual switch, a physical or virtual switch, a firewall or IPsec policy change, a network profile flip, packet loss under load, or real latency between sites. Microsoft's event text says as much: the node "was removed from the active failover cluster membership. The Cluster service on this node may have stopped. This could also be due to the node having lost communication with other active nodes in the failover cluster." It's just as plain about the forum fix: "Increasing these values to their maximum might make the event and node removal go away, it just masks the problem. It doesn't fix anything." Work through it as a chain: which node, which nodes logged it, and when → which neighbor events came with it? → is the node up, isolated, or quarantined, and was the Cluster service running? → which cluster networks carry heartbeats, and are there two? → what are the heartbeat values on this cluster, really? → what does the cluster log say about routes at that second? → what did the NIC, driver, team, and switch see? → did a firewall, IPsec, profile, or antivirus change block it? → change one link → prove it stays quiet. Get-WinEvent, Get-ClusterNode, Get-ClusterNetwork, Get-ClusterNetworkInterface, Get-Cluster, Get-ClusterLog, Get-NetAdapter, Get-NetAdapterStatistics, Get-Counter, Get-NetConnectionProfile, Get-NetFirewallRule, and Test-Cluster are the sources for that chain. Raise the thresholds "to be safe" and the next hard failure takes longer to detect, while the dropped packets that caused this one are still dropping.

If the 1135s line up with live migrations, Hyper-V Live Migration Failed (Event 21502) covers the migration network and virtual switch side. If they line up with the backup window and checkpoints fail too, Hyper-V Checkpoint Operation Failed covers the VSS and storage side. This post covers "a Windows Server failover cluster (Hyper-V, SQL Server, file server, or any other role) that logs FailoverClustering Event 1135 (cluster node was removed from the active failover cluster membership), with or without 1177 quorum loss, 1069 resource failures, VMs restarting or going unmonitored, or a node that ends up isolated or quarantined, and you need a read-first path before you raise heartbeat thresholds, evict the node, or rebuild networking."

TL;DR

• Read 1135 on every node, not one. Microsoft: "This event is logged on all nodes in the Cluster except for the node that was removed." Which nodes logged which names tells you whether one node, one site link, or the whole network was the problem (Steps 1 and 2).
• A node is removed only when all its routes are down. Microsoft's heartbeat article: "If all routes are marked down for W2K8-R2-NODE2, it's removed from active failover cluster membership and the Event 1135 that you see in the first section is logged." One working cluster network keeps a node in; that's why you want two (Step 4).
• Heartbeats are UDP 3343. Microsoft: "IPv6 is the default protocol that Failover Clustering will use for its heartbeats. The heartbeat itself is a UDP unicast network packet that communicates over Port 3343." Anything that drops, delays, or blocks that traffic for long enough removes the node (Steps 7 and 8).
• Read the thresholds before you change them. Defaults differ by Windows Server version and by Microsoft page, so Get-Cluster | Format-List *subnet* is the only number that counts for your cluster (Step 5).
• The cluster log has the route history. Get-ClusterLog -TimeSpan with -UseLocalTime shows which route was marked down and when; by default its timestamps are GMT (Step 6).
• Raising thresholds is a trade, not a fix. Microsoft: "cranking up the thresholds to high values does not fix nor resolve the transient network issue, it simply masks the problem by making health monitoring less sensitive." (Steps 5 and 9).

Where in the chain did it break?

Step 1: Capture the symptom (read-only)

Write these on the ticket first:

Then branch on what you see:

Step 2: Read the events on every node (read-only)

1135 is in the System log, from the FailoverClustering source. Microsoft's guidance is to collect them all: "Review the Event ID 1135 that you're seeing on the nodes and copy all the instances of this event." Read every node in one pass, with the events that usually come with it:

# Read-only: FailoverClustering events of interest on every node, last 7 days, oldest first
$ids   = 1006, 1069, 1127, 1129, 1130, 1135, 1146, 1177, 1554, 1592, 1653, 1672, 1673, 1681
$since = (Get-Date).AddDays(-7)
(Get-ClusterNode).Name | ForEach-Object -Process {
    $n = $_
    Get-WinEvent -ComputerName $n -FilterHashtable @{
        LogName      = 'System'
        ProviderName = 'Microsoft-Windows-FailoverClustering'
        Id           = $ids
        StartTime    = $since
    } -ErrorAction SilentlyContinue |
        Select-Object @{n='LoggedOn';e={$n}}, TimeCreated, Id, Message
} | Sort-Object TimeCreated | Format-Table -AutoSize -Wrap

Then build the who-logged-what grid (our form): one row per 1135, with the node that logged it and the node named in the message. If every row names the same node, it's Scenario A; if the names split by site, B; if they're scattered, C.

# Read-only: which node logged 1135 for which node, last 7 days
(Get-ClusterNode).Name | ForEach-Object -Process {
    $n = $_
    Get-WinEvent -ComputerName $n -FilterHashtable @{ LogName = 'System'; Id = 1135; StartTime = (Get-Date).AddDays(-7) } -ErrorAction SilentlyContinue |
        Select-Object @{n='LoggedOn';e={$n}}, TimeCreated, @{n='Removed';e={ ($_.Message -split "'")[1] }}
} | Sort-Object TimeCreated

Read the result against Microsoft's text for each event:

Two details matter when you compare with forum posts. First, read the removed node's own System log for the same minute: Microsoft's heartbeat article says that on the removed node "the Cluster Service is terminated and then restarted so it can try to rejoin the Cluster", so service stop and start events there are expected after the 1135, not before it. A service that stopped before the others logged 1135 is a different case (Step 3). Second, Microsoft's 1135 article lists the companion events it expects: 1069 for resources, "Event ID 1177 for Quorum loss", and "Event ID 1006 for Cluster service halted". If 1177 is in the list, the whole cluster went down, and that changes the urgency.

Step 3: Is the node up, isolated, or quarantined? (read-only)

Read node state and the Cluster service before you read the network:

# Read-only: name, id, and state of each node
Get-ClusterNode

# Read-only: the Cluster service on every node
Invoke-Command -ComputerName (Get-ClusterNode).Name -ScriptBlock { Get-Service -Name ClusSvc } |
    Select-Object PSComputerName, Name, Status, StartType

# Read-only: Microsoft's check that the cluster answers (Windows Server 2012 and later)
Get-ClusterResource

# Read-only: isolation and quarantine settings on this cluster
Get-Cluster | Format-List -Property Quarantine*, Resiliency*

Microsoft's 1135 article checks the service early: "Is the cluster service continuously running and available on all the nodes?" Its firewall section gives the service name, "System service name: ClusSvc".

On Windows Server 2016 and later, a node that loses membership on a cluster with VMs may not go straight to Down. Microsoft's VM compute resiliency article describes two states:

Microsoft's numbers for quarantine: "By default, a node is quarantined, if it ungracefully leaves the cluster, three times within an hour", and "No more than 25% of nodes can be quarantined at any given time". The Get-Cluster reference agrees: QuarantineThreshold "is set to 3 by default", and QuarantineDuration is "set to 7200 seconds or 2 hours by default". Microsoft also notes what isolation does to storage: for block storage (FC, FCoE, iSCSI, SAS) "The VM is placed in the Paused Critical state." while SMB-backed VMs keep running.

For the quorum branch, read the configuration once:

# Read-only: quorum type and witness for the local cluster
Get-ClusterQuorum

Microsoft's quorum guidance is short: "We recommend you always configure a witness if you're using Windows Server 2012 R2 and later." A two-node cluster with no witness that loses one node's heartbeats is the case where a single 1135 turns into an outage (our reading).

Step 4: Which cluster networks carry heartbeats? (read-only)

Microsoft's heartbeat article explains why the number of networks matters: each node tracks every route to every other node, and "If other routes are still considered to be up, W2K8-R2-NODE2 will remain as an active member." A node is removed only when the last route goes. Read the networks, their roles, and their metrics:

# Read-only: cluster networks, their state, role, and metric
Get-ClusterNetwork | Format-Table -Property Name, State, Role, Address, Metric, AutoMetric -AutoSize

# Read-only: every node's adapter on every cluster network, and its state
Get-ClusterNetworkInterface | Sort-Object Network, Node | Format-Table -AutoSize

Read the Role column against Microsoft's values for the network Role property:

Then read the metrics. Microsoft's Metric property: "the network path with the lowest sum of metrics will be used." Microsoft's network prioritization article explains what that means for traffic: the network with the lowest metric carries cluster and CSV traffic, and the second lowest carries live migration traffic. It also explains why live migration on the heartbeat network hurts: "During a live migration, large chunks of memory are copied from one server to another as quickly as possible." and "For this reason it is recommended to have a dedicated network for live migration traffic so that it does not interfere with other important network traffic."

Microsoft's validation page gives an alternative to separate networks: "you can connect your cluster nodes with one network that is constructed with teamed network adapters, redundant switches, redundant routers, or similar hardware that removes single points of failure." If that's your design, the team and the switches are what you read in Step 7.

Step 5: What are the heartbeat values on this cluster? (read-only)

Two kinds of setting decide how long the cluster waits. Microsoft's definitions: Delay "defines the frequency at which cluster heartbeats are sent between nodes", and Threshold "defines the number of heartbeats, which are missed before the cluster takes recovery action." Read the values this cluster actually has:

# Read-only: Microsoft's form for the heartbeat values
Get-Cluster | Format-List *subnet*

# Read-only: the same, plus cross-site values (Windows Server 2016 and later) and route history
Get-Cluster | Format-List -Property SameSubnet*, CrossSubnet*, CrossSite*, RouteHistoryLength

Don't compare them with a number from a forum. Microsoft's own pages give different defaults depending on the Windows Server version they describe:

The property reference pages list Windows Server 2012 as the minimum supported server. Microsoft's Azure IaaS article states the newer default: "By default, Windows Server sets the SameSubnetThreshold to 10 and SameSubnetDelay to 1000 ms." The tuning article adds a version trap: "If the Hyper-V role is installed on a Windows Server 2012 R2 Failover Cluster, the SameSubnetThreshold default will automatically be increased to 10 and the CrossSubnetThreshold default will automatically be increased to 20." Older Microsoft pages (the heartbeat article, whose examples use Windows Server 2008 R2 node names, and the Windows Server 2008 multi-site heartbeat page) still show thresholds of 5 with a range of 3 to 10. That's why the read above is the step that matters.

Microsoft's own descriptions of each value:

Then do the math (Microsoft's): "both the delay and threshold have a cumulative effect on the total health detection." Its example: a CrossSubnetDelay of 2 seconds with a CrossSubnetThreshold of 10 "means that the cluster can have a total network tolerance of 20 seconds before recovery action is taken." With the newer same-subnet defaults, the Azure article puts it as "if connectivity monitoring fails for 10 seconds, the failover Threshold is reached resulting in the unreachable that node being removed from cluster membership."

Step 6: What does the cluster log say? (read-only)

The System log tells you a node was removed. The cluster log tells you which route went first and what the heartbeat counters looked like. Generate it for a window around the 1135, in local time:

# Read-only: from an empty working folder, a cluster log from every node for the last 120 minutes, local timestamps, copied here
Get-ClusterLog -TimeSpan 120 -UseLocalTime -Destination .

Microsoft's reference for the two switches that matter here: -TimeSpan "Specifies the time span in minutes for which to generate the cluster log." and -UseLocalTime "Specifies that the time stamp for each cluster log entry uses local time. By default, the timestamp uses Greenwich Mean Time (GMT)." If the 1135 was more than a couple of hours ago, raise -TimeSpan to cover it; without -Destination, Microsoft says the log is created "in the cluster reports folder C:\Windows\Cluster\Reports on each node of the cluster." With -Destination ., Microsoft's example "creates a log file for each node of the local cluster, and copies all logs to the local folder."

Then search the logs for the route lines Microsoft's Azure thresholds article shows in its example:

# Read-only: route-down, unreachable, route history, and quorum lines in every node's cluster log
Select-String -Path .\*.log -Pattern 'Marking Route', 'unreachable', 'Route history', 'Lost quorum' |
    Select-Object Filename, LineNumber, Line

Microsoft's example log shows the sequence: "Signaled NetftRemoteUnreachable event", then "Marking Route from" one address "to" another "as down", then "All routes for route (virtual) local" and "are down", then "Route history" lines with "Heartbeats counter/threshold: 5/5", and finally, in that example, "Lost quorum". What to read (our reading):

The route history depth is RouteHistoryLength from Step 5. If you plan to raise thresholds, raise that too, or the next log won't show enough of the heartbeats before the drop.

Step 7: What did the NIC, driver, team, and switch see? (read-only)

Microsoft's heartbeat article puts hardware first among the causes: "Actual network hardware failures. If the packet is lost on the wire somewhere between the nodes, then the heartbeats fail." Read the adapters on the node that was removed and on one that wasn't:

# Read-only: adapters with Microsoft's Driver view (driver provider, version, and date)
Get-NetAdapter -Name * | Format-Table -View Driver

# Read-only: all statistics for one adapter, including discards and errors
Get-NetAdapterStatistics -Name 'Cluster1' | Format-List -Property "*"

# Read-only: Microsoft's discarded-packets counter, 12 samples 5 seconds apart, during the busy window
Get-Counter -Counter '\Network Interface(*)\Packets Received Discarded' -SampleInterval 5 -MaxSamples 12

Microsoft's guidance on that counter: "if they are any value higher than zero, then the receive buffer needs to be adjusted up for the adapter." Its heartbeat article adds how to read it: "This counter is cumulative and only increases until the server is rebooted. Seeing a large number of packets dropped here could be a sign that the receive buffers on the network card are set too low or that the server is performing slowly and can't handle the inbound traffic." So take two readings across the backup or migration window and compare them, rather than reading one number once (our reading).

Then read the team and virtual switch, if there is one, since a heartbeat route on a team or a virtual switch depends on both:

# Read-only: LBFO teams and their state
Get-NetLbfoTeam

# Read-only: Hyper-V virtual switches, and Switch Embedded Teaming members if used
Get-VMSwitch
Get-VMSwitchTeam -Name 'SETswitch'

For the VMware guest case, Microsoft's 1135 article gives the adapter settings it changes, in the guest's Device Manager under the vmxnet3 adapter's Advanced tab: "Select Small Rx Buffers and increase the value. The default value is 512 and the maximum is 8192." and "Select Rx Ring #1 Size and increase the value. The default value is 1024 and the maximum is 4096." Read the current values first:

# Read-only: the vmxnet3 adapter's advanced properties (look for the Rx buffer and ring settings)
Get-NetAdapterAdvancedProperty -Name 'Ethernet0'

Step 8: Did a firewall, IPsec, profile, or antivirus change block it? (read-only)

These are the causes that make a 1135 look random, because nothing is broken afterwards. Read each one for the minute of the event.

Firewall and ports. Microsoft's 1135 article lists what the cluster needs:

# Read-only: Microsoft's check for the cluster heartbeat rule
netsh advfirewall firewall show rule name="Failover Clusters (UDP-In)" verbose

# Read-only: every rule in the Failover Clusters group, and whether it's enabled
Get-NetFirewallRule -DisplayGroup 'Failover Clusters' | Format-Table -Property DisplayName, Enabled, Direction, Action -AutoSize

# Read-only: the IPv6 Neighbor Discovery rules Microsoft's heartbeat article names
Get-NetFirewallRule -DisplayName 'Core Networking - Neighbor Discovery*' | Format-Table -Property DisplayName, Enabled, Direction -AutoSize

Microsoft ties IPv6 to the heartbeat directly: "IPv6 is the default protocol that Failover Clustering will use for its heartbeats." Its list of causes includes IPv6 enabled with "Core Networking - Neighbor Discovery Advertisement" and "Core Networking - Neighbor Discovery Solicitation" disabled, and "If there are switches, firewalls, or routers not configured properly to allow this traffic through, you can experience issues like this." Event 1572 on a joining node says the same in its own words: "Also verify the Windows Firewall 'Failover Clusters' rules." Microsoft also says the Cluster Network Driver adapter is placed for you: "In Windows Server 2012 and Windows Server 2012 R2, the Cluster Network Driver (NETFT.SYS) adapter is automatically placed at the bottom in the binding order list."

Network profile. Microsoft: "The profile for your network connections could possibly be bouncing from Domain to Public and back to Domain again. During the transition of these changes, network I/O can be blocked." The log is Applications and Services Logs\Microsoft\Windows\NetworkProfile\Operational:

# Read-only: the current profile of each adapter
Get-NetConnectionProfile | Format-Table -Property InterfaceAlias, Name, NetworkCategory -AutoSize

# Read-only: find the exact name of the NetworkProfile Operational log on this node
$np = (Get-WinEvent -ListLog '*NetworkProfile*Operational').LogName

# Read-only: its events around the 1135 (set the window to your event time)
Get-WinEvent -FilterHashtable @{
    LogName   = $np
    StartTime = (Get-Date '2026-10-13 01:00')
    EndTime   = (Get-Date '2026-10-13 01:15')
} -ErrorAction SilentlyContinue | Select-Object TimeCreated, Id, Message

IPsec policy refresh. Microsoft: "during an IPSec group policy update all IPsec Security Associations (SAs) are torn down by Windows Firewall with Advanced Security (WFAS). While this is happening, all network connectivity is blocked." If renegotiation waits on Active Directory, the block can outlast the threshold. A 1135 that lines up with the Group Policy refresh interval on nodes with connection security rules is this case until proven otherwise (our reading); Event 1592's text names the same thing: "communication temporarily being blocked by a firewall or connection security policy update."

Antivirus. Microsoft: "Anti-virus software could be interfering with this process also." Its 1135 article gives the exclusions it expects, and they're worth reading against your antivirus policy:

Microsoft's caution about testing it: "Just disabling the antivirus software is insufficient in most cases. Even if you disable the antivirus software, the filter driver is still loaded when you restart the computer."

Validation. Microsoft's 1135 article puts this early: "A validation and the network tests would be recommended as one of the initial troubleshooting steps to ensure there are no configuration issues that might be a cause for problems." Run only the network tests on a live cluster, and list the test names on your version first:

# Read-only: list the validation tests and categories on this version
Test-Cluster -List

# Read-only for networking: run the network communication and firewall tests only, with a named report
Test-Cluster -Include 'Validate Network Communication', 'Validate Windows Firewall Configuration' -ReportName C:\Temp\cluster-network-validation

Microsoft's descriptions of those two tests: "This test validates that tested servers can communicate with acceptable latency on all networks. The test also checks to see if there are redundant communication paths between all servers." and "This test validates that Windows Firewall is configured correctly for failover clustering on the tested servers." In the report, Microsoft's example latency failure reads "the ping latency is greater than the maximum allowed 2000 ms". Keep storage tests out of a run on a live cluster: Microsoft's -Disk and -Force notes describe disks taken offline for the duration of the storage tests.

Step 9: Change one link (a change)

Fix only what Steps 2 to 8 showed, then read the result before changing anything else. If the change touches a node, drain it first:

# A change: move this node's roles off gracefully before you touch its NICs, drivers, or switch ports
Suspend-ClusterNode -Name HV02 -Drain -Wait

# After the change and a read of Steps 4 and 7 on it: bring it back, with its roles moved back now
Resume-ClusterNode -Name HV02 -Failback Immediate

Microsoft's -Drain: "Specifies that all of the workloads are gracefully moved to other nodes while maintaining the highest levels of availability and using the best placement logic." -Failback takes NoFailBack, Immediate, or Policy.

Drivers, firmware, and buffers (Step 7). Update the NIC driver and firmware to the versions your server and NIC vendors recommend, on one drained node at a time, and raise receive buffers to the manufacturer's recommended values. For a VMware guest cluster, Microsoft's direction is to "gradually increase the number of buffers in the guest operating system and verify." Re-read the discard counter under the same load afterwards.

Network roles and metrics (Step 4). Give the cluster a second path for heartbeats, and keep storage traffic off cluster communication. Microsoft's Role values: 1 for internal cluster communication, 3 for cluster and client, 0 for not used by the cluster. Microsoft's restriction: "If this network is the only one used for internal cluster communication, the Role property may not be changed to ClusterNetworkRoleNone or ClusterNetworkRoleClientAccess."

# A change: let a second network carry cluster communication (Role 1); use your own network name
(Get-ClusterNetwork -Name 'Cluster Network 2').Role = 1

# A change, Microsoft's form: rank a network by metric (lower is preferred for cluster traffic)
$n = Get-ClusterNetwork 'Cluster Network 3'
$n.Metric = 1050

Microsoft's network prioritization article shows how to undo a manual metric: set AutoMetric back to $true on that network.

Firewall, IPv6, profile, IPsec (Step 8). Re-enable the Failover Clusters rules and the Neighbor Discovery rules in the policy that disabled them, not by hand on one node (our reading). Fix the reason a profile drops to Public (usually domain detection at boot or a DNS path; our reading). For IPsec, Microsoft's article points at the refresh and the authentication delay; exempt cluster traffic in the connection security rules or fix the DC path the renegotiation waits on (our reading).

Antivirus (Step 8). Add Microsoft's exclusions from the table above in the antivirus policy for every node, then reboot one drained node at a time so filter drivers load with the new policy.

Quarantine (Step 3). Once the cause is fixed, clear the quarantine with Microsoft's command instead of waiting out the two hours:

# A change: start the Cluster service on the node and clear its quarantine
Start-ClusterNode -Name HV02 -ClearQuarantine

Thresholds, last and in moderation (Step 5). Raise them only for the case Microsoft describes (stretched clusters, Azure IaaS, or older versions still on 5), and only after the network read. Microsoft: "In general, continuing to send frequent heartbeats but having greater thresholds is the preferred method." Its forms:

# A change, Microsoft's form: raise the same-subnet threshold (example value from Microsoft's article)
(Get-Cluster).SameSubnetThreshold = 20

# A change, Microsoft's form: keep twice as much route history as the threshold
(Get-Cluster).RouteHistoryLength = 40

# Re-read
Get-Cluster | Format-List -Property SameSubnet*, CrossSubnet*, RouteHistoryLength

Microsoft's tuning article: "These setting can be set while the cluster is up and running with no downtime and will take effect immediately with no need to reboot or restart the cluster." Its limit: "it is generally not recommended to exceed the TCP reconnect timeouts. Generally, this means not going above ~20 seconds." So a 1000 ms delay with a threshold of 20 is about the ceiling for most workloads, and its warning still applies: "increasing resiliency to network hiccups comes at the cost of increased downtime when a hard failure occurs." (The RouteHistoryLength value of 40 is the reference maximum; 40 is twice 20.)

Step 10: Prove it stays quiet (read-only, through the same load window)

Wait through the window that produced the 1135 (the backup, the migration wave, the patch night, the Group Policy refresh), then run the reads again:

# Read-only: no new 1135, 1127, 1129, 1130, 1177, or 1672 on any node since the change
$since = Get-Date '2026-10-20 18:00'
foreach ($n in (Get-ClusterNode).Name) {
    Get-WinEvent -ComputerName $n -FilterHashtable @{ LogName = 'System'; Id = 1127, 1129, 1130, 1135, 1177, 1672; StartTime = $since } -ErrorAction SilentlyContinue |
        Select-Object @{n='LoggedOn';e={$n}}, TimeCreated, Id
}

# Read-only: nodes, networks, and interfaces all Up
Get-ClusterNode
Get-ClusterNetwork | Format-Table -Property Name, State, Role, Metric -AutoSize
Get-ClusterNetworkInterface | Sort-Object Network, Node | Format-Table -AutoSize

# Read-only: the cluster log for the window, with no route marked down
Get-ClusterLog -TimeSpan 120 -UseLocalTime -Destination .
Select-String -Path .\*.log -Pattern 'Marking Route' | Select-Object Filename, LineNumber, Line

# Read-only: discards across the same window, compared with the reading before the change
Get-Counter -Counter '\Network Interface(*)\Packets Received Discarded' -SampleInterval 5 -MaxSamples 12

# Read-only: the network validation tests again
Test-Cluster -Include 'Validate Network Communication', 'Validate Windows Firewall Configuration' -ReportName C:\Temp\cluster-network-validation-after

Pass criteria (our reading): no 1135 on any node through at least one full cycle of the window that caused it (a week, for a weekly backup); no route marked down in the cluster log for that window; every node, network, and interface Up; discarded packets flat across the busy window; at least two networks allowed for cluster communication; the network validation tests pass without latency or firewall warnings; no node quarantined; and heartbeat values at the defaults for your version, or raised in moderation with RouteHistoryLength at twice the threshold and a written reason.

Common false starts

When a change is justified

FAQ

What does Event 1135 "Cluster node was removed from the active failover cluster membership" mean? The other nodes stopped receiving the node's heartbeats on every cluster network for longer than the threshold allows, or its Cluster service stopped. Microsoft's text: "The Cluster service on this node may have stopped. This could also be due to the node having lost communication with other active nodes in the failover cluster." It's logged on the nodes that remain, not on the removed one (Steps 1 and 2).

What port does the failover cluster heartbeat use? UDP 3343. Microsoft: "The heartbeat itself is a UDP unicast network packet that communicates over Port 3343." TCP 3343 is used when a node joins, and IPv6 is the default heartbeat protocol, so the Neighbor Discovery rules matter too (Step 8).

What are the default SameSubnetDelay and SameSubnetThreshold values? It depends on the version and the page you read. Microsoft's property reference lists SameSubnetDelay 1000 ms and SameSubnetThreshold 5; its tuning article lists 10 heartbeats for Windows Server 2016, and its Azure article says Windows Server sets SameSubnetThreshold to 10 by default. Read your own with Get-Cluster | Format-List *subnet* (Step 5).

Should I increase the cluster heartbeat thresholds to stop Event 1135? Only after the network read, and in moderation. Microsoft says raising them "simply masks the problem by making health monitoring less sensitive", and that it's generally "not going above ~20 seconds". The real cases are stretched clusters, Azure IaaS, and older versions still on 5 (Steps 5 and 9).

Why is my cluster node quarantined, and how do I clear it? On Windows Server 2016 and later, "a node is quarantined, if it ungracefully leaves the cluster, three times within an hour", for 7200 seconds by default. Fix the cause of the 1135s, then run Start-ClusterNode -ClearQuarantine, the command Microsoft's quarantine event names (Steps 3 and 9).

Related posts

• Hyper-V Live Migration Failed (Event 21502) (when the 1135s line up with live migrations on the same hosts and networks)
• Hyper-V Checkpoint Operation Failed (when the 1135s line up with the backup window and checkpoints fail too)
• Active Directory Replication Error 1722 (when the same nodes also fail RPC to domain controllers through the same firewall or NIC)

Free checklist

Raising heartbeat thresholds on every cluster, or evicting the node, before anyone read which route dropped and why, is a change made everywhere at once. So is a Conditional Access policy enforced before anyone read the report-only results. The free Conditional Access pre-enforcement checklist keeps break-glass accounts, pilot groups, and report-only review on one page:

👉 https://cashflow4375.gumroad.com/l/ujkqxb?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=failover-cluster-event-1135

If you want the rest of the Intune runway written down for the team (enrollment hygiene, compliance phasing, break/fix cards and read-only PowerShell snapshots), the Intune & M365 Admin Starter Pack pairs with this triage: https://cashflow4375.gumroad.com/l/joonf?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=failover-cluster-event-1135

Admin Pack Studio; not affiliated with Microsoft. Steps 1-8 and 10 are read-only checks: Get-WinEvent, Get-ClusterNode, Get-ClusterResource, Get-ClusterQuorum, Get-Cluster, Get-ClusterNetwork, Get-ClusterNetworkInterface, Get-ClusterLog, Select-String, Get-NetAdapter, Get-NetAdapterStatistics, Get-NetAdapterAdvancedProperty, Get-Counter, Get-NetLbfoTeam, Get-VMSwitch, Get-VMSwitchTeam, Get-NetFirewallRule, netsh advfirewall firewall show rule, Get-NetConnectionProfile, Get-Service through Invoke-Command, and Test-Cluster with only the network tests (it writes a report file). Get-ClusterLog writes log files to the destination folder. Draining and resuming nodes, updating drivers and firmware, changing adapter buffers, network roles, and metrics, changing firewall, IPsec, and antivirus policy, clearing quarantine, and changing heartbeat thresholds or route history are deliberate changes. Change one link at a time, prove it through the same load window, and check current Microsoft Learn for version-specific behavior.

Not that way. Event 4013 usually isn't a broken DNS server. A DNS server that hosts AD-integrated zones waits after boot until AD DS finishes its first inbound replication, and AD DS can't replicate until it can resolve its partners' names through DNS. When the DC points only at itself, when its partners are rebooting too, offline, or long gone, or when the network isn't up yet, the two wait on each other. Microsoft says it plainly: "DNS Event ID 4013 in the DNS event log indicates that DNS service startup was delayed. It's because inbound replication of Active Directory partitions hadn't occurred." It's just as plain about the registry shortcut: "This configuration change isn't recommended for use in production environments, or in any environment on an ongoing basis." Pointing a DC with partners at itself only is the other half of the forum advice, and Microsoft's 4013 article lists that exact setup among the conditions that make 4013 worse. Work through it as a chain: which DC, which events, since when → is it a boot-time wait that ends, or one that never ends? → what are the DNS Server and AD DS services actually doing? → which DNS servers does this DC ask, and do they answer? → can it resolve its replication partners? → is inbound replication happening, and from whom? → is it the only DC, a role holder, or a recovery case? → is it really 4015, 4000, or 4007? → change one link → prove a clean boot. Get-WinEvent, Get-Service, Get-DnsClientServerAddress, Resolve-DnsName, Test-NetConnection, repadmin /showrepl, repadmin /replsummary, Get-ADReplicationPartnerMetadata, Get-ADReplicationFailure, dcdiag /test:dns, and Get-ItemProperty are the sources for that chain. Set the registry value "to be safe" and you may bring up a DC that serves stale zones and acts on stale role data, and the next reboot of two DCs at once still hangs the same way.

If the partners show replication error 1722 once you can read them, Active Directory Replication Error 1722 covers the DNS, RPC, and firewall side. If member servers log Netlogon 5719 in the same boot window, Netlogon Event ID 5719 covers the client view of the same timing problem. This post covers "a domain controller that runs DNS with AD-integrated zones and logs DNS Server Event 4013 (the DNS server is waiting for AD DS initial synchronization), with or without 4015, 4000, or 4007, a slow 'Preparing network connections' boot, DNS Manager or Active Directory Users and Computers failing to connect, and you need a read-first path before you change the registry, point the DC at itself, or restart services in a loop."

TL;DR

• A short 4013 run after boot can be normal. Microsoft: "The delay won't occur if Active Directory has completed its initial synchronization during Windows startup." The event "will be logged every two minutes until AD DS has signaled that the initial synchronization has successfully completed", so count the events and read how long they lasted (Step 2).

• DNS waits for AD DS, and AD DS needs DNS to find its partners. Replication "is delayed until it can resolve the CNAME GUID of its source domain controller to an IP address on the DNS servers used by the destination domain controller for name resolution." (Steps 4 and 5).

• A DC with partners shouldn't point only at itself. Microsoft: "Domain controllers hosting AD-integrated DNS zones shouldn't point to a single domain controller and especially only to themselves as preferred DNS for name resolution." A forest with one DC is the exception (Steps 4 and 7).

• Stale or offline partners keep the wait going. Read repadmin /showrepl and remove the metadata of DCs that are really gone; don't wait on them (Step 6).

• The registry value is a temporary emergency tool. Repl Perform Initial Synchronizations set to 0 "should be used only in critical situations to resolve temporary and specific problems. The default setting should be restored after such problems are resolved." (Steps 7 and 9).

• 4015 is a different problem. It's "a critical error from the Active Directory" with an error code that picks the cause: RODC, permissions, or too many orphaned records (Step 8).

Where in the chain did it break?

Step 1: Capture the symptom (read-only)

Write these on the ticket first:

Then branch on what you see:

Step 2: Read the events and how long they lasted (read-only)

4013 is in the DNS Server log (Event Viewer: Applications and Services Logs > DNS Server), not the System log. Read it with its neighbors, and read the Directory Service log for the same window:

# Read-only: DNS Server events since the last boot, oldest first
$boot = (Get-CimInstance -ClassName Win32_OperatingSystem).LastBootUpTime
Get-WinEvent -FilterHashtable @{
    LogName   = 'DNS Server'
    Id        = 4000, 4007, 4011, 4013, 4015
    StartTime = $boot
} -ErrorAction SilentlyContinue |
    Sort-Object TimeCreated |
    Select-Object TimeCreated, Id, LevelDisplayName, Message

# Read-only: first and last 4013 since boot, and how many
$w = Get-WinEvent -FilterHashtable @{ LogName = 'DNS Server'; Id = 4013; StartTime = $boot } -ErrorAction SilentlyContinue
if ($w) {
    $first = ($w | Sort-Object TimeCreated | Select-Object -First 1).TimeCreated
    $last  = ($w | Sort-Object TimeCreated | Select-Object -Last 1).TimeCreated
    [pscustomobject]@{ Boot = $boot; Count = @($w).Count; First = $first; Last = $last; Minutes = [int]($last - $first).TotalMinutes }
}

# Read-only: DNS lookup failures for replication partners in the same window
Get-WinEvent -FilterHashtable @{
    LogName      = 'Directory Service'
    ProviderName = 'Microsoft-Windows-ActiveDirectory_DomainService'
    Id           = 2087, 2088
    StartTime    = $boot
} -ErrorAction SilentlyContinue |
    Select-Object TimeCreated, Id, Message

Read the result against Microsoft's text for each event:

Two details matter when you compare with forum posts. First, the waiting text is the one on Microsoft's event reference page (symbolic name DNS_EVENT_DS_OPEN_WAIT), and it ends: "This event will be logged every two minutes until AD DS has signaled that the initial synchronization has successfully completed." Microsoft's current troubleshooting article for 4013 shows the older "unable to open the Active Directory" wording instead. Both are 4013 from the DNS server; search for the event ID, not only the text. Second, the waiting text gives a direction of its own: "If events in the AD DS event log indicate that there is a problem with DNS name resolution, consider adding the IP address of another DNS server for this domain to the DNS server list in the Internet Protocol properties of this computer."

Microsoft's 4013 article lists event data values it can carry, and notes they "may not be logged":

Then read the timing (our reading):

Microsoft's own test explains why the wait can be that long: when the DNS Server service isn't available, "NTDS will need to send out 10 DNS queries to resolve the IP address", and with the default DNSQueryTimeouts "DNS client will wait 12 (1 + 1 + 2 + 4 + 4) seconds for the DNS server response." Multiply by the partitions (schema, configuration, domain, and the two DNS application partitions) and the minutes add up.

Step 3: What are the services doing? (read-only)

Read the services the chain depends on, and whether the DC is advertising yet. DNS Server, AD DS, Netlogon, and KDC are the ones that matter here:

# Read-only: the services behind DNS and AD DS on this DC
Get-Service -Name DNS, NTDS, Netlogon, Kdc |
    Select-Object Name, DisplayName, Status, StartType

# Read-only: which DC does DC locator return from here? (this DC itself once it advertises; our reading)
nltest /dsgetdc:contoso.com

Then try the consoles once, without restarting anything. Microsoft's example scenarios show what a DC stuck in this state looks like:

Restarting the DNS Server service while AD DS is still synchronizing doesn't shorten the wait (our reading): Microsoft's resolution for the event says "Ensure that AD DS is functioning properly, troubleshoot any problems, and then restart the DNS Server service." Fix the AD DS side first.

Step 4: Which DNS servers does this DC ask? (read-only)

This is the step that most often explains a long 4013. Read every adapter's DNS server list, in order:

# Read-only: DNS servers per adapter, in the order the DC uses them
Get-DnsClientServerAddress -AddressFamily IPv4 |
    Where-Object { $_.ServerAddresses } |
    Select-Object InterfaceAlias, InterfaceIndex, ServerAddresses

# Read-only: the same from ipconfig, with suffix and registration details
ipconfig /all

Read the list against Microsoft's guidance for DCs:

Microsoft's summary of what a DC's DNS servers must do is short: they should be servers that "Are available at Windows startup." and that "Host, forward, or delegate the _msdcs.<forest root domain> and primary DNS suffix zones for current and potential source domain controllers." Two client details from Microsoft's best-practice article explain why a good-looking list can still fail: the client uses the preferred server until it fails to respond, and "Only a failure to respond will cause the DNS client to switch Preferred DNS servers; receiving an authoritative but incorrect response does not cause the DNS client to try another server."

Then prove each listed server answers for the AD zones right now. Run this from the DC (or from another machine, against the same servers):

# Read-only: does each DNS server in the DC's list answer on port 53 and hold the zones?
$servers = '10.0.0.11', '10.1.0.10', '127.0.0.1'
foreach ($s in $servers) {
    $tcp = Test-NetConnection -ComputerName $s -Port 53 -WarningAction SilentlyContinue
    $soa = Resolve-DnsName -Name 'contoso.com' -Type SOA -Server $s -DnsOnly -ErrorAction SilentlyContinue
    $msd = Resolve-DnsName -Name '_msdcs.contoso.com' -Type SOA -Server $s -DnsOnly -ErrorAction SilentlyContinue
    [pscustomobject]@{ Server = $s; Tcp53 = $tcp.TcpTestSucceeded; DomainSOA = [bool]$soa; MsdcsSOA = [bool]$msd }
}

Step 5: Can this DC resolve its replication partners? (read-only)

AD DS finds a source DC by the GUID-based alias in _msdcs. Microsoft: "You should ensure that domain controllers can successfully resolve the guided CNAME records to host records of current and potential source domain controllers." Read the partner's CNAME from replication data, then resolve it from each server in this DC's list:

# Read-only: inbound partners with their fully qualified CNAME (look for the _msdcs name in the output)
repadmin /showrepl DC01 /verbose

# Read-only: resolve one partner's GUID CNAME and host record from a specific DNS server
$guidName = 'dded5a29-fc25-4fd8-aa98-7f472fc6f09b._msdcs.contoso.com'
Resolve-DnsName -Name $guidName -Type CNAME -Server 10.0.0.11 -DnsOnly
Resolve-DnsName -Name 'DC02.contoso.com' -Type A -Server 10.0.0.11 -DnsOnly

Microsoft documents /verbose as showing the "fully qualified CNAME, invocation ID, replication flags, and update sequence number (USN) values" for each source partner, which is the name you need. If AD DS isn't answering on this DC yet, run repadmin /showrepl against a healthy DC instead and read its view of this DC.

Then run Microsoft's DNS tests from the 2087 procedure, against this DC and against the partner:

# Read-only: basic DNS health (connectivity, services, client config, records, zone and SOA)
dcdiag /test:dns /s:DC01 /DnsBasic /f:C:\Temp\dcdiag-dns-DC01.txt

# Read-only: are the A, CNAME, and SRV records for DC location registered?
dcdiag /test:dns /s:DC02 /DnsRecordRegistration

Microsoft's description of the basic test covers what you'd otherwise check by hand: it confirms the DNS Client, Net Logon, KDC, and DNS Server services are running, that "DNS servers on all adapters are reachable", and that each DC's host record "is registered on at least one of the DNS servers that is configured on the client." /DnsRecordRegistration checks "the address (A), canonical name (CNAME) and well-known service (SRV) resource records". Note that /DnsDynamicUpdate is not read-only: Microsoft says it performs "registration of a test record (_dcdiag_test_record)", deleted automatically after the test.

Step 6: Is inbound replication happening, and from whom? (read-only)

Microsoft's cause statement is about partners: these DCs "try to inbound replicate all locally held directory partitions during Windows startup, as part of an initial synchronization or init sync." Read who they are and whether each one works:

# Read-only: inbound replication status for this DC, by partition and partner
repadmin /showrepl DC01

# Read-only: only the failing partnerships, for every DC in the forest
repadmin /showrepl * /errorsonly

# Read-only: failures and largest deltas, summarized per source and destination
repadmin /replsummary * /bysrc /bydest /sort:delta

The PowerShell view gives the same data as objects, once AD DS on the DC you query is answering:

# Read-only: inbound partners of DC01, last success and last result
Get-ADReplicationPartnerMetadata -Target DC01 -PartnerType Inbound |
    Select-Object Partner, Partition, LastReplicationAttempt, LastReplicationSuccess, LastReplicationResult

# Read-only: current replication failures recorded for DC01
Get-ADReplicationFailure -Target DC01 |
    Select-Object Partner, FailureCount, FirstFailureTime, LastError

# Read-only: every DC the domain knows about, with site and address
Get-ADDomainController -Filter * |
    Select-Object HostName, Site, IPv4Address, IsGlobalCatalog, OperationMasterRoles

Read the partners against Microsoft's list of conditions that keep initial sync from finishing:

Microsoft's guidance for the gone-for-good case is direct: "Remove references to stale domain controllers." and "Make offline or non-functioning domain controllers operational." Its 2087 article adds the check before cleanup: confirm the source DC is really down, with net view \\<SourceDomainControllerName> (which "displays the Netlogon and SYSVOL shares, indicating that the server is functioning as a domain controller.").

Step 7: Is it the only DC, a role holder, or a recovery case? (read-only)

Three situations look like 4013 trouble but need their own reading.

The only DC in the forest. Microsoft's best-practice article says the first and only DC that runs DNS should point at itself: "Do not list any other DNS servers until you have another domain controller hosting DNS in that domain." With no partner, there's nothing to replicate from, so a long 4013 on a lone DC usually means its copy of AD DS still lists a partner (our reading): a second DC that was removed without demotion, or a DC from a test you forgot. Run Step 6's Get-ADDomainController -Filter * and repadmin /showrepl on it; any partner listed there is one it's waiting on.

A role holder. Operations masters have their own initial-sync rule, and Microsoft's 4013 article keeps the two apart: "The initial sync requirements required for FSMO roles to become operational is different from the initial sync discussed in this article, where Active Directory must inbound replicate to start the DNS Server service immediately." The role rules, from Microsoft's FSMO article:

So a role holder that logs 4013 may also be refusing role operations, such as RID pool requests, for the same reason (our reading); fix the replication, not the roles.

The registry value. Read it on every DC you touch, because an old fix often left it behind:

# Read-only: is the initial synchronization bypass set? (absent means the default)
Get-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Services\NTDS\Parameters' -Name 'Repl Perform Initial Synchronizations' -ErrorAction SilentlyContinue |
    Select-Object 'Repl Perform Initial Synchronizations'

Microsoft's 4013 article gives the bypass as the value Repl Perform Initial Synchronizations, type REG_DWORD, data 0, under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTDS\Parameters, and then says "The use of Repl Perform Initial Synchronizations should be used only in critical situations to resolve temporary and specific problems." The forest recovery guide is where it's meant to be used: for a restored DC that holds operations master roles, so AD DS isn't unavailable until it has replicated a writeable partition. That guide's restore instruction is "After the forest is recovered completely, you can reset the value of this entry to 1, which requires a domain controller that restarts and holds operations master roles to have successful AD DS inbound and outbound replication with its known replica partners before it advertises itself as domain controller and starts providing services to clients."

Step 8: Is it really 4015, 4000, or 4007? (read-only)

These come from the same DNS Server and AD DS relationship but aren't boot waits. Read the full text of the most recent ones:

# Read-only: the newest 4015, 4000, and 4007 with their full message, including the extended error
Get-WinEvent -FilterHashtable @{ LogName = 'DNS Server'; Id = 4015, 4000, 4007 } -MaxEvents 10 -ErrorAction SilentlyContinue |
    Select-Object TimeCreated, Id, Message |
    Format-List

Microsoft's 4015 text is "The DNS server has encountered a critical error from the Active Directory. Check that the Active Directory is functioning properly. The extended error debug information (which may be empty) is", followed by the error. Its 4015 article sorts the error into three cases:

The read-only checks for each branch:

# Read-only (00002095, on the RODC): which writable DC does DC locator return?
nltest /dsgetdc:contoso.com /WRITABLE /AVOIDSELF /TRY_NEXT_CLOSEST_SITE /DS_6

# Read-only (0000051B): members of the domain's Users group
net localgroup users /domain

# Read-only (0000051B): owner of each AD-integrated zone object in the domain DNS partition
Get-ADObject -SearchBase 'CN=MicrosoftDNS,DC=DomainDnsZones,DC=contoso,DC=com' -LDAPFilter '(objectClass=dnsZone)' -Properties nTSecurityDescriptor |
    Select-Object Name, @{n='Owner';e={$_.nTSecurityDescriptor.Owner}}

# Read-only (ADMIN_LIMIT_EXCEEDED): record counts per name in the forest DNS partition, to a file
repadmin /showattr . "CN=MicrosoftDNS,DC=ForestDnsZones,DC=contoso,DC=com" /subtree /filter:"(objectclass=dnsnode)" /atts:"dnsRecord" /allvalues > C:\Temp\dns_ForestDnsZones.txt

Microsoft's own route to the zone in the 0000051B case is Field Engineering diagnostic logging and Directory Service Event 1644 at the time of the 4015, then ADSI Edit: "Right-click the zone object, and then select Properties > Security > Advanced." and "Make sure that Owner is set to SYSTEM." Field Engineering logging is a registry change; turn it off afterwards, as Microsoft's steps do. For the record-count case, Microsoft explains why the values pile up: "However, when you demote a DC that acts as a DNS server, Active Directory doesn't automatically clean up the related DNS objects and attributes." Its large-zone article puts the ceiling at "approximately 1,200 values that can be associated with a single object in Windows Server 2022 and earlier versions."

4000 and 4007 mean zones didn't load, and Microsoft's article for that pair names a different cause: "This issue happens when that particular DC/DNS server has lost its Secure channel with itself or PDC." It adds: "This issue can also happen in a single DC environment where that DC/DNS server holds all the FSMO roles and is pointing to itself as Primary DNS server." Microsoft's symptoms include that "You notice that the DNS Server service is up and running." while DNS Manager gives Access Denied, which is how you tell it from a 4013 wait (our reading). Microsoft's fix for that case is in Step 9.

Step 9: Change one link, then let initial sync finish (a change)

Fix only what Steps 2 to 8 showed, then read the result before changing anything else.

The DC's DNS server list (Step 4). For a hub-site DC with partners, point it at in-site partners first and itself last, as Microsoft describes. Use the addresses from your own design:

# A change: partner DNS servers first, this DC's own loopback last
Set-DnsClientServerAddress -InterfaceAlias 'Ethernet' -ServerAddresses '10.0.0.11', '10.0.0.12', '127.0.0.1'

# Re-read it, then clear the resolver cache and re-register, as Microsoft's article says to after any change
Get-DnsClientServerAddress -InterfaceAlias 'Ethernet' -AddressFamily IPv4
ipconfig /flushdns
ipconfig /registerdns

Microsoft's rule for that last step: "If you change any DNS client settings, you must clear the DNS resolver cache and register the DNS resource records." For a branch DC, the order is hub first, then in-site, then itself (Step 4's table). For a lone DC, its own address is correct; don't add a server that doesn't host its zones.

The reboot plan (Step 4). If the 4013 came from DCs that point at each other and rebooted together, the fix is the schedule as much as the list. Microsoft: "Stagger the reboots of DNS servers in your enterprise when possible." and, among its single points of failure, "Placing all DNS servers on guest virtual machines on the same physical host computer". Put DCs in different patch waves, and keep at least one DNS server that each DC lists up while the others restart.

A partner that's gone for good (Step 6). Confirm it's down, then clean up its metadata. Microsoft's simplest path: deleting the DC's computer account from the Domain Controllers OU in Active Directory Users and Computers, where "the cleanup of server metadata is performed automatically." Then remove the DNS records it left behind, by record data, the way Microsoft's 4015 article does:

# A change: preview, then remove one gone DC's NS record from a zone (Microsoft's form, with -WhatIf added)
Remove-DnsServerResourceRecord -ZoneName 'contoso.com' -RRType 'NS' -Name '@' -RecordData 'dc9.contoso.com' -WhatIf
Remove-DnsServerResourceRecord -ZoneName 'contoso.com' -RRType 'NS' -Name '@' -RecordData 'dc9.contoso.com'

A partner whose records are missing (Step 5). On that partner, re-register. Microsoft: "The Net Logon service on a domain controller registers the DNS resource records that are required for the domain controller to be located on the network. The DNS Client service registers the host (A) resource record that the alias (CNAME) record points to."

# A change, on the partner DC: re-register its locator records and its host record
Restart-Service -Name Netlogon
ipconfig /registerdns

Microsoft's next instruction is "Wait 15 minutes, and then review events in Event Viewer to ensure proper registration of the resource records." Then re-run Step 5's Resolve-DnsName from the DC that logged 4013.

Pull replication once names resolve. Then restart DNS only after AD DS has synchronized:

# A change: synchronize DC01 from all its partners, all partitions, across sites, DNs in messages
repadmin /syncall DC01 /A /e /d

# When repadmin /showrepl DC01 is clean, restart the DNS Server service
Restart-Service -Name DNS

A known bad boot (Steps 4 and 6). When you must boot a DC into a setup you know will stall (its partners are down for a planned outage, for example), Microsoft's documented path is to stop DNS from holding up the boot: "Set the DNS Server service startup value to manual.", then "Reboot, wait for the domain controller to advertise.", then "Restart the DNS Server service." It explains why that works: "If the service startup value for DNS Server service is set to manual, Active Directory doesn't wait for the DNS Server service to start."

# A change, before the planned boot: DNS Server service to manual
Set-Service -Name DNS -StartupType Manual

# After the boot, once nltest /dsgetdc returns this DC: start DNS, then put the startup type back
Start-Service -Name DNS
Set-Service -Name DNS -StartupType Automatic

The registry value (Step 7). If it's 0 from an old fix, restore the default once replication is proven, as Microsoft's recovery guide does:

# A change: restore initial synchronization (Microsoft's recovery guide resets the value to 1)
Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Services\NTDS\Parameters' -Name 'Repl Perform Initial Synchronizations' -Value 1

Set it to 0 only in the situation Microsoft describes, a critical and temporary one, with a written plan to restore it, and expect the next reboot to be the real test.

4015 with 0000051B (Step 8). Fix the zone owner in ADSI Edit as Microsoft describes, or restore the Users group membership with Microsoft's command, then wait for replication and restart the DNS Server service on all the DCs in the domain, as Microsoft's steps say:

net localgroup users "NT AUTHORITY\Authenticated Users" /add /domain

4000 and 4007 (Step 8). Microsoft's resolution starts with the DNS list ("configure the server experiencing the issue to point to other active DNS server in TCP/IP properties"), then: "Stop the KDC service on the DC experiencing the issue.", reset the DC's machine account password against the PDC with netdom resetpwd, and "Once the command executes, reboot the server." Microsoft's command, with your PDC and admin account in place of its placeholders:

netdom resetpwd /server:DC02.contoso.com /userd:CONTOSO\admin1 /passwordd:*

That's a secure-channel repair, not a fix for 4013 (our reading); use it only when Step 8 showed 4000 or 4007 with the zones not loading.

Step 10: Prove a clean boot (read-only, after a planned reboot)

Reboot the DC in a window, alone, with its partners up, and then run the reads again:

# Read-only: DNS Server events since this boot; Microsoft's verify step wants none from 4000 to 4019
$boot = (Get-CimInstance -ClassName Win32_OperatingSystem).LastBootUpTime
Get-WinEvent -FilterHashtable @{ LogName = 'DNS Server'; Id = 4000..4019; StartTime = $boot } -ErrorAction SilentlyContinue |
    Select-Object TimeCreated, Id, Message

# Read-only: the events Microsoft's verify step expects to see
Get-WinEvent -FilterHashtable @{ LogName = 'DNS Server'; Id = 4523, 4524; StartTime = $boot } -ErrorAction SilentlyContinue |
    Select-Object TimeCreated, Id

# Read-only: replication and DNS health after the boot
repadmin /showrepl DC01 /errorsonly
dcdiag /test:dns /s:DC01 /DnsBasic

# Read-only: every DNS server returns the DC locator records and the zones load here
foreach ($s in '10.0.0.10', '10.0.0.11', '10.0.0.12') {
    Resolve-DnsName -Name '_ldap._tcp.dc._msdcs.contoso.com' -Type SRV -Server $s -DnsOnly -ErrorAction SilentlyContinue |
        Select-Object @{n='Server';e={$s}}, Name, NameTarget, Port
}
Get-DnsServerZone -ComputerName DC01 | Select-Object ZoneName, ZoneType, IsDsIntegrated

Microsoft's verify step for the event is: "Ensure that Event IDs 4523 and 4524 are being logged and that no events in the range 4000 to 4019 appear in the Domain Name System (DNS) event log."

Pass criteria (our reading): no 4013 after the first few minutes of boot (none at all is better), and none in the 4000 to 4019 range after that; the boot no longer hangs at "Preparing network connections"; repadmin /showrepl shows every inbound partner succeeding; dcdiag /test:dns /DnsBasic passes for the DC; every DNS server returns the SRV records; DNS Manager opens and lists the AD-integrated zones; Repl Perform Initial Synchronizations is absent or 1; and the next patch night has DCs rebooting in separate waves.

Common false starts

When a change is justified

FAQ

What does Event 4013 "The DNS server is waiting for Active Directory Domain Services (AD DS) to signal that the initial synchronization of the directory has been completed" mean? The DNS Server service on a DC with AD-integrated zones is holding off until AD DS finishes its first inbound replication after boot, because "critical DNS data might not yet be replicated onto this domain controller." A few events right after boot that then stop are normal; events every two minutes for an hour mean replication isn't finishing (Steps 2 and 6).

Why does a domain controller take 15 to 20 minutes to boot with Event 4013? AD DS needs DNS to find its replication partners, and DNS is waiting on AD DS. When the DC's DNS servers are itself or partners that are also rebooting, each lookup times out before falling back, for each partition. Microsoft's test puts it at "greater than 14 minutes" for five partitions and one source. Fix the DC's DNS server list and the reboot schedule (Steps 4 and 9).

Should I set Repl Perform Initial Synchronizations to 0 to fix Event 4013? Not as a fix. Microsoft says it "isn't recommended for use in production environments, or in any environment on an ongoing basis" and should be used only in critical, temporary situations, with the default restored afterwards. Its forest recovery guide uses it for a restored role holder and then resets it to 1 (Steps 7 and 9).

Should a domain controller point to itself for DNS? A lone DC should. A DC with partners shouldn't point only at itself: Microsoft says DCs "shouldn't point to a single domain controller and especially only to themselves as preferred DNS for name resolution." Put partners first and the DC itself last, following Microsoft's hub and branch guidance (Step 4).

What's the difference between DNS Event 4013 and 4015? 4013 is a wait: DNS hasn't started serving because AD DS initial synchronization hasn't finished. 4015 is an error: "The DNS server has encountered a critical error from the Active Directory." Its extended error code picks the cause: 00002095 (an RODC can't reach a writable DC), 0000051B (zone owner or Users group permissions), or ADMIN_LIMIT_EXCEEDED (too many orphaned records) (Step 8).

Related posts

• Active Directory Replication Error 1722 (when a partner in repadmin /showrepl fails with the RPC server unavailable)

• Netlogon Event ID 5719 (when member servers can't find a DC in the same boot window)

• DNS Scavenging Not Working (when a partner's CNAME or host record was aged out and deleted)

• Windows DNS Server Not Resolving External Names (when a DC lists a public resolver because the forwarders were wrong)

Free checklist

Setting the initial synchronization bypass, or pointing every DC at itself, before anyone read which partner and which name lookup was failing, is a change made everywhere at once. So is a Conditional Access policy enforced before anyone read the report-only results. The free Conditional Access pre-enforcement checklist keeps break-glass accounts, pilot groups, and report-only review on one page:

👉 https://cashflow4375.gumroad.com/l/ujkqxb?utm\_source=hashnode&utm\_medium=article&utm\_campaign=aps\_first\_sale&utm\_content=dns-event-4013-initial-sync

If you want the rest of the Intune runway written down for the team (enrollment hygiene, compliance phasing, break/fix cards and read-only PowerShell snapshots), the Intune & M365 Admin Starter Pack pairs with this triage: https://cashflow4375.gumroad.com/l/joonf?utm\_source=hashnode&utm\_medium=article&utm\_campaign=aps\_first\_sale&utm\_content=dns-event-4013-initial-sync

Admin Pack Studio; not affiliated with Microsoft. Steps 1-8 and 10 are read-only checks: Get-WinEvent, Get-CimInstance, Get-Service, nltest /dsgetdc, Get-DnsClientServerAddress, ipconfig /all, Test-NetConnection, Resolve-DnsName, repadmin /showrepl, repadmin /replsummary, repadmin /showattr, dcdiag /test:dns /DnsBasic and /DnsRecordRegistration, Get-ADReplicationPartnerMetadata, Get-ADReplicationFailure, Get-ADDomainController, Get-ADObject, net localgroup users /domain, Get-ItemProperty, and Get-DnsServerZone. Changing a DC's DNS server list, re-registering records, cleaning up DC metadata, removing DNS records, forcing replication, restarting or changing the startup type of services, changing the Repl Perform Initial Synchronizations value, changing group membership or zone ownership, resetting a machine account password, and rebooting DCs are deliberate changes. Change one link at a time, prove it with a planned reboot, and check current Microsoft Learn for version-specific behavior.

Not that way. "Cluster network name resource failed registration" usually isn't a broken cluster. The network name resource asks a DNS server to create or update its A (and optionally PTR) records, from the node that owns the role, with the cluster's identity; it fails when that node asks the wrong DNS server, when the zone won't take the update, when an existing record belongs to an identity the cluster can't act for, or when the computer object behind the name is disabled, missing, or out of sync. The event's reason text tells you which. Microsoft's own guidance for the secure-zone case is narrow: "Ensure that cluster name object (CNO) is granted permissions to the Secure DNS Zone." Its failover clustering blog adds the detail that explains most 1196 and 1257 events after a rebuild or a hand-made record: "By default, all Authenticated Users have permissions to create a new record inside a secure zone. But if a record already exists, the security principal (in this case the cluster name identity) should have Full Control over the existing DNS record." Opening the whole zone to nonsecure updates fixes the symptom by removing the security that keeps one machine from overwriting another's name. Work through it as a chain: which name, which event, which node → what does the event's reason or error code say? → what does the name resource itself report? → which DNS servers does the owner node ask, and can it reach them? → does the zone accept the update? → who owns the existing record? → is the computer object behind the name healthy? → is time or Kerberos in the way? → change one link → prove every DNS server answers with the right address. Get-WinEvent, Get-ClusterResource, Get-ClusterParameter, Get-ClusterResourceDependency, Get-DnsClientServerAddress, Get-DnsClient, Resolve-DnsName, Get-DnsServerZone, Get-DnsServerResourceRecord, Get-ADComputer, Get-ADObject, and Get-ClusterLog are the sources for that chain. Delete and recreate records "to be safe" and you may hand the name to the wrong owner, break every client that cached the old address, and still see the same event on the next failover.

If the record vanished because DNS scavenging deleted it, DNS Scavenging Not Working covers how aging and timestamps decide that. If a node lists a public resolver because someone wanted the internet to work, Windows DNS Server Not Resolving External Names covers the forwarder side, so the nodes can point only at your own DNS servers. This post covers "a failover cluster network name (the cluster name or a role's client access point) that won't register in DNS: Event 1196 with a reason such as 'DNS bad key' or 'DNS operation refused', Event 1195 with an error code, Event 1257 'access to update the secure DNS Zone was denied', 1256, 1258, 1259, 1260, 1578, 1579, or 1580, a name that resolves to an old address after a failover, or a role that won't come online on one node, and you need a read-first path before you delete records, loosen zone security, or run Repair."

TL;DR

• Read the reason, not just the event ID. Event 1196 carries the DNS error as text ("for the following reason: %2"), and the name resource keeps the last result in its StatusDNS parameter, which Microsoft defines as "The returned status code from the last DNS operation that the NetName resource performed." (Steps 2 and 3).
• The owner node registers, through its own adapters. The cluster registers "over some physical adapter that can reach a DNS server", so one node with a public or retired DNS server in its list can fail where the others succeed (Step 4).
• The zone is the primary DNS suffix's zone. Microsoft: "The registration for a cluster network name always happens in the zone corresponding to the Primary DNS Suffix of the host." (Step 5).
• An existing record needs the cluster's identity on its ACL. A record created by hand, by DHCP, or by a cluster you've since rebuilt can block the update; read the record's owner before you touch it (Step 6).
• Repair fixes the computer object, not DNS permissions. Microsoft's Repair Active Directory Object is for the cluster name account's password mismatch, and it "will be greyed out unless the cluster's Name resource is offline." (Steps 7 and 9).
• Re-register without an outage. Update-ClusterNetworkNameResource "registers existing Network Name resources with a DNS server in a way that doesn't interrupt cluster availability." (Step 9).

Where in the chain did it break?

Step 1: Capture the symptom (read-only)

Write these on the ticket first:

Then branch on what you see:

Step 2: Read the event and its reason (read-only)

Read the events from every node, not just the one you're on. Microsoft's own Event Viewer path is the System log filtered to the FailoverClustering source; this is the same read in PowerShell:

# Read-only: network name DNS events from the System log on every node
$ids = 1069, 1195, 1196, 1205, 1256, 1257, 1258, 1259, 1260, 1261, 1262, 1578, 1579, 1580
foreach ($n in (Get-ClusterNode).Name) {
    Get-WinEvent -ComputerName $n -MaxEvents 50 -FilterHashtable @{
        LogName      = 'System'
        ProviderName = 'Microsoft-Windows-FailoverClustering'
        Id           = $ids
    } -ErrorAction SilentlyContinue |
        Select-Object @{n='Node';e={$n}}, TimeCreated, Id, Message
}

Then match the text against Microsoft's failover clustering event list:

Two text details matter when you search: Microsoft's 1256 and 1257 text says "associated DNS names(s)", and 1258 says "because the a DNS server could not be reached." Copy the event text as it is rather than retyping it.

The reasons and codes come from the Win32 DNS error list. The ones that show up most in these events:

Microsoft's older 1196 troubleshooting page also says to "Check the system event log for Netlogon or DNS events that occurred near the time of the failover cluster event." Those are usually the same root cause seen from the node's own registration.

Step 3: Read what the name resource reports (read-only)

Find every network name, its state, and the node that owns it. The core cluster name lives in the group Microsoft's tools call Cluster Core Resources; roles carry their own:

# Read-only: every Network Name resource, its role, state, and owner node
Get-ClusterResource |
    Where-Object { $_.ResourceType.Name -eq 'Network Name' } |
    Select-Object Name, OwnerGroup, State, OwnerNode

# The IP address resources each name depends on (an OR list in multi-subnet clusters)
Get-ClusterResourceDependency -Resource 'Cluster Name'
Get-ClusterResourceDependency -Resource 'FS01'

Then read the private parameters that drive DNS registration. Get-ClusterParameter is Microsoft's tool for this: used with a Network Name resource, "you can obtain DNS-related information about the resource."

# Read-only: the DNS-related parameters of one network name
Get-ClusterResource -Name 'Cluster Name' |
    Get-ClusterParameter -Name DnsName, HostRecordTTL, RegisterAllProvidersIP, PublishPTRRecords, StatusDNS, LastDNSUpdateTime, CreatingDC

# The last DNS update time is a FILETIME; this turns it into a local date and time
$p = Get-ClusterResource -Name 'Cluster Name' | Get-ClusterParameter -Name LastDNSUpdateTime
[DateTime]::FromFileTime([int64]$p.Value)

Read the values against Microsoft's definitions:

Also check the IP address resources the name depends on. Microsoft's dependency example shows the name coming online when "EITHER" provider address does, so a name can be online while one address is offline and never registered:

# Read-only: state and address of the IP resources behind the name
Get-ClusterResource |
    Where-Object { $_.ResourceType.Name -eq 'IP Address' } |
    ForEach-Object {
        $a = $_ | Get-ClusterParameter -Name Address
        [pscustomobject]@{ Resource = $_.Name; State = $_.State; Group = $_.OwnerGroup; Address = $a.Value }
    }

Step 4: Which DNS servers does the owner node ask? (read-only)

Microsoft's 1196 text is a DNS client instruction: "Ensure that the network adapters associated with dependent IP address resources are configured with at least one accessible DNS server." Read the adapters on every node, because the owner node does the work:

# Read-only: DNS servers, registration settings, and suffix per adapter, on every node
foreach ($n in (Get-ClusterNode).Name) {
    Invoke-Command -ComputerName $n -ScriptBlock {
        $c = Get-DnsClient
        Get-DnsClientServerAddress -AddressFamily IPv4 |
            Where-Object { $_.ServerAddresses } |
            ForEach-Object {
                $i = $_.InterfaceIndex
                $d = $c | Where-Object InterfaceIndex -eq $i
                [pscustomobject]@{
                    Node      = $env:COMPUTERNAME
                    Adapter   = $_.InterfaceAlias
                    Servers   = $_.ServerAddresses -join ', '
                    Register  = $d.RegisterThisConnectionsAddress
                    UseSuffix = $d.UseSuffixWhenRegistering
                    Suffix    = $d.ConnectionSpecificSuffix
                }
            }
    }
}

Then prove the node can reach and query each server it lists, by name and by the zone's SOA:

# Read-only: from the owner node, ask each listed DNS server about the zone and the name
$zone = 'contoso.com'
$name = 'CLUSTER1.contoso.com'
foreach ($s in '10.0.0.10', '10.0.0.11') {
    Test-NetConnection -ComputerName $s -Port 53 |
        Select-Object ComputerName, TcpTestSucceeded
    Resolve-DnsName -Name $zone -Type SOA -Server $s -DnsOnly -ErrorAction SilentlyContinue |
        Select-Object Name, PrimaryServer
    Resolve-DnsName -Name $name -Type A -Server $s -DnsOnly -ErrorAction SilentlyContinue |
        Select-Object Name, IPAddress, TTL
}

A server that can't return the zone's SOA isn't a server this node should be updating through (our reading). Microsoft's dynamic update article explains why the SOA matters: for a directory-integrated zone, "any DNS server that is loading the zone can respond and dynamically insert its own name as the primary server of the zone in the SOA query response."

Step 5: Does the zone accept the update? (read-only)

The name registers in a predictable zone. From Microsoft's failover clustering blog: "The registration for a cluster network name always happens in the zone corresponding to the Primary DNS Suffix of the host." Check the node's primary suffix, then the zone on each DNS server:

# Read-only: the primary DNS suffix of each node (the zone the name registers in)
foreach ($n in (Get-ClusterNode).Name) {
    Invoke-Command -ComputerName $n -ScriptBlock {
        [pscustomobject]@{
            Node   = $env:COMPUTERNAME
            Domain = (Get-CimInstance -ClassName Win32_ComputerSystem).Domain
            Suffix = (Get-ItemProperty 'HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters').Domain
        }
    }
}

# Read-only: zone type, AD integration, and dynamic update setting on each DNS server
foreach ($s in 'DC01', 'DC02') {
    Get-DnsServerZone -Name 'contoso.com' -ComputerName $s |
        Select-Object @{n='Server';e={$s}}, ZoneName, ZoneType, IsDsIntegrated, DynamicUpdate, ReplicationScope
}

If scavenging is on, read the record's timestamp too (Step 6). Microsoft's scavenging article: "Windows clients dynamically update DNS every 24 hours. All DDNS records are set to scavenge." A cluster name that can't refresh because of a 1257 can then age out and disappear, which turns a warning into an outage (DNS Scavenging Not Working covers the aging rules).

Step 6: Who owns the existing record? (read-only)

This is the step most 1257, 1259, 1579, and 1580 events end at. Microsoft's dynamic update article states the default policy: clients "try to overwrite a previously registered resource record, unless they are specifically blocked by update security." In a secure zone, the record's ACL is that security. Read the record first:

# Read-only: the record as each DNS server holds it (Timestamp empty or 0 = static)
foreach ($s in 'DC01', 'DC02') {
    Get-DnsServerResourceRecord -ZoneName 'contoso.com' -Name 'CLUSTER1' -ComputerName $s -ErrorAction SilentlyContinue |
        Select-Object @{n='Server';e={$s}}, HostName, RecordType, Timestamp, TimeToLive,
            @{n='Data';e={$_.RecordData.IPv4Address}}
}

Then read its owner and ACL. In an AD-integrated zone the record is a dnsNode object; this reads it without changing anything:

# Read-only: owner and ACL entries of the record object (domain-wide replication scope shown)
$dom = (Get-ADDomain).DistinguishedName
$rec = Get-ADObject -Server DC01 -SearchBase "DC=DomainDnsZones,$dom" `
    -LDAPFilter '(&(objectClass=dnsNode)(name=CLUSTER1))' -Properties nTSecurityDescriptor, whenCreated
$rec | Select-Object DistinguishedName, whenCreated
$rec.nTSecurityDescriptor.Owner
$rec.nTSecurityDescriptor.Access |
    Where-Object { $_.IdentityReference -match 'CLUSTER1|Authenticated Users|Everyone|DnsUpdateProxy' } |
    Select-Object IdentityReference, ActiveDirectoryRights, AccessControlType

In DNS Manager, the same read is the record's Properties > Security tab (on older consoles, View > Advanced first). Read the results like this:

For a zone stored in the forest-wide partition, use DC=ForestDnsZones and the forest root in -SearchBase instead. For a role's name (FS01, a SQL Server instance), repeat with that name; Microsoft's 1580 text refers to "the cluster identity" and its record, so compare the record's owner with both the CNO and the role's own computer object rather than guessing which one should be there.

Step 7: Is the computer object behind the name healthy? (read-only)

The network name acts as a computer account in AD: the CNO for the cluster name, and a VCO for each role's client access point. Microsoft's description: "Through the CNO, virtual computer objects (VCOs) are automatically created when you set up clustered roles that use client access points." Read both:

# Read-only: the CNO and a VCO, as one DC sees them
Get-ADComputer -Identity 'CLUSTER1' -Server DC01 -Properties Enabled, DistinguishedName, whenCreated, PasswordLastSet, ObjectGUID |
    Select-Object Name, Enabled, DistinguishedName, whenCreated, PasswordLastSet, ObjectGUID
Get-ADComputer -Identity 'FS01' -Server DC01 -Properties Enabled, DistinguishedName, PasswordLastSet |
    Select-Object Name, Enabled, DistinguishedName, PasswordLastSet

# The GUID the name resource expects, to compare with the AD object above
Get-ClusterResource -Name 'Cluster Name' | Get-ClusterParameter -Name ObjectGUID

The rights Microsoft documents for these objects:

Step 8: Is time or Kerberos in the way? (read-only)

A secure update is signed with the identity's Kerberos credentials, so "DNS bad key", "DNS signature failed to verify.", and "DNS signature validity expired." can be identity or clock problems rather than DNS ones (our reading). Read time and tickets from the owner node:

# Read-only: time offset of each node against a DC, and the node's own secure channel
foreach ($n in (Get-ClusterNode).Name) {
    Invoke-Command -ComputerName $n -ScriptBlock {
        w32tm /stripchart /computer:DC01 /samples:3 /dataonly
        w32tm /query /status
    }
}

REM On the owner node, elevated: tickets held by the computer (SYSTEM) logon session
klist -li 0x3e7

Step 9: Change one link, then re-register (a change)

Fix only what Steps 2 to 8 showed, then re-register the name and read the result before changing anything else.

The node's DNS server list (Step 4). Point every cluster-facing adapter at your AD DNS servers only, the same list on every node:

# A change, on the node that was wrong: replace its DNS server list on one adapter
Set-DnsClientServerAddress -InterfaceAlias 'Mgmt' -ServerAddresses '10.0.0.10', '10.0.0.11'

# Re-read it
Get-DnsClientServerAddress -InterfaceAlias 'Mgmt' -AddressFamily IPv4

The adapter's registration setting (Step 4). If the cluster-facing adapter had registration turned off and the zone is dynamic, turn it back on for that adapter (Set-DnsClient -InterfaceAlias 'Mgmt' -RegisterThisConnectionsAddress $true). If the zone is meant to be static, leave it off and maintain the records yourself, on purpose.

The zone's dynamic update setting (Step 5). If an AD-integrated zone was set to None by mistake, set it to secure only, not to nonsecure. Microsoft's -DynamicUpdate values are None, Secure, and NonsecureAndSecure:

# A change: AD-integrated zone back to secure dynamic updates only
Set-DnsServerPrimaryZone -Name 'contoso.com' -DynamicUpdate Secure -ComputerName DC01

The record's ACL (Step 6). For an existing record the cluster can't update, the narrow fix is the record, not the zone: in DNS Manager, open the record's Properties > Security, add the cluster's computer object (CLUSTER1$; enable Computers in Object Types), and give it Full control, as Microsoft's blog describes. For the rebuilt-cluster case, Microsoft's documented resolution is the zone-level grant: on the zone's Properties > Security, "Verify that the CNO has the Create all child objects and Write all properties permissions. If the permissions are missing, configure them."

A stale record you've decided to remove (Step 6). Only when its owner is gone or wrong and the cluster can't take it over. Microsoft's Remove-DnsServerResourceRecord deletes "all records that match RRtype and Name" unless you give -RecordData, so name the address:

# A change: remove one stale A record by its address, previewing first
Remove-DnsServerResourceRecord -ZoneName 'contoso.com' -RRType 'A' -Name 'CLUSTER1' -RecordData '10.0.0.50' -ComputerName DC01 -WhatIf
Remove-DnsServerResourceRecord -ZoneName 'contoso.com' -RRType 'A' -Name 'CLUSTER1' -RecordData '10.0.0.50' -ComputerName DC01

Clients that cached the old record keep it for its TTL, and the name doesn't resolve until the cluster re-registers, so do this in a window and re-register right away.

The computer object (Step 7). If the object is disabled and it's the running cluster's CNO, enable it (Enable-ADAccount -Identity 'CLUSTER1$'). For the password mismatch, use Microsoft's procedure in Failover Cluster Manager: expand Cluster Core Resources, under Cluster Name right-click the Name item and select Take Offline, then More Actions > Repair Active Directory Object. Microsoft notes that "Taking the cluster's Name resource offline shouldn't negatively impact other cluster groups, such as the SQL Server Failover Cluster Instance." Bring the name back online afterwards. Microsoft's account troubleshooting article also covers the CNO's rights on its container and on each VCO, and the computer account quota (ms-DS-MachineAccountQuota) when VCO creation fails.

Re-register the name. Use the online path first, then read the status:

# A change, without taking the name offline: re-register network names in DNS
Get-ClusterResource -Name 'Cluster Name' | Update-ClusterNetworkNameResource
Get-ClusterResource -Name 'FS01' | Update-ClusterNetworkNameResource

# Read the result
Get-ClusterResource -Name 'Cluster Name' | Get-ClusterParameter -Name StatusDNS, LastDNSUpdateTime

If it still fails, take that one name offline and back online in a window (Stop-ClusterResource -Name 'FS01' then Start-ClusterResource -Name 'FS01'). Microsoft's Stop-ClusterResource example notes: "Before taking the resource offline, it takes offline any dependent resources." For a role's name, that means the role's clients lose it, so read the dependency report first. Taking a name offline doesn't clean DNS either; the blog: "Offlining a network name resources does not delete the DNS records on the server. They are only cleaned up when a network name resource is deleted from the cluster."

When the event is gone, collect the logs anyway if it took more than one change; Microsoft's data collection step for this issue is Get-ClusterLog -UseLocalTime -Destination with a folder:

# Read-only: cluster log from every node for the last 30 minutes, local time, copied here
Get-ClusterLog -UseLocalTime -TimeSpan 30 -Destination C:\Temp\ClusterLogs
Select-String -Path C:\Temp\ClusterLogs\*.log -Pattern 'DNS' | Select-Object -Last 40

Step 10: Prove every DNS server answers with the right address (read-only)

Ask every DNS server that clients use, not just the one the node used, and then make the cluster register from another node:

# Read-only: what each DNS server returns for each network name
$names   = 'CLUSTER1.contoso.com', 'FS01.contoso.com'
$servers = '10.0.0.10', '10.0.0.11', '10.1.0.10'
foreach ($s in $servers) {
    foreach ($nm in $names) {
        Resolve-DnsName -Name $nm -Type A -Server $s -DnsOnly -ErrorAction SilentlyContinue |
            Select-Object @{n='Server';e={$s}}, Name, IPAddress, TTL
    }
}

# Read-only: the cluster's own view after the change
Get-ClusterResource | Where-Object { $_.ResourceType.Name -eq 'Network Name' } | ForEach-Object {
    $st = $_ | Get-ClusterParameter -Name StatusDNS
    [pscustomobject]@{ Name = $_.Name; State = $_.State; Owner = $_.OwnerNode; StatusDNS = $st.Value }
}

Then, in a window, move the role to each other node and read the events and StatusDNS again. Microsoft's own resolution for the rebuild case ends the same way: "To test the cluster, perform a manual failover." and "Review the event logs to verify that DNS registration succeeds and doesn't generate permission errors."

# A deliberate move (the role restarts on the other node), then the same reads
Move-ClusterGroup -Name 'FS01' -Node HVN01
Get-ClusterResource -Name 'FS01' | Get-ClusterParameter -Name StatusDNS, LastDNSUpdateTime

Pass criteria (our reading): every DNS server returns the expected addresses (all subnets if RegisterAllProvidersIP is 1); StatusDNS is 0 for every network name; LastDNSUpdateTime is after your change; no new 1195, 1196, 1257, 1258, 1259, 1578, 1579, or 1580 on any node after a move; the record's owner and ACL include the cluster's identity; and Test-Cluster -Include 'Network' (Microsoft lists Network among the validation test types) runs clean if you changed adapters.

Common false starts

When a change is justified

FAQ

What does "Cluster network name resource failed registration of one or more associated DNS name(s)" mean? The network name couldn't create or update its DNS records from the node that owns it. Event 1196 adds the DNS error as text after "for the following reason:", and Event 1195 gives it as a code. Read that reason, the name's StatusDNS, and the owner node's DNS server list before you change anything (Steps 2 to 4).

How do I fix Event 1196 "DNS bad key" on a failover cluster? Check, in order: that every node's cluster-facing adapter lists only your AD DNS servers; that the zone is AD-integrated and set to secure updates; that the existing record grants the cluster's computer object rights to update it; and that the node's clock and Kerberos are healthy. Fix the one that's wrong, then run Update-ClusterNetworkNameResource and read StatusDNS again (Steps 4 to 9).

What permissions does the cluster name object need on DNS for Event 1257? Microsoft's guidance in the event is "Ensure that cluster name object (CNO) is granted permissions to the Secure DNS Zone." For a rebuilt cluster, its resolution is the zone-level "Create all child objects and Write all properties" for the CNO; for one existing record, the narrower fix is Full control for the cluster's identity on that record (Steps 6 and 9).

Does Repair Active Directory Object fix DNS registration? Not by itself. Microsoft documents Repair for the case where "the password for the cluster name account no longer matches the corresponding password stored by the clustering software." It's greyed out until the cluster Name resource is offline. A 1257 caused by a record ACL needs the ACL fixed (Steps 7 and 9).

How do I re-register a cluster name in DNS without downtime? Run Get-ClusterResource -Name 'Cluster Name' | Update-ClusterNetworkNameResource (or the role's name), which Microsoft says works "in a way that doesn't interrupt cluster availability." Then read StatusDNS and LastDNSUpdateTime, and ask every DNS server for the name (Steps 9 and 10).

Related posts

• DNS Scavenging Not Working (when the cluster name's record was aged out and deleted)
• Windows DNS Server Not Resolving External Names (when nodes list a public resolver because the forwarders were wrong)
• The Target Principal Name Is Incorrect (when the secure update fails on Kerberos between the node and the DNS server)
• Windows Time Not Syncing With Domain Controller (when "DNS bad key" or "signature validity expired" comes from clock skew)

Free checklist

Giving the whole zone to Authenticated Users, or switching it to nonsecure updates, before anyone read which record and which identity failed, is a change made everywhere at once. So is a Conditional Access policy enforced before anyone read the report-only results. The free Conditional Access pre-enforcement checklist keeps break-glass accounts, pilot groups, and report-only review on one page:

👉 https://cashflow4375.gumroad.com/l/ujkqxb?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=cluster-network-name-failed-registration

If you want the rest of the Intune runway written down for the team (enrollment hygiene, compliance phasing, break/fix cards and read-only PowerShell snapshots), the Intune & M365 Admin Starter Pack pairs with this triage: https://cashflow4375.gumroad.com/l/joonf?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=cluster-network-name-failed-registration

Admin Pack Studio; not affiliated with Microsoft. Steps 1-8 and 10 are read-only checks: Get-WinEvent, Get-ClusterNode, Get-ClusterResource, Get-ClusterParameter, Get-ClusterResourceDependency, Get-DnsClient, Get-DnsClientServerAddress, Test-NetConnection, Resolve-DnsName, Get-DnsServerZone, Get-DnsServerResourceRecord, Get-ADObject, Get-ADComputer, w32tm /stripchart and /query /status, klist, and Get-ClusterLog. Changing DNS server lists or adapter registration, zone dynamic update settings, record or zone permissions, removing records, enabling or repairing computer objects, re-registering or restarting network names, and moving roles are deliberate changes. Change one link at a time, prove it from every DNS server and every node, and check current Microsoft Learn for version-specific behavior.

Not that way. "FSMO role transfer failed" usually isn't a broken role. A transfer is a conversation between the current holder and the new one, so it fails whenever the current holder is offline, unreachable, not yet replicated since it last started, or recorded as a server that no longer exists; a seizure skips that conversation, which is why it's a one-way decision about the old DC, not a stronger retry. Microsoft's guidance is blunt about both halves: "Role seizure should be performed only as a last resort", and "Do not allow a former operations master role holder whose role has been seized to return to an online domain controller." Seizing all five roles and then powering DC01 back on is the one combination that page warns about. Microsoft also notes that "FSMO roles assigned to domain controllers that are offline or in an error state only have to be transferred or seized if role-dependent operations are being done", so the first question is which operations are actually failing. Work through it as a chain: which roles, and what's failing because of them → who owns each role, according to more than one DC? → is the current holder online and replicating? → is the target DC the right one? → does your account have the rights for this role? → transfer, wait, or seize? → transfer it properly → seize it properly, only if Step 6 says so → clean up the old DC and the owners that still point at it → prove every DC agrees. netdom query fsmo, Get-ADDomain, Get-ADForest, Get-ADObject on the role objects, dcdiag /test:KnowsOfRoleHolders, repadmin /replsummary, Get-ADReplicationPartnerMetadata, Get-ADDomainController, and whoami /groups are the sources for that chain. Seize every role "to be safe" and you may burn RID space, leave a stale owner in a DNS application partition, and create a duplicate role holder the day the old hardware comes back.

If the role holder can't be reached because replication itself fails with RPC errors, Active Directory Replication Error 1722 covers that side first. If the new PDC emulator now has to become the forest's time source, Windows Time Not Syncing With Domain Controller covers the time chain. This post covers "an operations master (FSMO) role won't transfer, or you're deciding whether to seize it: 'The requested FSMO operation failed. The current FSMO holder could not be contacted.', 0x20AF or 0x20AE from ntdsutil or PowerShell, 'Access is denied' when you try, a role owner that shows a deleted DC, a demotion that can't determine role ownership, or a role holder that died and isn't coming back, and you need a read-first path before you run seize or -Force."

TL;DR

• Transfer needs the old holder; seize doesn't, and that's the risk. Microsoft: "We recommend that you only seize all roles when the previous role holder isn't returning to the domain." (Step 6).
• Read the owners from more than one DC. dcdiag's KnowsOfRoleHolders test "doesn't check all DCs knowledge for consistency", so compare netdom query fsmo and the fSMORoleOwner values from each DC (Step 2).
• A holder that just restarted isn't a holder yet. "The DC waits until it receives inbound replication for its naming context" before acting on a role, and so does a new holder after a transfer (Steps 3 and 10).
• Connect ntdsutil to the new holder, not the dead one. In Microsoft's procedure, the server you connect to "is the name of the DC that you want to assign the Operation Master role to." (Steps 7 and 8).
• Seizing the RID master costs RIDs. "The Move-ADDirectoryServerOperationMasterRole cmdlet increases the next Rid pool by 30,000 from what it finds in Active Directory." (Step 8).
• After a seizure, the old DC never comes back as it was. Clean up its metadata, and rebuild it from scratch if you want it as a DC again (Step 9).

Where in the chain did it break?

Step 1: Capture the symptom (read-only)

Write these on the ticket first:

Then branch on what you see:

Step 2: Read who owns each role, from more than one DC (read-only)

Start with the two quick reads (Microsoft says netdom query must run from an elevated command prompt), then repeat them against the target DC and at least one other DC. Each DC answers from its own copy of the directory:

# The role owners as this computer's DC sees them (Microsoft's netdom query fsmo)
netdom query fsmo

# The same question answered by specific DCs (/server is the DC that performs the query)
netdom query /server:DC02 fsmo
netdom query /server:DC03 fsmo

# Domain roles and forest roles, asked of one DC at a time
Get-ADDomain -Server DC02 | Select-Object PDCEmulator, RIDMaster, InfrastructureMaster
Get-ADForest -Server DC02 | Select-Object SchemaMaster, DomainNamingMaster

dcdiag has a test for this, but read its scope: KnowsOfRoleHolders "returns the DCs knowledge of the five Flexible Single Master Operation (FSMO) roles but doesn't check all DCs knowledge for consistency. Using the /e parameter provides data for comparison." The similarly named FsmoCheck test isn't the same thing; Microsoft's description ends with "This test does not test any of the servers for operations master roles."

REM Every DC in the enterprise reports what it knows about the five roles; compare the answers
dcdiag /test:KnowsOfRoleHolders /e /v

When the friendly names look wrong, read the raw owner. Each role is an fSMORoleOwner value on one object, and Microsoft's own instruction for a partition's infrastructure master is to "query the fSMORoleOwner attribute on the infrastructure object under the naming context root in question." The same read works for every role object:

# Read-only: the raw owner (an NTDS Settings DN) of every role object, as DC02 sees it
$dc   = 'DC02'
$root = Get-ADRootDSE -Server $dc
$dom  = $root.defaultNamingContext
$fr   = $root.rootDomainNamingContext
$objs = @(
    $dom                                          # PDC emulator (domain head)
    "CN=RID Manager`$,CN=System,$dom"             # RID master
    "CN=Infrastructure,$dom"                      # infrastructure master
    "CN=Partitions,$($root.configurationNamingContext)"  # domain naming master
    $root.schemaNamingContext                     # schema master
    "CN=Infrastructure,DC=DomainDnsZones,$dom"    # DNS application partition (domain)
    "CN=Infrastructure,DC=ForestDnsZones,$fr"     # DNS application partition (forest)
)
foreach ($o in $objs) {
    try {
        Get-ADObject -Identity $o -Server $dc -Properties fSMORoleOwner -ErrorAction Stop |
            Select-Object DistinguishedName, fSMORoleOwner
    } catch { "Not found on ${dc}: $o" }
}

Run it with $dc set to each DC you care about. Read the results like this:

Step 3: Is the current holder online and replicating? (read-only)

A transfer only works when the current holder takes part, and a holder only acts as one after it has replicated. Microsoft's transfer-or-seize article: "When a DC that has been acting as a role holder starts to run (for example, after a failure or a shutdown), it doesn't immediately resume behaving as the role holder." The per-role details are in Microsoft's FSMO roles article: the RID master "is active only when the role owner has inbound replicated the domain NC successfully since the Directory Service started", and the schema and domain naming masters have the same rule for their partitions. The PDC emulator is the exception: "There is no inbound replication requirement."

So read replication for the holder and the target before anything else:

REM Forest-wide summary: which DCs are failing inbound or outbound replication, errors first
repadmin /replsummary /errorsonly

REM Inbound replication status, per partition and partner, for the target and for the holder
repadmin /showrepl DC02
repadmin /showrepl DC01

# Read-only: the target's inbound partners for every partition it holds, with the last result
Get-ADReplicationPartnerMetadata -Target DC02 -Partition * |
    Select-Object Server, Partner, Partition, LastReplicationAttempt, LastReplicationSuccess,
        LastReplicationResult, ConsecutiveReplicationFailures |
    Format-Table -AutoSize

# Read-only: replication failures recorded for the target and the holder
Get-ADReplicationFailure -Target DC02, DC01

# Is the holder's name resolving and answering at all? (ICMP can be blocked; this is a hint, not proof)
Test-NetConnection -ComputerName DC01

Microsoft's note on the /showrepl output: "If @ [Never] appears in the output for a directory partition, replication of that directory partition has never succeeded from the identified source replication partner over the listed connection." Then look at the events around the failed attempt on the DC that did the work:

# Directory Service events Microsoft's role articles mention (1586 after a PDC seizure, 2091 in a demotion log)
Get-WinEvent -FilterHashtable @{
    LogName   = 'Directory Service'
    Id        = 1586, 2091
    StartTime = (Get-Date).AddDays(-2)
} -ErrorAction SilentlyContinue | Format-List TimeCreated, Id, Message

# RID issuance events (Microsoft: logged in the System log under Directory-Services-SAM)
Get-WinEvent -FilterHashtable @{
    LogName   = 'System'
    Id        = 16642, 16643, 16644, 16645, 16655, 16656, 16657
    StartTime = (Get-Date).AddDays(-7)
} -ErrorAction SilentlyContinue | Format-List TimeCreated, Id, Message

Step 4: Is the target DC the right one? (read-only)

Microsoft's criteria for the new role holder are short: "It resides in the same domain as the previous role holder." and "It has the most recent replicated writable copy of the role partition." Its placement guidance adds: "Place the RID master on the domain PDC in the same domain." and "Place the schema master on the PDC of the forest root domain." The usual answer is a healthy DC in the same site that's a direct replication partner of the old holder. Microsoft's standby-master guidance says to "Ensure that the standby operations masters are direct replication partners of the actual operations masters."

# Read-only: every DC in the domain, its site, and whether it's a GC or read-only
Get-ADDomainController -Filter * |
    Select-Object Name, Site, IsGlobalCatalog, IsReadOnly |
    Sort-Object Site, Name | Format-Table -AutoSize

# Microsoft's single-DC global catalog check (True or False)
(Get-ADDomainController -Filter { Name -Eq 'DC02' }).IsGlobalCatalog

# Is the AD Recycle Bin enabled? (changes whether the infrastructure master matters)
Get-ADOptionalFeature -Identity 'Recycle Bin Feature'

REM Health of the proposed target, verbose
dcdiag /s:DC02 /v

If you need a property that isn't in that list, Microsoft's note on Get-ADDomainController is to pipe one DC to Get-Member and read all of them.

Step 5: Does your account have the rights for this role? (read-only)

Microsoft's current article: "The signed-in user should be a member of the Enterprise Administrators group to transfer Schema master or Domain naming master roles, or a member of the Domain Administrators group of the domain where the PDC emulator, RID master and the infrastructure master roles are being transferred." Its older per-role procedures name Schema Admins for the schema master ("Membership in Schema Admins, or equivalent, is the minimum required to complete this procedure."), so for a schema master move, be in both (our reading).

# Read-only: what the current session's token actually contains (look for the groups above)
whoami /groups

# Read-only: who's in the forest-level groups (ask a forest root DC)
Get-ADGroupMember -Identity 'Schema Admins' -Server DC02 | Select-Object Name, SamAccountName
Get-ADGroupMember -Identity 'Enterprise Admins' -Server DC02 | Select-Object Name, SamAccountName

Step 6: Transfer, wait, or seize? (decision, no changes yet)

This is the step the forum post skips. Microsoft's rules:

• Transfer when "The current role holder is operational and can be accessed on the network by the new Operation Master owner."
• Seize when "The current role holder is experiencing an operational error that prevents an Operation Master-dependent operation from completing successfully, and you can't transfer the role.", after a forced demotion, or when "The operating system on the computer that originally owned a specific role no longer exists or has been reinstalled."
• Wait when you can: "If the role holder can be made operational before the role is needed, you may delay seizing the role." Except for one role: "The PDC role in each domain should always be online."

How much each role can wait, and how bad a duplicate would be (Microsoft's function list and its replication-island table):

Then decide:

Before a seizure, Microsoft also asks for patience: "do not perform a role seizure until enough time has passed to complete at least one end-to-end replication cycle across your network." The more of the old holder's last changes reach the target, the less you lose.

Step 7: Transfer it properly (a change)

Only when Step 6 says transfer. In PowerShell, -Identity is the DC that gets the roles; Microsoft: "The Identity parameter specifies the directory server that receives the roles." The cmdlet doesn't have to run on either DC: it "can be remotely executed from any domain joined computer where the Active Directory module for Windows PowerShell administration module is installed and available for use."

# Preview, then move the three domain roles to DC02 (Domain Admins of that domain)
Move-ADDirectoryServerOperationMasterRole -Identity DC02 -OperationMasterRole PDCEmulator, RIDMaster, InfrastructureMaster -WhatIf
Move-ADDirectoryServerOperationMasterRole -Identity DC02 -OperationMasterRole PDCEmulator, RIDMaster, InfrastructureMaster

# The two forest roles (Enterprise Admins; Schema Admins as well for the schema master)
Move-ADDirectoryServerOperationMasterRole -Identity DC02 -OperationMasterRole SchemaMaster, DomainNamingMaster

# Only have the FQDN? Microsoft's Example 5: resolve it with Get-ADDomainController first
$Server = Get-ADDomainController -Identity "dc02.contoso.com"
Move-ADDirectoryServerOperationMasterRole -Identity $Server -OperationMasterRole PDCEmulator

That last pattern matters: "There is a known issue where the Move-ADDirectoryServerOperationMasterRole cmdlet fails when an FQDN is specified directly as the value of the Identity parameter."

With ntdsutil, run it on the DC that will receive the roles (Microsoft's recommendation), connect to that DC, then transfer. Microsoft's syntax is transfer <role>; type ? at the fsmo maintenance: prompt to see the exact role names your version accepts:

C:\> ntdsutil
ntdsutil: roles
fsmo maintenance: connections
server connections: connect to server DC02
server connections: q
fsmo maintenance: ?
fsmo maintenance: transfer pdc
fsmo maintenance: transfer rid master
fsmo maintenance: transfer infrastructure master
fsmo maintenance: q
ntdsutil: q

The consoles do the same thing, one role at a time, as long as you point them at the target DC first:

Step 8: Seize it properly, only if Step 6 says so (a change)

A seizure is a decision about the old DC, so make it once and write it on the ticket. Microsoft's older procedure explains the risk: "If two domain controllers are performing the same operations master role simultaneously, the severity of the effect from duplicate operations master roles varies, depending on the role that was seized. The effect can range from no visible effect to potential corruption of the Active Directory database."

On the target DC, in an elevated prompt, connect ntdsutil to that DC (Microsoft: "If ServerFQDN does not succeed, use the NetBIOS name of the DC."), then seize. The seize commands are Microsoft's; note the two exceptions: "Exceptions are for the PDC emulator role, whose syntax is seize pdc, and the Domain naming master, whose syntax is seize naming master."

C:\> ntdsutil
ntdsutil: roles
fsmo maintenance: connections
server connections: connect to server DC02.contoso.com
server connections: quit
fsmo maintenance: seize pdc
fsmo maintenance: seize rid master
fsmo maintenance: seize infrastructure master
fsmo maintenance: seize naming master
fsmo maintenance: seize schema master
fsmo maintenance: quit
ntdsutil: quit

Seize only the roles the dead DC held (from Step 2), not every role in the forest. Expect ntdsutil to try a transfer first: "When you use Ntdsutil.exe to seize an operations master role, the tool first attempts a transfer from the current role owner." You'll see a transfer error and then the seizure. For the RID master, Microsoft adds that the target "tries to synchronize with a replication partner before accepting this role", and if it can't, "a dialog box appears asking you whether you want to continue with the operation despite this computer not being able to synchronize with a partner. Click Yes."

The PowerShell equivalent is the Step 7 command with -Force. Microsoft: "This operation still attempts graceful transfer first, then seizes if transfer is not possible."

# Seize only the roles the dead DC held (Microsoft's Example 4 form, with this post's role list)
Move-ADDirectoryServerOperationMasterRole -Identity DC02 -OperationMasterRole PDCEmulator, RIDMaster, InfrastructureMaster -Force

Know what a RID master seizure costs before you run it. Microsoft: "Rid Master seizures increment the next available RID in the pool when you seize the RID master role." The amount depends on the tool: "The Move-ADDirectoryServerOperationMasterRole cmdlet increases the next Rid pool by 30,000 from what it finds in Active Directory." and with ntdsutil "it increases the next Rid pool by 10,000." Microsoft's conclusion: "So seize the Rid Master only when you're sure that the current Rid Master can't be brought back into service." A domain has "capacity for roughly one billion security principals", so one seizure is affordable; a habit of seizing isn't (our reading). Microsoft's read of the remaining pool:

REM Microsoft's form: the available RID pool for the domain, from the RidManager test
Dcdiag.exe /TEST:RidManager /v | find /i "Available RID Pool for the Domain"

Step 9: Clean up the old DC and every owner that still points at it (a change)

After a seizure, the old DC doesn't get its old life back. Microsoft's options for it: format and reinstall, or forcibly demote it, then remove its metadata from another DC; "After you clean up the metadata, you can repromote the computer to a DC, and transfer a role back to it." Restoring it from a backup is the wrong rebuild: "If you plan to use the repaired computer as a DC, we recommend that you rebuild the computer into a DC from scratch instead of restoring the DC from a backup." When the board in the opener arrives, keep DC01 off the network until it's reinstalled or demoted (our reading).

Metadata cleanup. Microsoft: "Metadata cleanup is a required procedure after a forced removal of Active Directory Domain Services (AD DS)." The GUI way is to delete the DC's computer object from the Domain Controllers OU in Active Directory Users and Computers and select "This Domain Controller is permanently offline and can no longer be demoted using the Active Directory Domain Services Installation Wizard (DCPROMO)". If it still holds roles, that dialog moves them to a DC it picks: "You cannot change this domain controller. If you want to move the role to a different domain controller, you must move the role after you complete the server metadata cleanup procedure." The command-line way, connected to a replication partner of the removed DC (Microsoft: "This domain controller must be a replication partner of the removed domain controller."):

C:\> ntdsutil
ntdsutil: metadata cleanup
metadata cleanup: connections
server connections: connect to server DC02
server connections: quit
metadata cleanup: remove selected server DC01
metadata cleanup: quit
ntdsutil: quit

If cleanup says "Access is denied", Microsoft's note is to "make sure that the computer object and the NTDS Settings object for the domain controller are not protected against accidental deletion." Afterward, Microsoft's check is that the DC is gone from the Domain Controllers OU and that its server object in Sites and Services no longer has an NTDS Settings object.

DNS application partition owners. Rerun the Step 2 read for CN=Infrastructure,DC=DomainDnsZones,... and CN=Infrastructure,DC=ForestDnsZones,.... If either still names a deleted owner, demotions and adprep /rodcprep fail later. Microsoft's resolution: "Use ADSIEDIT.MSC to assign the DN path for the fsMORoleOwner attribute to a live DC that was a direct replication partner of the original FSMO role owner." or run the script in its adprep article, cscript fixfsmo.vbs DC=DomainDnsZones,DC=contoso,DC=com (and again for ForestDnsZones), copied from that article, not retyped.

Time. If the PDC emulator of the forest root moved, the time source moved with it. Microsoft: "The PDC emulator at the root of the forest becomes authoritative for the enterprise, and should be configured to gather the time from an external source." Microsoft's w32tm /config example for a manual peer list, plus its /reliable:YES option ("This setting is only meaningful on domain controllers."), with your own time servers in place of its example names:

REM On the new forest-root PDC emulator (Microsoft's example peer names; use your own)
w32tm /config /manualpeerlist:"ntpserver.contoso.com clock.adatum.com" /syncfromflags:manual /reliable:YES /update

REM On a former PDC that's still a DC (transfer case): follow the domain hierarchy again
w32tm /config /syncfromflags:domhier /update

Windows Time Not Syncing With Domain Controller covers checking that chain afterward. The PDC emulator is also Microsoft's "Default target domain controller for Group Policy updates.", so Group Policy edits now land on the new holder; if they don't show up on clients, Group Policy Not Applying covers that path. Microsoft's placement article also notes that NTDS Replication event 1586 occurs when "the PDC FSMO role for a particular domain has been seized.", so expect it after a PDC seizure.

Step 10: Prove every DC agrees (read-only)

A new holder doesn't start work the moment the command returns. Microsoft: "After a role transfer or seizure, the new role holder doesn't act immediately." It waits for a successful inbound replication cycle of the role's partition, and "To have full effect, the change must replicate to all domain controllers in the domain for a domain-level role and to all domain controllers in the forest for a forest-level role."

# Every DC's answer, not just the one you're signed in to
Get-ADDomainController -Filter * | ForEach-Object {
    $n = $_.Name
    $d = Get-ADDomain -Server $n
    $f = Get-ADForest -Server $n
    [pscustomobject]@{
        AskedDC = $n
        PDC     = $d.PDCEmulator
        RID     = $d.RIDMaster
        Infra   = $d.InfrastructureMaster
        Schema  = $f.SchemaMaster
        Naming  = $f.DomainNamingMaster
    }
} | Format-Table -AutoSize

REM The same check, the dcdiag way, plus RID master health and replication
dcdiag /test:KnowsOfRoleHolders /e
dcdiag /test:RidManager /v
repadmin /replsummary

Microsoft's ntdsutil view of the connected DC's roles is roles > connections > connect to server DC02 > quit > select operation target > list roles for connected server.

Pass criteria (our reading): every DC names the same live holder for every role, the Step 2 fSMORoleOwner read (including both DNS application partitions) shows no deleted owner, repadmin /replsummary shows no failures, dcdiag /test:RidManager passes, and the operation from the ticket (creating an account on the DC that couldn't, a password reset, a Group Policy edit) works. Then close the ticket with the decision from Step 6 written down.

Common false starts

When a change is justified

FAQ

What's the difference between transferring and seizing FSMO roles? A transfer moves a role with the current holder's cooperation, so nothing is lost; Microsoft recommends it when "The current role holder is operational and can be accessed on the network by the new Operation Master owner." A seizure assigns the role without the old holder, so it's for a holder that isn't coming back, and that old DC must not return as it was (Steps 6-8).

Why does FSMO role transfer fail with "The current FSMO holder could not be contacted"? The DC doing the work couldn't get the current holder to take part: it's offline, unreachable from that DC, or it restarted and hasn't completed inbound replication for the role's partition yet. Read replication for the holder and the target with repadmin /replsummary and repadmin /showrepl before you think about seizing (Steps 3 and 6).

Can I bring the old domain controller back after seizing its roles? Not as it was. Microsoft: "Do not allow a former operations master role holder whose role has been seized to return to an online domain controller." Reinstall it or force-demote it, clean up its metadata, then promote it fresh if you want it back as a DC (Step 9).

Why does seizing FSMO roles say "Access is denied"? Usually the account isn't in the group the role needs (Schema Admins for the schema master, Enterprise Admins for the domain naming master, Domain Admins for the domain roles), the membership was added after you signed in, or the prompt isn't elevated. Check whoami /groups in the same window (Step 5). During metadata cleanup, it can also mean the objects are protected from accidental deletion (Step 9).

How do I seize FSMO roles with PowerShell? Use Move-ADDirectoryServerOperationMasterRole with -Identity set to the DC that receives the roles, -OperationMasterRole listing only the roles the dead DC held, and -Force. It "still attempts graceful transfer first, then seizes if transfer is not possible." For the RID master, it raises the next RID pool by 30,000, three times what ntdsutil does (Step 8).

Related posts

• Active Directory Replication Error 1722 (when the role holder can't be reached because replication fails with RPC errors)
• Windows Time Not Syncing With Domain Controller (when the new forest-root PDC emulator has to become the time source)
• The Target Principal Name Is Incorrect (when the holder and the target fail Kerberos between them)
• Group Policy Not Applying (when policy edits stall after the PDC emulator moves)

Free checklist

Seizing all five roles before anyone read who owns them, or whether the old DC is coming back, is a change made everywhere at once. So is a Conditional Access policy enforced before anyone read the report-only results. The free Conditional Access pre-enforcement checklist keeps break-glass accounts, pilot groups, and report-only review on one page:

👉 https://cashflow4375.gumroad.com/l/ujkqxb?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=fsmo-role-transfer-failed

If you want the rest of the Intune runway written down for the team (enrollment hygiene, compliance phasing, break/fix cards and read-only PowerShell snapshots), the Intune & M365 Admin Starter Pack pairs with this triage: https://cashflow4375.gumroad.com/l/joonf?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=fsmo-role-transfer-failed

Admin Pack Studio; not affiliated with Microsoft. Steps 1-6 and 10 are read-only checks: netdom query, Get-ADDomain, Get-ADForest, Get-ADRootDSE, Get-ADObject, dcdiag, repadmin /replsummary and /showrepl, Get-ADReplicationPartnerMetadata, Get-ADReplicationFailure, Test-NetConnection, Get-WinEvent, Get-ADDomainController, Get-ADOptionalFeature, whoami /groups, and Get-ADGroupMember. Transferring or seizing roles, editing fSMORoleOwner, metadata cleanup, demoting or rebuilding a DC, changing group membership, and changing the time configuration are deliberate changes. Change one link at a time, prove it from every DC, and check current Microsoft Learn for version-specific behavior.

Not yet. "Hyper-V live migration failed" is almost never one setting. Event 21502 is the summary Hyper-V logs when a live migration fails (you'll see it most on clustered hosts), and the reason is in the message text under it: a virtual switch name that doesn't exist on the destination, processor features the destination can't offer, a configuration version it doesn't support, migration settings that don't match, an authentication path that can't delegate, a connection to TCP 6600 that never opens, or a destination that can't hold the VM (memory, storage path, ISO, owners). Microsoft says it directly: "live migration can fail due to a wide array of factors", and its troubleshooting page lists about twenty different messages under the single heading Event ID 21502. Turning on processor compatibility for every VM hides CPU features from workloads that didn't need it (and needs each VM shut down), switching to CredSSP trades a delegation problem for a sign-in problem (and stops working on Windows Server 2025 with Credential Guard), any-network migration sends unencrypted VM memory over networks you didn't choose, and turning off the host firewall proves nothing about why the listener on 6600 isn't answering. Work through it as a chain: which VM, which hosts, which inner message → what does the full error say on both hosts? → what does Compare-VM report? → do the virtual switches, adapters, and attached media exist on the destination? → can the destination offer the VM's processor features? → does the destination support the VM's configuration version? → are migration settings and networks the same on both sides? → can the source authenticate to the destination? → does TCP 6600 connect? → can the destination hold the VM? Get-WinEvent on the Hyper-V logs of both hosts, Compare-VM, Get-VMNetworkAdapter, Get-VMSwitch, Get-VMProcessor, Get-VM, Get-VMHostSupportedVersion, Get-VMHost, Get-VMMigrationNetwork, Get-ADComputer, Test-NetConnection, and the failover cluster cmdlets are the sources for that chain. Change processor compatibility, authentication, and networks at once and you'll never know which one was broken.

If the inner message is "The target principal name is incorrect", The Target Principal Name Is Incorrect covers the Kerberos side of that error. If the VM won't move because a checkpoint or its AVHDX chain is broken, Hyper-V Checkpoint Operation Failed covers that side. This post covers "a running Hyper-V VM won't live migrate between Windows Server hosts, clustered or not: Event 21502 in the cluster, 'failed at migration source' or 'failed at migration destination' in Hyper-V Manager, a processor or hardware compatibility error, a failed-to-authenticate or no-credentials error, a failed-to-establish-a-connection error, or a failure that happens one way only, and you need a read-first path before you change processor compatibility, authentication, migration networks, or firewall rules."

TL;DR

• Event 21502 is the envelope, not the letter. Read the inner message and its hex code on both hosts before you change anything (Step 2). Microsoft's first check is a compatibility report: run Compare-VM against the destination (Step 3).
• Switch names must match exactly. Microsoft: "The virtual switch must have the same name on all nodes." (Step 4).
• Processor compatibility is per VM, needs the VM off, and has limits. "Even when processor compatibility is enabled, you can't move VMs between hosts with different processor manufacturers." (Step 5).
• One way only usually means version. A VM upgraded on a newer host can't go back: "This action can't be rolled back." (Step 6).
• Authentication is a choice with consequences. "Kerberos lets you avoid signing in to the server, but requires you to set up constrained delegation." CredSSP needs a sign-in on the source and breaks with Credential Guard on Windows Server 2025 (Step 8).
• Read the listener before the firewall. Microsoft's checklist: "Check whether the TCP Port 6600 and 3343 (for clustering) are listening on both sides." (Step 9).

Where in the chain did it break?

Step 1: Capture the symptom (read-only)

Write these on the ticket first:

Then branch on what you see:

Step 2: Read the full error on both hosts (read-only)

Event 21502 is a summary. The reason is in the lines under it, and often in a second event on the other host. Microsoft's own list for Event ID 21502 runs from a missing virtual switch to processor features, Kerberos errors, memory, ISO files, and storage oplocks, and in Failover Cluster Manager it points you to the VM's Information details > Show details for the full text.

Find the Hyper-V and clustering logs that have events on each host, then pull the migration events around the failed attempt from both the source and the destination:

# On the source and on the destination: which Hyper-V and cluster logs have events here
Get-WinEvent -ListLog '*Hyper-V*', '*FailoverClustering*' -ErrorAction SilentlyContinue |
    Where-Object { $_.RecordCount -gt 0 } |
    Sort-Object LogName |
    Format-Table LogName, RecordCount -AutoSize

# The migration events from the last hour, oldest first, with the full message text
$logs = (Get-WinEvent -ListLog '*Hyper-V*' -ErrorAction SilentlyContinue |
    Where-Object { $_.RecordCount -gt 0 }).LogName
Get-WinEvent -FilterHashtable @{
    LogName   = $logs
    Id        = 21502, 21501, 21125, 21024, 20413, 20417, 10698
    StartTime = (Get-Date).AddHours(-1)
} -ErrorAction SilentlyContinue |
    Sort-Object TimeCreated |
    Format-List TimeCreated, MachineName, LogName, Id, Message

The event IDs in that filter are the ones Microsoft's live migration troubleshooting pages describe: 21502 (the failure itself), 21501 (migration limit), 21125 (configuration setup failed on the destination), 21024 (lost network after migration), 20413 (configuration registration failed on the destination), 20417 (long blackout time), and 10698 (listed with 21502 for configuration version problems). Widen the time window or drop the Id filter if nothing comes back; the inner error is often logged under a different ID on the other host. Microsoft's data collection list also names the worker log ("Microsoft-Windows-Hyper-V-Worker-Admin") and, for clusters, Get-ClusterLog -UseLocalTime -Destination <Path>:

# Clustered hosts only: write the cluster log for every node, in local time, to one folder
Get-ClusterLog -UseLocalTime -Destination C:\Temp\ClusterLogs

Read the hex code first. Microsoft's 21502 entries map them like this:

Step 3: What does Compare-VM report? (read-only)

Microsoft's basic checklist says to "Check compatibilities issues by running a Compare-VM command." The cmdlet doesn't move anything: "The Compare-VM cmdlet compares a virtual machine and a virtual machine host for compatibility, returning a compatibility report." Run it on the source host, against the destination, for the VM that failed:

# On the source host: compare the running VM with the destination host
$report = Compare-VM -Name APP01 -DestinationHost HV02

# Each incompatibility has a message, an ID, and the object it came from
$report.Incompatibilities | Format-Table Message, MessageId, Source -AutoSize -Wrap

# If the move includes storage, compare that move instead
$report = Compare-VM -Name APP01 -DestinationHost HV02 -IncludeStorage -DestinationStoragePath D:\VMs\APP01
$report.Incompatibilities | Format-Table Message, MessageId, Source -AutoSize -Wrap

Message, MessageId, and Source are the columns Microsoft's own Compare-VM example shows, where the report says "Could not find Ethernet switch 'Production'." for a VM whose adapter used a switch the host didn't have. An empty Incompatibilities list means the destination can take the VM's configuration, so look at Steps 7-10 (settings, authentication, connection, capacity). A non-empty list usually tells you which of Steps 4-6 to read next (our reading). If Compare-VM itself fails with a credentials or connection error, that's the same problem the migration has: go to Steps 8 and 9.

Step 4: Do the switches, adapters, and attached media exist on the destination? (read-only)

The first entry under Event ID 21502 on Microsoft's live migration page is a switch: "Live migration of VM failed. VM failed to live migrate because a virtual switch used by the VM doesn't exist on the destination node". The action is short: "The virtual switch must have the same name on all nodes." Microsoft also notes "If a virtual machine is connected to a virtual switch that doesn't exist on the destination node, then live migration fails at < 10%."

# On the source: which switch each of the VM's adapters uses
Get-VMNetworkAdapter -VMName APP01 | Format-Table VMName, Name, SwitchName -AutoSize

# On both hosts: the switches that exist, by exact name and type
Get-VMSwitch | Sort-Object Name | Format-Table Name, SwitchType -AutoSize
Get-VMSwitch -ComputerName HV02 | Sort-Object Name | Format-Table Name, SwitchType -AutoSize

# On the source: anything attached to the VM's DVD drives
Get-VMDvdDrive -VMName APP01 | Format-Table VMName, ControllerType, ControllerNumber, Path -AutoSize

What to look for:

In a cluster, after you rename or create a switch, Microsoft's steps end with Failover Cluster Manager > the VM role > More Actions > Refresh virtual machine configuration before you retry. That refresh is a change (Step 11), not a check.

Microsoft's broader checklist adds team consistency: "Validate network teaming (SET or LBFO) consistency." A switch with the right name bound to a team with a different configuration is worth a second look (our reading).

Step 5: Can the destination offer the VM's processor features? (read-only)

A running VM sees the features of the host it started on. Microsoft explains why that blocks a move: "If software in the VM detects and starts using a particular processor capability, and then the VM is moved to a new virtualization host without that capability, the software can fail." So "Hyper-V performs initial checks whenever a VM live migration or save/restore operation is initiated", and "If these feature sets don't match, the migration or restore operation is canceled."

# On the source: is processor compatibility turned on for this VM, and in which mode
Get-VMProcessor -VMName APP01 | Format-List VMName, Count, CompatibilityFor*

# Same check for every VM on the host
Get-VM | Get-VMProcessor | Format-Table VMName, Count, CompatibilityForMigrationEnabled -AutoSize

# Does the VM have checkpoints (Microsoft ties this to the hardware-not-compatible error)
Get-VMSnapshot -VMName APP01 | Format-Table VMName, Name, SnapshotType -AutoSize

Read the result against the messages:

Two rules from Microsoft keep this from turning into a fleet-wide change. First, "You don't need to enable processor compatibility mode if you plan to stop and restart the VMs", because the guest reads the new host's features at boot. Second, "Processor compatibility mode can't be enabled or disabled while the VM is running." So for a running VM that has to move now, the choice is a short shutdown either way; the guide's tip is "Always start VMs for the first time on the oldest (least capable) CPU host." The CompatibilityFor* wildcard is our form; it shows CompatibilityForMigrationMode only where the host's Hyper-V module has it.

Step 6: Does the destination support the VM's configuration version? (read-only)

Microsoft's troubleshooting guide describes the pattern exactly: "Migration works one way but not in reverse (especially after moving to a newer OS)." with Event IDs 10698 and 21502. The VM's configuration version decides which hosts can run it:

# On the source: configuration version of every VM (Microsoft's form)
Get-VM * | Format-Table Name, Version

# On the destination: which configuration versions it can run, and its default
Get-VMHostSupportedVersion -ComputerName HV02
Get-VMHostSupportedVersion -ComputerName HV02 -Default

# On the source: integration services that report a version mismatch
Get-VMIntegrationService -VMName APP01 |
    Format-Table VMName, Name, SecondaryOperationalStatus -AutoSize

Don't upgrade a VM's configuration version to make a mixed-version move work. Microsoft's guidance is not to upgrade until "You upgrade your Hyper-V hosts to the latest version of Windows or Windows Server.", "You upgrade the cluster functional level.", and "You're sure that you won't need to move the virtual machine back to a Hyper-V host that runs a previous version of Windows or Windows Server." The troubleshooting guide adds: "This action can't be rolled back. VMs with the upgraded version can't be migrated back to older hosts." During a rolling upgrade, keep VMs at the version the oldest host supports until the last old host is gone (our reading).

Step 7: Are migration settings and networks the same on both sides? (read-only)

Microsoft's basic checklist has two lines for this: "Check whether the migration is authorized on each side." and "Check whether the protocol used is identical on each side." Read the host settings on both hosts, then the networks each one accepts migrations on:

# On both hosts: every live migration setting (the wildcard is our form)
Get-VMHost | Format-List Name, *Migration*
Get-VMHost -ComputerName HV02 | Format-List Name, *Migration*

# Networks each host accepts incoming live migration on
Get-VMMigrationNetwork
Get-VMMigrationNetwork -ComputerName HV02

# Clustered hosts: cluster networks and their role, the cluster-wide limit,
# and which networks the Virtual Machine resource type excludes from migration
Get-ClusterNetwork | Format-Table Name, Role, Address -AutoSize
(Get-Cluster).MaximumParallelMigrations
Get-ClusterResourceType -Name "Virtual Machine" | Get-ClusterParameter -Name MigrationExcludeNetworks

What the settings mean, from Microsoft's Set-VMHost reference:

Read the result:

Microsoft's cluster network guidance sets the rule for every host involved: "All Hyper-V hosts that can initiate or receive a live migration must have connectivity to a network that is configured to allow live migrations." It also explains why "allow any network" is not a free fix: "live migration traffic isn't encrypted when sent over the network", so isolate it "onto trusted, private networks". In a cluster, "By default, live migration traffic uses the cluster network topology to discover available networks and to establish priority." The order and exclusions you set in Failover Cluster Manager (right-click Networks > Live Migration Settings) are what MigrationExcludeNetworks shows.

Step 8: Can the source authenticate to the destination? (read-only)

The source host opens the migration connection to the destination, so it needs credentials it can present there. Microsoft's setup page gives the two options and their cost: "Kerberos lets you avoid signing in to the server, but requires you to set up constrained delegation." and "CredSSP lets you avoid setting up constrained delegation, but requires you to sign in to the source server." When neither works you get this, from the same page:

Virtual machine migration operation failed at migration Source.
Failed to establish a connection with host computer name: No credentials are available in the security package 0x8009030E.

Read which protocol each host uses, what each host may delegate to in AD, and what the account starting the move allows:

# Which protocol each host uses for live migration
Get-VMHost -ComputerName HV01, HV02 |
    Format-Table Name, VirtualMachineMigrationAuthenticationType -AutoSize

# Kerberos: what each host may delegate to (Microsoft's blog form; run for both hosts)
Get-ADComputer -Identity HV01 -Properties msDS-AllowedToDelegateTo |
    Select-Object -ExpandProperty msDS-AllowedToDelegateTo
Get-ADComputer -Identity HV02 -Properties msDS-AllowedToDelegateTo |
    Select-Object -ExpandProperty msDS-AllowedToDelegateTo

# Protocol transition ("Use any authentication protocol") on each host's account
Get-ADComputer -Identity HV01 -Properties TrustedToAuthForDelegation |
    Format-Table Name, TrustedToAuthForDelegation -AutoSize

# The admin account that starts the move: marked sensitive, or in Protected Users?
Get-ADUser -Identity jsmith-adm -Properties AccountNotDelegated |
    Format-Table Name, AccountNotDelegated -AutoSize
Get-ADGroupMember -Identity "Protected Users" | Format-Table Name, SamAccountName -AutoSize

REM SPNs registered on each host (Microsoft's data collection form)
setspn -L HV01
setspn -L HV02

Read the result:

Two timing notes. Delegation changes take effect after "The changes replicate to the domain controllers that the servers running Hyper-V are logged into." and "The domain controller issues a new Kerberos ticket." Microsoft's 0x8009030D fix says to "Wait up to 15 minutes for Keberos tickets to time out" (sic) or purge the computer's tickets (Step 11). A fix that "didn't work" five minutes after the change may simply not have reached the host yet (our reading). Microsoft's blog also confirms that Hyper-V Manager and Move-VM fail the same way, so switching tools doesn't help.

Step 9: Does TCP 6600 connect from the source to the destination? (read-only)

Microsoft's checks use TCP 6600 for the Hyper-V migration listener and 3343 for clustering. Microsoft's form of the check is netstat -ano | findstr /I /C:"6600". The same reading in PowerShell, plus a test from the source:

# On the destination: is the Hyper-V Virtual Machine Management service running and listening
Get-Service -Name vmms | Format-Table Name, Status, StartType -AutoSize
Get-NetTCPConnection -LocalPort 6600 -State Listen -ErrorAction SilentlyContinue |
    Format-Table LocalAddress, LocalPort, State, OwningProcess -AutoSize

# Clustered nodes: the cluster service port
Get-NetTCPConnection -LocalPort 3343 -State Listen -ErrorAction SilentlyContinue |
    Format-Table LocalAddress, LocalPort, State -AutoSize

# From the source: does 6600 answer by name and on the migration network address
Test-NetConnection -ComputerName HV02 -Port 6600
Test-NetConnection -ComputerName 192.168.10.12 -Port 6600

# On both hosts: the Hyper-V firewall rules and their state (the wildcard is our form)
Get-NetFirewallRule -DisplayName '*Hyper-V*' |
    Format-Table DisplayName, Enabled, Direction, Action -AutoSize

Read the result:

Test-NetConnection -Port tests TCP only. The listener answering by name but not on the migration address (or the reverse) is the most useful split you can get here: it tells you whether the problem is the path or the host (our reading).

Step 10: Can the destination hold the VM? (read-only)

The last group of 21502 messages is about the destination itself: memory, the storage path, cluster ownership, user rights, and leftover state.

# On the source: the VM's memory settings
Get-VMMemory -VMName APP01 | Format-List *

# On the destination: what's already running and how much memory it holds
Get-VM -ComputerName HV02 | Where-Object { $_.State -eq 'Running' } |
    Format-Table Name, State, MemoryAssigned -AutoSize

# Where the VM's disks live, and whether the destination path exists
Get-VMHardDiskDrive -VMName APP01 | Format-Table VMName, ControllerType, Path -AutoSize
Invoke-Command -ComputerName HV02 -ScriptBlock { Test-Path -Path 'D:\VMs' }

# Clustered: which nodes may own the VM and its configuration (Microsoft's form)
Get-ClusterResource -Name "Virtual Machine APP01" | Get-ClusterOwnerNode
Get-ClusterResource -Name "Virtual Machine Configuration APP01" | Get-ClusterOwnerNode
Get-ClusterNode | Format-Table Name, State -AutoSize

Read the result:

MemoryAssigned, Path, and the other property lists above are our forms; Microsoft's own wording for the memory case is simply "Check the available memory on the destination host."

Common false starts

Step 11: Change one link, then prove it

Before any change, write down what you're about to change: the Get-VMHost migration settings, the switch list, the VM's processor and version settings, and the delegation lists from Steps 4-8. Then change only the link Steps 1-10 pointed to.

Switches and media (Step 4). Create or rename the switch on the destination so it matches the name the VM uses, with the same type and uplink, following Microsoft's resolution: "If the virtual switch doesn't exist, create or rename it with the same name and settings such as in other hosts." In a cluster, refresh the VM configuration in Failover Cluster Manager afterward. For an ISO the destination can't open, eject it:

# On the source: remove the ISO from the VM's DVD drive (Microsoft's fix is to select None)
Get-VMDvdDrive -VMName APP01 | Set-VMDvdDrive -Path $null

For a clustered VM on an Internal or Private switch, Microsoft's workaround is to clear the adapter's Protected network option, which in PowerShell is Set-VMNetworkAdapter -NotMonitoredInCluster $true for that adapter. That stops the cluster from checking that adapter's connectivity, so use it only for adapters that are meant to be host-local (our reading).

Processor (Step 5). Shut the VM down, turn on compatibility for that VM only, start it, and move it:

# Microsoft's forms; the VM must be off
Stop-VM -Name APP01
Set-VMProcessor -VMName APP01 -CompatibilityForMigrationEnabled $true
# Windows Server 2025 cluster, configuration version 10.0 or later: dynamic mode instead
# Set-VMProcessor -VMName APP01 -CompatibilityForMigrationEnabled $true -CompatibilityForMigrationMode CommonClusterFeatureSet
Start-VM -Name APP01

After the hardware refresh is done, Microsoft recommends turning it back off so the VM can use the full processor; that's another short shutdown. If BIOS settings or Get-SpeculationControlSettings results differ between hosts, fix the host that's out of line instead of masking it on every VM. Across microcode levels, move old to new.

Configuration version (Step 6). Don't downgrade; you can't. Keep the VM on hosts that support its version, or, once every host in its path runs the newer Windows Server and the cluster functional level is raised, upgrade it with Microsoft's Update-VMVersion <vmname> (VM shut down, one-way).

Settings and networks (Step 7). Enable migration on the host that's missing it, make the authentication protocol the same on both hosts, and add the shared migration network to the destination. Microsoft's PowerShell form:

# On each host that needs it (Microsoft's three-cmdlet example)
Enable-VMMigration
Set-VMMigrationNetwork 192.168.10.1
Set-VMHost -VirtualMachineMigrationAuthenticationType Kerberos

Raise the simultaneous migration limit only if the host and the network can take it; Microsoft's note on that setting is "Consider host performance when changing this number." In a cluster, change the live migration network order in Failover Cluster Manager (Networks > Live Migration Settings) or the MigrationExcludeNetworks parameter, not each host. For a cross-cluster move that picked a private network, Microsoft's fix is "Change the Live Migration Network order for cross-cluster and change it back after the migration."

Authentication (Step 8). Prefer Kerberos with constrained delegation. With a Domain Admins account, on the source host's computer object in Active Directory Users and Computers: Delegation tab > Trust this computer for delegation to the specified services only > Use any authentication protocol > Add the destination host, and select cifs and Microsoft Virtual System Migration Service. Then repeat on the destination for the source. Set both hosts to Kerberos (Step 7 example). Then let the change take effect, either by waiting or, on each host, with Microsoft's ticket purge for the computer account's logon session:

REM Purge the host's own (SYSTEM, 0x3e7) Kerberos tickets so it requests new ones with the delegation change
klist purge -li 0x3e7

Microsoft's klist reference warns: "Purging tickets destroys all tickets that you have cached, so use this attribute with caution." On Windows Server 2025, don't disable Credential Guard to bring CredSSP back; Microsoft's resolution is that "Kerberos Constrained Delegation and Resource-Based Kerberos Constrained Delegation are recommended" instead. If the admin account is in Protected Users or marked sensitive, start the move while signed in on the source host, or use an account meant for that, rather than removing the protection (our reading). If the Microsoft Virtual System Migration Service SPN is missing, Microsoft's guide registers it with setspn -s Microsoft Virtual System Migration Service/<FQDN> <ComputerName>.

Connection (Step 9). If the listener was disabled by Group Policy, run gpupdate /force on that host and read the result; if Registry.pol is corrupt, Microsoft's fix is to restore it from a backup or copy the GroupPolicy folder from a working server. Open TCP 6600 between the hosts on the migration network (and 3343 between cluster nodes) in whatever firewall sits between them. Add Microsoft's Hyper-V process exclusions to third-party antivirus.

Destination capacity (Step 10). Free memory on the destination or pick another one; add the missing node to the possible owners of both cluster resources; give the destination the same system drive letter as the other nodes; restore the default user rights for the NT VIRTUAL MACHINE\Virtual Machines group in the GPO that removed them; fix the folder permissions on the destination path to match a working host.

Then prove it. Run Compare-VM -Name APP01 -DestinationHost HV02 again and confirm the Incompatibilities list is empty, then move the VM the way the user did (same tool, same host it was started from):

# Standalone hosts (Microsoft's Move-VM form; add -IncludeStorage for a storage move)
Move-VM -Name APP01 -DestinationHost HV02

# Clustered VM (Microsoft's form)
Move-ClusterVirtualMachineRole -Name "APP01" -Node HV02 -MigrationType Live

Pass criteria (our reading): the move completes, Get-VM -ComputerName HV02 shows the VM Running there, no new 21502 or worker errors on either host for that time, the VM's network works on the destination (no Event 21024), and the move back to the source works too. Then retry the other VMs from the ticket.

When a change is justified

FAQ

What is Event ID 21502 in Hyper-V? It's the summary event Hyper-V logs when a live migration fails (most often seen on clustered hosts), and the real reason is in its message text. Microsoft lists about twenty different causes under that one ID, from "a virtual switch used by the VM doesn't exist on the destination node" to processor features, Kerberos errors, memory, and ISO files. Read the full message on both hosts and run Compare-VM (Steps 2 and 3).

Why does Hyper-V live migration fail with a processor compatibility error? The destination can't offer every CPU feature the running VM saw when it started. Turn on processor compatibility for that VM (it has to be off to change the setting), make the hosts' BIOS settings and speculation control results match, and move from older to newer microcode. It won't help between Intel and AMD hosts (Step 5).

Should I use CredSSP or Kerberos for live migration? Kerberos with constrained delegation, set on both hosts for each other with cifs and Microsoft Virtual System Migration Service. CredSSP only works when you're signed in on the source host, and Microsoft says "you can't use CredSSP-based live migration with Hyper-V after upgrading to Windows Server 2025" with Credential Guard on (Step 8).

Why does live migration fail with 0x8009030E "No credentials are available in the security package"? Either CredSSP is in use and the move was started from somewhere other than the source host, or Kerberos is in use and delegation is missing, one-way, set to "Kerberos only", pointed at the wrong service, or blocked because the admin account can't be delegated (Protected Users or sensitive). Read msDS-AllowedToDelegateTo for both hosts (Step 8).

Why does live migration work one way but not the other? Usually the VM's configuration version was upgraded on the newer host, or the newer host has CPU features or microcode the older host lacks. Compare Get-VM versions with Get-VMHostSupportedVersion on the older host and check processor compatibility (Steps 5 and 6). With CredSSP, it can also be because you're signed in on one host but not the other.

Related posts

• The Target Principal Name Is Incorrect (when the migration fails with 0x80090322 and the Kerberos name is the problem)
• Hyper-V Checkpoint Operation Failed (when a broken checkpoint, saved state, or AVHDX chain blocks the move)
• Group Policy Not Applying (when the migration listener is disabled because policy processing failed)
• Active Directory Replication Error 1722 (the same name, port, and firewall reading for RPC between servers)

Free checklist

Turning on processor compatibility, CredSSP, and any-network migration on every host before anyone read the inner error is a change made everywhere at once. So is a Conditional Access policy enforced before anyone read the report-only results. The free Conditional Access pre-enforcement checklist keeps break-glass accounts, pilot groups, and report-only review on one page:

👉 https://cashflow4375.gumroad.com/l/ujkqxb?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=hyper-v-live-migration-failed

If you want the rest of the Intune runway written down for the team (enrollment hygiene, compliance phasing, break/fix cards and read-only PowerShell snapshots), the Intune & M365 Admin Starter Pack pairs with this triage: https://cashflow4375.gumroad.com/l/joonf?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=hyper-v-live-migration-failed

Admin Pack Studio; not affiliated with Microsoft. Steps 1-10 favor read-only checks: Get-WinEvent, Get-ClusterLog, Compare-VM, Get-VMNetworkAdapter, Get-VMSwitch, Get-VMDvdDrive, Get-VMProcessor, Get-VMSnapshot, Get-VM, Get-VMHostSupportedVersion, Get-VMIntegrationService, Get-VMHost, Get-VMMigrationNetwork, Get-ClusterNetwork, Get-Cluster, Get-ClusterParameter, Get-ADComputer, Get-ADUser, Get-ADGroupMember, setspn -L, Get-Service, Get-NetTCPConnection, Test-NetConnection, Get-NetFirewallRule, Get-VMMemory, Get-VMHardDiskDrive, Test-Path, Get-ClusterResource, Get-ClusterOwnerNode, and Get-ClusterNode. Writing the cluster log creates files on every node. Changing switches, DVD media, processor compatibility, configuration versions, migration settings and networks, cluster network order, AD delegation, SPNs, Kerberos tickets, Group Policy, firewall rules, antivirus exclusions, user rights, cluster owners, or VM memory, and stopping, starting, or moving VMs, are deliberate changes. Change one link at a time, prove it with Compare-VM and a move in both directions, and check current Microsoft Learn for version-specific behavior.

Not yet. "RD Gateway not working" is almost never the role itself. A connection through Remote Desktop Gateway is a chain of separate checks, and each one fails with a different symptom: the public name resolves to the gateway and TCP 443 reaches it, the certificate it serves matches that name and is trusted and current on the client, the TSGateway service is listening, the user passes a connection authorization policy (CAP), the target passes a resource authorization policy (RAP), NPS and the MFA extension answer inside the RADIUS timeout, and the gateway can reach the target on 3389. Microsoft states the authorization part plainly: "For users to be granted access to network resources through a Remote Desktop Gateway, they must meet the conditions specified in one RD Connection Authorization Policy (RD CAP) and one RD Resource Authorization Policy (RD RAP)." Reinstalling the role throws away the CAPs, RAPs, and certificate binding you need to read, turning off the firewall proves nothing about the edge firewall in front of it, an allow-everything CAP and RAP opens the gateway to the internet, and removing the NPS extension removes MFA instead of fixing it. Work through it as a chain: which user, which client, which name, which error → does the public name reach the gateway on 443? → is the certificate it serves the right one, trusted, current, and checkable? → is the gateway service healthy and listening? → does the user pass a CAP? → does the target pass a RAP? → do NPS and the MFA extension answer in time? → can the gateway reach the target? → is UDP 3391 working, and why is Gateway Manager empty? Resolve-DnsName, Test-NetConnection, Test-Certificate, Get-RDCertificate, netsh http show sslcert, Get-CimInstance against the RD Gateway WMI classes, and Get-WinEvent on the TerminalServices-Gateway log are the sources for that chain. Change the certificate, the CAP, and the firewall at once and you'll never know which one was broken.

If the session host itself refuses users after the gateway lets them through with a licensing message, Remote Desktop Licensing Mode Is Not Configured covers that side. If NPS logs Event 6273 for the gateway's RADIUS requests, NPS Event 6273 Reason Code 16 or 22 covers reason codes. This post covers "users can't connect through a Windows Server RD Gateway (usually from outside the network): 'temporarily unavailable', a certificate expired or untrusted warning, an authorization failure, no MFA prompt, a session that fails after authorization, or an empty RD Gateway Manager, and you need a read-first path before you reinstall the role, replace the certificate, open the policies, or change NPS."

TL;DR

• Inside working proves little. Internal clients may skip the gateway entirely: BypassLocal "Indicates whether authorized remote users bypass the RD Gateway server for local connections." Test from outside (Step 2).
• The certificate has to match the name clients type. Microsoft: "The name of the certificate must match the FQDN used to access RD Gateway" (Step 3). A renewed certificate that isn't bound to 443 is still the old one.
• Two policies, two events. A CAP failure is event 201, a RAP failure is event 301; read the TerminalServices-Gateway log before you touch either (Steps 5 and 6).
• MFA needs time and the right methods. Microsoft's RD Gateway guide says to change the RADIUS timeout "from 3 to a value between 30 and 60 seconds", and SMS codes don't work through the gateway (Step 7).
• Authorized but not connected is the target, not the gateway. Event 304 means the user passed both policies and the gateway couldn't reach the computer (Step 8).
• UDP 3391 is a second transport, not the only one. Read which transport live connections use before you blame UDP (Step 9).

Where in the chain did it break?

Step 1: Capture the symptom (read-only)

Write these on the ticket first:

Then branch on what you see:

Step 2: Does the public name reach the gateway on TCP 443? (read-only)

Microsoft's role page sets the public requirement: "The RD Gateway virtual machine must be accessible through a public IP address that allows inbound TCP connections to port 443 and inbound UDP connections to port 3391." Test it from where the users are, not from your desk. On an outside PC (a home connection or a phone hotspot):

# Read-only. From an OUTSIDE client: where does the public name point, and does TCP 443 answer? (name is an example)
Resolve-DnsName -Name rdg.contoso.com -Type A
Test-NetConnection -ComputerName rdg.contoso.com -Port 443

Then read what the deployment tells clients, on the RD Connection Broker (deployments managed from Server Manager):

# Read-only. On the RD Connection Broker: gateway mode, logon method, and the external FQDN handed to clients
Get-RDDeploymentGatewayConfiguration

Its documented output is Gatewaymode (Automatic, DoNotUse, or Custom), LogonMethod, and GatewayExternalFQDN. The matching Set- cmdlet also has -BypassLocal, which "Indicates whether authorized remote users bypass the RD Gateway server for local connections." With that on, an inside test can succeed without ever touching the gateway (our reading), so "it works from the office" is not a gateway test.

Test-NetConnection -Port only proves TCP: its -Port parameter "Specifies the TCP port number on the remote computer." UDP 3391 needs a different test (Step 9).

Step 3: Is the certificate it serves the right one, trusted, current, and checkable? (read-only)

This is where most "it broke after the renewal" tickets end. The client checks the certificate the gateway serves on 443, not the one you imported last week. Microsoft's requirements for the gateway certificate (from the original TS Gateway guidance, still the clearest list): "The name in the Subject line of the server certificate (certificate name, or CN) must match the DNS name that the client uses to connect to the TS Gateway server, unless you are using wildcard certificates or the SAN attributes of certificates.", "The certificate has not expired.", and "The certificate must be trusted on clients." The current RDS certificates page adds the usage: "Issued for Server Authentication (EKU 1.3.6.1.5.5.7.3.1)."

On the RD Gateway, compare the binding, the gateway's own setting, and the store:

# Read-only. On the RD Gateway: every HTTP.sys SSL binding (look at the :443 entries and their certificate hash)
netsh http show sslcert

# The certificate hash the RD Gateway service uses (CertHash), as a thumbprint string (our form)
$gw = Get-CimInstance -Namespace root/CIMV2/TerminalServices -ClassName Win32_TSGatewayServerSettings
$thumb = ($gw.CertHash | ForEach-Object { $_.ToString('X2') }) -join ''
$thumb

# Server authentication certificates in the computer store: names, expiry, thumbprints
Get-ChildItem -Path Cert:\LocalMachine\My -SSLServerAuthentication |
  Select-Object Subject, DnsNameList, NotAfter, Thumbprint, Issuer | Format-List

# Does the certificate the gateway uses pass SSL policy for the public name? (name is an example)
Test-Certificate -Cert "Cert:\LocalMachine\My\$thumb" -Policy SSL -DNSName rdg.contoso.com

netsh http show sslcert "Lists SSL server certificate bindings along with their associated client certificate policies." CertHash "Specifies the certificate hash for HTTPS binding on port 443 in IIS." Test-Certificate -DNSName "Specifies the DNS name to verify as valid for the certificate." In a deployment managed from Server Manager, read what the deployment thinks too:

# Read-only. On the RD Connection Broker: the certificate assigned to the RD Gateway role
Get-RDCertificate -Role RDGateway | Format-List Subject, SubjectAlternateName, IssuedBy, ExpiresOn, Thumbprint, Level

Test-Certificate runs on the gateway, which trusts your internal CA and can reach your internal CRLs. A home PC can't. So check one failing outside client as well: export the gateway certificate's public part (.cer, no private key) and run Microsoft's chain check there:

:: Read-only. On a failing OUTSIDE client, with the exported gateway certificate (file name is an example)
certutil -urlfetch -verify rdg.cer

-urlfetch is documented as "Retrieve and verify AIA Certs and CDP CRLs." So it shows whether that client can build the chain and download the CRL from where it sits.

Step 4: Is the gateway service healthy and listening? (read-only)

# Read-only. On the RD Gateway: the gateway service and the local NPS service it uses
Get-Service -Name TSGateway, IAS

# Listeners: TCP 443 (HTTPS) and UDP 3391 (RDP over UDP)
Get-NetTCPConnection -LocalPort 443 -State Listen
Get-NetUDPEndpoint -LocalPort 3391 -ErrorAction SilentlyContinue

# Gateway settings from its WMI provider (namespace Root\CIMv2\TerminalServices)
$gw = Get-CimInstance -Namespace root/CIMV2/TerminalServices -ClassName Win32_TSGatewayServerSettings
$gw | Select-Object IsConfigured, CentralCAPEnabled, MaxConnections, UnlimitedConnections, SslBridging, EnforceChannelBinding

# Read-only. Which transports are enabled and where they listen: 0 = RPC over HTTP, 1 = HTTP, 2 = UDP (our form)
foreach ($t in 0, 1, 2) {
  $on = Invoke-CimMethod -InputObject $gw -MethodName IsTransportEnabled -Arguments @{ TransportType = [uint16]$t }
  $ep = Invoke-CimMethod -InputObject $gw -MethodName GetIPAndPort -Arguments @{ TransportType = [uint16]$t }
  [pscustomobject]@{ Transport = $t; Enabled = $on.Enabled; IPAddress = $ep.IPAddress; Port = $ep.Port }
}

# Errors and warnings in the gateway log, last two days
Get-WinEvent -FilterHashtable @{ LogName = 'Microsoft-Windows-TerminalServices-Gateway/Operational'; Level = 2, 3; StartTime = (Get-Date).AddDays(-2) } -MaxEvents 30 -ErrorAction SilentlyContinue |
  Select-Object TimeCreated, Id, Message | Format-List

The RD Gateway WMI classes need an elevated session: "You must be a member of the Administrators group to use this class." The NPS service shows up as IAS (Microsoft's NPS extension guide says to "Restart the Network Policy Server (IAS) service."), and it's on the gateway by design: "By default, the role is already present on the computer configured as the Remote Desktop Gateway."

Step 5: Does the user pass a connection authorization policy (CAP)? (read-only)

The gateway logs every authorization decision. Microsoft's RD Gateway guide reads the success events with Get-WinEvent on Microsoft-Windows-TerminalServices-Gateway/Operational; read the failures the same way, for one user, around the failure time:

# Read-only. On the RD Gateway: CAP, RAP, and connection events for one user, last 4 hours (user name is an example)
Get-WinEvent -FilterHashtable @{ LogName = 'Microsoft-Windows-TerminalServices-Gateway/Operational'; Id = 200, 201, 300, 301, 302, 303, 304; StartTime = (Get-Date).AddHours(-4) } -ErrorAction SilentlyContinue |
  Where-Object { $_.Message -like '*jsmith*' } |
  Select-Object TimeCreated, Id, Message | Format-List

Microsoft's event table for the gateway (written for TS Gateway, same IDs): 200 "Successful Connection Authorization", 201 "Failed Connection Authorization", 300 "Successful Resource Authorization", 301 "Failed Resource Authorization", 302 "Successful User Connection to the Resource", 303 when the client disconnects, 202 when an administrator disconnects it, and 304 "Failed User Connection to the Resource". A good attempt reads 200, 300, 302 in that order.

Then read the local CAPs in the order the gateway evaluates them:

# Read-only. On the RD Gateway: local CAPs, in evaluation order
Get-CimInstance -Namespace root/CIMV2/TerminalServices -ClassName Win32_TSGatewayConnectionAuthorizationPolicy |
  Sort-Object Order |
  Select-Object Order, Name, Enabled, UserGroupNames, ComputerGroupNames, PasswordAllowed, SmartcardAllowed, DeviceRedirectionType, AllowOnlySDRServers | Format-List

Step 6: Does the target pass a resource authorization policy (RAP)? (read-only)

RAPs always live on the gateway itself: "RD RAPs can't use a central policy, as they're processed on the RD Gateway." Read them, and the resource groups they point at:

# Read-only. On EACH RD Gateway: RAPs and the resource groups behind them
Get-CimInstance -Namespace root/CIMV2/TerminalServices -ClassName Win32_TSGatewayResourceAuthorizationPolicy |
  Select-Object Name, Enabled, UserGroupNames, ResourceGroupType, ResourceGroupName, PortNumbers | Format-List
Get-CimInstance -Namespace root/CIMV2/TerminalServices -ClassName Win32_TSGatewayResourceGroup |
  Select-Object Name, AssociatedRAPs, Resources | Format-List

ResourceGroupType is RG (a gateway-managed group, listed in Win32_TSGatewayResourceGroup), CG (an Active Directory computer group), or ALL. Resources is a "Semicolon-separated list of resources in this resource group."

Step 7: Do NPS and the MFA extension answer in time? (read-only)

Skip this step if CentralCAPEnabled is False and no MFA is in the path. With a central NPS (the usual MFA design), the gateway is a RADIUS client: "Acting as a RADIUS client, the Remote Desktop Gateway server converts the request to a RADIUS Access-Request message and sends the message to the RADIUS (NPS) server where the NPS extension is installed." Read which NPS servers it uses:

# Read-only. On the RD Gateway: central NPS servers and their order (the class also exposes SharedSecret; don't select it)
Get-CimInstance -Namespace root/CIMV2/TerminalServices -ClassName Win32_TSGatewayRADIUSServer | Select-Object Name, Priority

Priority matters: "The RD Gateway server looks for RD CAPs on RADIUS servers based on the priority." Then open the NPS console on the gateway and read the timeout Microsoft tells you to change: NPS (Local) → RADIUS Clients and Servers → Remote RADIUS Server → TS GATEWAY SERVER GROUP → the server → Edit → Load Balancing. Microsoft: "change the default value from 3 to a value between 30 and 60 seconds". Three seconds isn't enough for a person to find a phone and approve a push.

On the NPS server where the extension is installed:

# Read-only. On the central NPS server: NPS decisions (6272 granted, 6273 denied) for the last 4 hours
Get-WinEvent -FilterHashtable @{ LogName = 'Security'; Id = 6272, 6273; StartTime = (Get-Date).AddHours(-4) } -MaxEvents 40 -ErrorAction SilentlyContinue |
  Select-Object TimeCreated, Id, Message | Format-List

# The NPS extension's own logs (AzureMfa): list them, then read the channels it shows (our form)
Get-WinEvent -ListLog '*AzureMfa*' -ErrorAction SilentlyContinue | Select-Object LogName, RecordCount

# Can this server reach the MFA endpoints on TCP 443?
Test-NetConnection -ComputerName login.microsoftonline.com -Port 443
Test-NetConnection -ComputerName strongauthenticationservice.auth.microsoft.com -Port 443
Test-NetConnection -ComputerName adnotifications.windowsazure.com -Port 443

:: Read-only. On the NPS server: is NPS auditing on? (Microsoft's form)
auditpol /get /subcategory:"Network Policy Server"

Microsoft's NPS checklist says to look for 6273 or 6274: "Most authentication failures produce these events." The extension's logs are under Applications and Services Logs → Microsoft → AzureMfa, and the endpoints come from the extension's network requirements: "The NPS server must be able to communicate with the following URLs over TCP port 443:".

Discarded duplicate requests in the NPS log are expected while a user is still answering the prompt: "Discarded requests in the NPS server event log don't indicate there's a problem with the NPS server or the Microsoft Entra multifactor authentication NPS extension."

Step 8: Can the gateway reach the target, and does the target accept the user? (read-only)

Once events 200 and 300 are logged, the gateway's job is to open RDP to the target from inside. Microsoft's ports page lists that leg: "TCP and UDP 3389: Used by RDP." From the gateway:

# Read-only. From the RD Gateway: does the target resolve, and does it answer on 3389, by the name users type? (names are examples)
Resolve-DnsName -Name app01.contoso.com -Type A
Test-NetConnection -ComputerName app01.contoso.com -Port 3389

Step 9: Is UDP 3391 working, and why is RD Gateway Manager empty? (read-only)

Read the live connections on the gateway. This is the same kind of list RD Gateway Manager's Monitoring node shows (our reading):

# Read-only. On EACH RD Gateway: live connections and their transport (0 = RPC over HTTP, 1 = HTTP, 2 = UDP)
Get-CimInstance -Namespace root/CIMV2/TerminalServices -ClassName Win32_TSGatewayConnection |
  Select-Object UserName, ClientAddress, ConnectedResource, ConnectedPort, TransportProtocol, ConnectedTime, ConnectionDuration | Format-Table -AutoSize

UDP is the second transport: "UDP 3391: Used for RDP over UDP." Microsoft also notes that firewalls doing directional UDP analysis need the UDP rule set to Send Receive. Step 4 already showed whether transport 2 is enabled and listening.

Common false starts

Step 10: Change one link, then prove it

Only after Steps 1-9 agree on which link failed. Save the gateway's configuration first. The Export method "Returns the Remote Desktop Gateway (RD Gateway) server configuration as an XML string.", and its export bits add up to everything that matters here:

# Read-only, but store the file like a secret. On EACH RD Gateway: export CAPs (1), RADIUS servers (2), RAPs (4), resource groups (8), load-balancing servers (16), and server settings (32) = 63 (our form; path is an example)
$srv = Get-CimInstance -Namespace root/CIMV2/TerminalServices -ClassName Win32_TSGatewayServer
$out = Invoke-CimMethod -InputObject $srv -MethodName Export -Arguments @{ ExportType = [uint32]63 }
$out.XmlString | Out-File -FilePath C:\Temp\rdg-before.xml -Encoding utf8

Keep the Step 3 thumbprints, the Step 5 and 6 policy output, and the NPS timeout values with it (our habit).

The public name or the edge (Step 2). Fix the public DNS record, or the edge rule and NAT to the gateway: TCP 443 inbound, and UDP 3391 if you use UDP, with the UDP rule allowing both directions. If RD Web launches hand out the wrong name, set it on the RD Connection Broker (names are examples):

# Change. On the RD Connection Broker: the external gateway name clients get from the deployment
Set-RDDeploymentGatewayConfiguration -GatewayMode Custom -GatewayExternalFqdn rdg.contoso.com -LogonMethod Password -UseCachedCredentials $true -BypassLocal $false -ConnectionBroker rdcb.contoso.com

Custom means "Remote users get Gateway settings from other parameters to this cmdlet." Setting -BypassLocal $false also makes inside tests go through the gateway, which you want while you troubleshoot (our reading).

The certificate (Step 3). Get a certificate whose name matches the public FQDN, with Server Authentication, from a CA your outside clients trust (Microsoft recommends public for the gateway: "Because the RD Gateway role is supposed to be public, we recommend you use a publicly issued certificate."). In a deployment, assign it to the RD Gateway role on the RD Connection Broker with Microsoft's form:

# Change. On the RD Connection Broker: assign the PFX to the RD Gateway role (Microsoft's form with Role RDGateway; path is an example)
$password = Read-Host -AsSecureString -Prompt "Enter Password"
Set-RDCertificate -Role RDGateway -ImportPath 'C:\Certs\rdg-2026.pfx' -Password $password -ConnectionBroker rdcb.contoso.com

On Windows Server 2019 and later, -Thumbprint can point at a certificate that's already installed ("The parameter is available on Windows Server 2019 and later."). For a standalone gateway, use RD Gateway Manager: the server's Properties → SSL Certificate → Import a certificate into the RD Gateway → Browse and Import Certificate. Microsoft's high-availability steps add a note worth keeping: "You may need to manually restart the TSGateway service running on each RD Gateway server, either through Server Manager or Task Manager." Then rerun Step 3 on the gateway and on an outside client. If outside clients can't reach your CRL, publish it where they can before you move on.

The service or limits (Step 4). Start TSGateway and read why it stopped. Raise MaxConnections only if the limit, not an attack, was the cause (in RD Gateway Manager, server Properties).

The CAP (Step 5). Add the user to the group the CAP already names, or add the right group to that CAP in RD Gateway Manager (Policies → Connection Authorization Policies). Don't widen it to Domain Users or remove the computer group condition unless you've decided those devices should connect.

The RAP (Step 6). Add the name users actually type (short name and FQDN) to the RD Gateway-managed computer group, or the computer to the AD group the RAP uses, or the custom port to the RAP. Do it on every gateway in the farm.

NPS and MFA (Step 7). Raise the gateway's RADIUS timeout to 30-60 seconds on the Load Balancing tab, and set "Number of seconds between requests when server is identified as unavailable" to the same or more (Microsoft's step). If the NPS server doesn't know the gateway, add it as a RADIUS client with the same shared secret; if event 18 shows a secret mismatch, set a new long secret on both sides. If the NPS server isn't registered, Microsoft's step is Register server in Active Directory: "To function properly in this scenario, the NPS server needs to be registered in Active Directory." For users with both TOTP and push registered, Microsoft's fix is a registry value under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\AzureMfa on the NPS extension server, then an NPS service restart:

# Change. On the NPS EXTENSION server (not the gateway): fall back to Approve/Deny for RD Gateway (Microsoft's value, string FALSE)
New-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\AzureMfa' -Name OVERRIDE_NUMBER_MATCHING_WITH_OTP -Value 'FALSE' -PropertyType String -Force
Restart-Service -Name IAS

Microsoft's wording: "set OVERRIDE_NUMBER_MATCHING_WITH_OTP = FALSE to fall back to Approve/Deny push notifications with Authenticator." If the extension sits on the gateway, install it on a separate NPS server and point the gateway's central CAP store there, following Microsoft's RD Gateway integration steps, instead of patching it in place.

The target (Step 8). Fix the firewall from the gateway to the target, the target's RDP listener, or the user's rights on the target. Nothing on the gateway changes.

UDP (Step 9). Allow UDP 3391 both ways at the edge and on any load balancer. If you can't yet, sessions keep working over HTTP; leave UDP enabled on the gateway so it starts working when the edge is fixed (our reading).

Then prove it. From an outside client, as the user who failed: the connection succeeds, the gateway log shows 200, 300, and 302 for that user at that time, Win32_TSGatewayConnection lists the session (with TransportProtocol 2 if UDP should work), Test-Certificate -DNSName returns True for the public name, and NPS logs 6272 when MFA is in the path (our reading). Then have the helpdesk retest the original users.

When a change is justified

FAQ

Why is my RD Gateway not working externally when it works inside? Usually the public name points somewhere else, the edge firewall or NAT doesn't pass TCP 443 to the gateway, or inside clients bypass the gateway so the inside test proves nothing. Microsoft requires "inbound TCP connections to port 443 and inbound UDP connections to port 3391" on the public address. Test Resolve-DnsName and Test-NetConnection -Port 443 from an outside network (Step 2).

What does "Remote Desktop Gateway server is temporarily unavailable" mean? The client didn't get a usable answer from the gateway: the name, 443, the TLS handshake, the service, or (with MFA) a RADIUS timeout. For Windows 365 Boot, Microsoft says "This issue might be a transient issue because of network congestion.", but for your own gateway, read Steps 2-4 and 7 before you wait it out.

What ports does RD Gateway use? From clients: TCP 443 for HTTPS and UDP 3391 for "RDP over UDP". From the gateway inward: 3389 to the targets, DNS, Kerberos and LDAP to AD, CRL ports, and "UDP 1812 and 1813: Required when Network Policy Server (NPS) is used."

How do I fix an expired RD Gateway certificate? Get a new certificate with the public name and Server Authentication, assign it to the RD Gateway role (Set-RDCertificate -Role RDGateway, or RD Gateway Manager's SSL Certificate tab), restart TSGateway if needed, and confirm CertHash and netsh http show sslcert show the new thumbprint (Steps 3 and 10).

Why isn't RD Gateway MFA with the NPS extension prompting or working? Most often the gateway's RADIUS timeout is still 3 seconds, the user only has SMS or a code (RD Gateway supports phone call and Authenticator Approve/Deny), the user has both TOTP and push registered, or the extension is installed on the gateway. Read the AzureMfa log and Security events 6272/6273 on the NPS server (Step 7).

Related posts

• NPS Event 6273 Reason Code 16 or 22 (when the central NPS denies the gateway's RADIUS request)
• Revocation Server Was Offline 0x80092013 (when outside clients can't check the gateway certificate's CRL)
• Microsoft Authenticator Not Receiving Notifications (when the MFA push is sent but never shows up)
• Remote Desktop Licensing Mode Is Not Configured (when the session host refuses users after the gateway lets them through)
• Certificate Autoenrollment Not Working (when an internal CA should be renewing the gateway or NPS certificates)

Free checklist

Reinstalling the gateway and opening every policy before anyone read events 201 and 301 is a change made everywhere at once. So is a Conditional Access policy enforced before anyone read the report-only results. The free Conditional Access pre-enforcement checklist keeps break-glass accounts, pilot groups, and report-only review on one page:

👉 https://cashflow4375.gumroad.com/l/ujkqxb?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=rd-gateway-not-working

If you want the rest of the Intune runway written down for the team (enrollment hygiene, compliance phasing, break/fix cards and read-only PowerShell snapshots), the Intune & M365 Admin Starter Pack pairs with this triage: https://cashflow4375.gumroad.com/l/joonf?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=rd-gateway-not-working

Admin Pack Studio; not affiliated with Microsoft. Steps 1-9 favor read-only checks: Resolve-DnsName, Test-NetConnection, Get-RDDeploymentGatewayConfiguration, netsh http show sslcert, Get-ChildItem Cert:, Test-Certificate, Get-RDCertificate, certutil -urlfetch -verify, Get-Service, Get-NetTCPConnection, Get-NetUDPEndpoint, Get-CimInstance and read-only Invoke-CimMethod calls on the RD Gateway WMI classes, Get-WinEvent, and auditpol /get. Exporting the gateway configuration creates a sensitive file. Changing DNS, edge rules, certificates, gateway settings, CAPs, RAPs, resource groups, RADIUS timeouts or secrets, NPS registration, or the NPS extension registry, and restarting services, are deliberate changes. Change one link at a time, test from an outside client and from the gateway, and check current Microsoft Learn for version-specific behavior.

Not yet. "DNS not resolving external names" is almost never the client. Your domain controllers answer internal names from their own zones; external names depend on a chain the client never sees: recursion allowed on the server, a forwarder that answers inside the timeout, root hints the server can actually reach as a fallback, a conditional forwarder that exists on the DC the client asked, and a network that passes DNS over UDP and TCP 53. Microsoft describes the end of that chain plainly: "If the server configuration disables recursion, the server normally returns REFUSED for a query that requires recursion. If recursive processing can't complete, the server returns SERVFAIL." And the part most fixes miss: "An unresponsive forwarder doesn't guarantee that the server reaches root hint fallback." Pointing PCs at a public resolver breaks Active Directory lookups, five more forwarders can't all be reached in the default timeout, and a service restart clears the cache and the forwarder state you need to read. Work through it as a chain: which client, which DNS server, which name, which error → is the client asking the server you think? → does the server resolve it itself, and is recursion allowed? → which path does the server pick for this name? → do the forwarders answer, inside the timeout? → can root hints fallback actually happen? → does the conditional forwarder exist on this DC, and do its targets answer? → is a cached answer hiding the fix? → is the network dropping large or TCP DNS? Resolve-DnsName, Get-DnsServerForwarder, Get-DnsServerRecursion, Get-DnsServerZone, Get-DnsServerRootHint, Test-DnsServer, Show-DnsServerCache, and nslookup are the sources for that chain. Change the forwarders first and you'll never know which one was dead.

If external lookups fail from domain controllers at boot and Netlogon logs 5719, Netlogon Event ID 5719 covers the DC locator side. If records go missing from your internal zones, DNS Scavenging Not Working covers aging. This post covers "a Windows DNS server (usually a domain controller) resolves internal names but not internet names, resolves them slowly or only on retry, or can't resolve one partner or cloud namespace through a conditional forwarder, and you need a read-first path before you change client DNS settings, forwarders, root hints, timeouts, or recursion."

TL;DR

• Prove the server, not the client. Microsoft: "Check whether the resolution fails from the DNS server itself. If so, you can rule out any issue with the client or the network between the client and the DNS server." (Step 3).
• The server picks one path per name. Hosted zone or cache first, then a matching conditional forwarder, a delegation, the server-level forwarders, and root hints last (Step 4).
• Dead forwarders cost seconds each. Microsoft: "with default settings, a Windows DNS server will be able to query at most three forwarders." (Step 5).
• Root hints are a fallback only if the clock and the firewall allow it. Microsoft: "If the overall recursion timeout expires first, the query fails without reaching the root hints." (Step 6).
• Conditional forwarders are zones, and zones have a replication scope. One stored in the registry of DC1 doesn't exist on DC2 (Step 7).
• A negative answer can outlive the fix. Clear the cache once, after the fix, not as the fix (Step 8).

Where in the chain did it break?

Step 1: Capture the symptom (read-only)

Write these on the ticket first:

Then branch on what you see:

Step 2: Is the client asking the server you think? (read-only)

Microsoft's client checklist starts with the configuration: run ipconfig /all and "Check the DNS servers that are listed in the output, and verify that the IP addresses listed are correct." On the failing client:

# Read-only. On the failing client: which DNS servers does each adapter use?
Get-DnsClientServerAddress -AddressFamily IPv4 | Where-Object { $_.ServerAddresses } |
  Select-Object InterfaceAlias, ServerAddresses | Format-Table -AutoSize

# Ask each internal DNS server directly, DNS protocol only, hosts file skipped (addresses are examples)
Resolve-DnsName -Name www.microsoft.com -Server 10.0.0.10 -DnsOnly -NoHostsFile
Resolve-DnsName -Name www.microsoft.com -Server 10.20.0.10 -DnsOnly -NoHostsFile

# What the client has cached for the name right now
Get-DnsClientCache -Entry www.microsoft.com -ErrorAction SilentlyContinue

-Server is documented as "Specifies the IP addresses or host names of the DNS servers to be queried." and without it "the interface DNS servers are queried". So a lookup that works with -Server and fails without it says the client is asking someone else.

Why nslookup and the browser disagree: Microsoft notes that "Because nslookup doesn't use the client's DNS cache, name resolution will use the client's configured DNS server." Ping and browsers go through the DNS Client service, which checks its cache first, and that cache is preloaded from the hosts file: "If a Hosts file is configured locally, any host name-to-address mappings from that file are loaded into the cache when the DNS Client service is started."

Step 3: Does the DNS server resolve it itself, and is recursion allowed? (read-only)

Microsoft's server checklist runs the same lookup against the server's own address: nslookup <name> <IP address of the DNS server>, and "If you get a failure or time-out response, see Checking for recursion problems." On the DNS server:

# Read-only. On the DNS server: resolve through itself, on the address clients use (example address)
Resolve-DnsName -Name www.microsoft.com -Server 10.0.0.10 -DnsOnly -NoHostsFile
Get-Service -Name DNS

# Recursion: the server-wide setting, recursion scopes, and any query resolution policies
Get-DnsServerRecursion
Get-DnsServerRecursionScope
Get-DnsServerQueryResolutionPolicy | Format-List *

# Which addresses the service listens on
Get-DnsServerSetting -All | Select-Object ListeningIPAddress, AllIPAddress

# Read-only. On the DNS server: recent errors and warnings in the DNS Server log
Get-WinEvent -FilterHashtable @{ LogName = 'DNS Server'; Level = 2, 3; StartTime = (Get-Date).AddDays(-2) } -MaxEvents 30 -ErrorAction SilentlyContinue |
  Select-Object TimeCreated, Id, Message | Format-List

Microsoft's recursion cmdlet defines the term: "Recursion occurs when a DNS server queries other DNS servers on behalf of a requesting client, and then sends the answer back to the client." Recursion scopes are the newer layer on top: "The legacy recursion setting and list of forwarders are referred to as the default recursion scope." That default scope is named with a single dot, and a policy can send some queries to a different scope.

Step 4: Which path does the server pick for this name? (read-only)

This is the step most fixes skip. Microsoft's current forwarding page lists the order a Windows DNS server follows for a recursive query (with the default ForwardDelegations off): local zones and cache, then "Use a matching conditional forwarder.", then "Use an applicable delegation.", then "Use server-level forwarders.", then "Use root hints when the configuration permits." A zone the server hosts wins outright: "If a hosted authoritative zone covers the query, that zone determines the result."

# Read-only. On the DNS server: which hosted zone, conditional forwarder, or root zone matches the failing name? (our form)
$name = 'portal.fabrikam.com'
Get-DnsServerZone | Where-Object { $name -eq $_.ZoneName -or $name -like "*.$($_.ZoneName)" -or $_.ZoneName -eq '.' } |
  Select-Object ZoneName, ZoneType, IsDsIntegrated, IsAutoCreated | Format-Table -AutoSize
Get-DnsServerSetting -All | Select-Object ForwardDelegations

Microsoft's forwarder article starts the same way: Get-DnsServerZone -Name <domainname> to check for a conditional forwarder, then Get-DnsServerForwarder, then Get-DnsServerRootHint. "If none of them are configured, the name resolution is expected to fail."

Step 5: Do the forwarders answer, inside the timeout? (read-only)

Read the forwarder list and the two timers that decide how many of them the server ever tries:

# Read-only. On the DNS server: forwarder list, per-forwarder timeout, root-hint fallback, dynamic reordering
Get-DnsServerForwarder
# Overall recursion budget (Timeout) and whether recursion is on
Get-DnsServerRecursion

Then ask each forwarder directly from the DNS server, and time it. This is the test the server itself is making:

# Read-only. From the DNS server: does each forwarder answer, and how fast? (our form)
foreach ($fwd in (Get-DnsServerForwarder).IPAddress) {
  $ip = "$fwd"
  $sw = [System.Diagnostics.Stopwatch]::StartNew()
  $ok = [bool](Resolve-DnsName -Name www.microsoft.com -Server $ip -DnsOnly -QuickTimeout -ErrorAction SilentlyContinue)
  $sw.Stop()
  [pscustomobject]@{ Forwarder = $ip; Answered = $ok; Milliseconds = $sw.ElapsedMilliseconds }
}
# Microsoft's built-in check of the configured forwarders (the address is the DNS server's own)
Test-DnsServer -IPAddress 10.0.0.10 -Context Forwarder

How the timers interact, from Microsoft's timeout article (KB 2834250 in the old numbering): the server queries the first forwarder at 0 seconds, the second "after 3.5 seconds", the third "after 3.5 + 4 = 7.5 seconds", and then "At the eighth second, RecursionTimeout expires". The client gets its answer late: "We'll send the Server Failure response then after 11.5 seconds." Two Microsoft pages disagree on the default per-forwarder timeout (the timeout article says 3 seconds, the Set-DnsServerForwarder reference says "The default value is 5."), so read the value Get-DnsServerForwarder returns rather than trusting either.

Test-NetConnection -Port 53 proves nothing here: its -Port parameter "Specifies the TCP port number on the remote computer.", and most DNS queries go over UDP (Step 9).

Step 6: Can root hints fallback actually happen? (read-only)

Root hints are the server's way to resolve without forwarders: "Root hints are resource records that identify DNS servers authoritative for the root of a DNS namespace." With forwarders configured, they're only a fallback, and Microsoft lists four conditions for it, including "The UseRootHint setting is $true" and "The server has usable root hint data and network access to the required DNS servers."

# Read-only. On the DNS server: are root hints present, and is fallback allowed?
Get-DnsServerRootHint
(Get-DnsServerForwarder).UseRootHint
# Microsoft's built-in root hints check (the address is the DNS server's own)
Test-DnsServer -IPAddress 10.0.0.10 -Context RootHints
# Can this server reach a root server directly? (198.41.0.4 is a.root-servers.net; take any address from Get-DnsServerRootHint)
Resolve-DnsName -Name '.' -Type NS -Server 198.41.0.4 -DnsOnly -NoRecursion

Microsoft's own root test uses interactive nslookup against the server being examined:

:: Read-only. On the DNS server (Microsoft's form; the address is the DNS server's own)
nslookup
server 10.0.0.10
set q=NS

Per that article, "If the resolver returns the IP address of a root server, you probably have a broken delegation" further down, and a timeout means checking the root hints, the network, or a firewall: "It's also possible that the recursive time-out default is too short."

Microsoft's design note is worth reading before you change anything: keep fallback "when direct iterative resolution is acceptable" and "Disable it when policy requires all unresolved queries to remain within the forwarding boundary, and enforce the boundary with network controls."

Step 7: Does the conditional forwarder exist on this DC, and do its targets answer? (read-only)

A conditional forwarder is stored as a zone: "DNS Server stores conditional forwarders as forwarder zones." That means it has a storage location and a replication scope, and a DC that doesn't have the zone sends the partner name down the general forwarder path instead. Read it on every DC:

# Read-only. Every DC: is the fabrikam.com conditional forwarder there, and how is it set? (our form)
foreach ($dc in (Get-ADDomainController -Filter *).HostName) {
  Get-DnsServerZone -ComputerName $dc -Name fabrikam.com -ErrorAction SilentlyContinue |
    Select-Object @{ n = 'DC'; e = { $dc } }, ZoneName, ZoneType, IsDsIntegrated, ReplicationScope, MasterServers, ForwarderTimeout, UseRecursion
}

:: Read-only. One DC at a time: list only conditional forwarder zones (dnscmd's /forwarder filter)
dnscmd DC2 /enumzones /forwarder

Then ask each target server directly, from the DC that fails:

# Read-only. From the failing DC: does each conditional forwarder target answer? (addresses are examples)
Resolve-DnsName -Name portal.fabrikam.com -Server 192.0.2.10 -DnsOnly -QuickTimeout
Resolve-DnsName -Name portal.fabrikam.com -Server 192.0.2.11 -DnsOnly -QuickTimeout

Azure private endpoints are a common version of this. Microsoft: "DNS queries for private endpoints must originate from the virtual network that is linked to the private DNS zone." So an on-premises conditional forwarder that points straight at Azure's 168.63.129.16 can't work; it points at a DNS forwarder or Private Resolver inside the linked virtual network, and "The conditional forwarding must be made to the recommended public DNS zone forwarder." (for example the public service zone rather than the privatelink zone).

Step 8: Is a cached answer hiding the fix? (read-only)

Both the DNS server and the client cache negative answers. Read before you clear:

# Read-only. On the DNS server: what's cached for the failing name, and the cache limits
Show-DnsServerCache | Where-Object { $_.HostName -like '*fabrikam*' }
Get-DnsServerCache

# Read-only. On the client: what the DNS Client service has cached for the name
Get-DnsClientCache -Entry portal.fabrikam.com -ErrorAction SilentlyContinue

Get-DnsServerCache returns the limits, including "MaxTTL, MaxNegativeTTL"; Microsoft's example shows a MaxNegativeTTL of 15 minutes, so a negative answer cached before the fix can linger that long (our reading). On the client, Microsoft's check is ipconfig /displaydns: a "Name does not exist" entry means a negative response "was returned from a DNS server and was cached on the client."

Step 9: Is the network dropping large or TCP DNS? (read-only, then one optional logging change)

DNS uses both transports. Microsoft's port page: "Messages are sent over UDP and DNS servers bind to UDP port 53." When an answer is too big, "The DNS client (message sender) can then choose to reissue the request to the DNS server using TCP (over TCP port 53)." And Windows DNS advertises bigger UDP answers by default through EDNS0, which some firewalls drop: "EDNS0 allows larger User Datagram Protocol (UDP) packet sizes. However, some firewall programs may not allow UDP packets that are larger than 512 bytes."

# Read-only. From the DNS server: the same name over UDP and over TCP to one forwarder (address is an example)
Resolve-DnsName -Name portal.fabrikam.com -Server 203.0.113.53 -DnsOnly
Resolve-DnsName -Name portal.fabrikam.com -Server 203.0.113.53 -DnsOnly -TcpOnly
# The server's EDNS settings
Get-DnsServerEDns

-TcpOnly is documented as "Uses only TCP for this query." Microsoft's symptom for the EDNS case: "After you deploy a Windows-based DNS server, DNS queries to some domains may not be resolved successfully."

To see what the server itself sent and what timed out, the DNS Server analytic log records outbound recursive queries and timeouts (event 260 is RECURSE_QUERY_OUT, 262 is RECURSE_QUERY_TIMEOUT, 258 is RESPONSE_FAILURE). It's off by default; Microsoft's steps turn it on in Event Viewer (Applications and Services Logs → Microsoft → Windows → DNS-Server → View → Show Analytic and Debug Logs → Analytical → Properties → Enable logging). Microsoft's cost note: "There's no apparent performance impact for query rates of 50,000 QPS and lower." Reproduce once, then read it:

# Read-only. On the DNS server: recursive timeouts and response failures from the analytic log (our form; -Oldest is required for analytic logs)
Get-WinEvent -LogName 'Microsoft-Windows-DNSServer/Analytical' -Oldest -ErrorAction SilentlyContinue |
  Where-Object { $_.Id -in 258, 262 } |
  Select-Object -Property TimeCreated, Id, Message -Last 40 | Format-List

-Oldest "is required to get events from .etl and .evt files and from debug and analytic logs." If the read errors while the log is enabled, disable the log first and read it then (our reading of Microsoft's note that a circular log can't be viewed while it's being written).

Common false starts

Step 10: Change one link, then prove it

Only after Steps 1-9 agree on which link failed. Write down the current state first: Get-DnsServerForwarder, Get-DnsServerRecursion, Get-DnsServerRootHint, the Step 7 conditional forwarder table, and Get-DnsServerQueryResolutionPolicy from every DNS server (our habit). Set-DnsServerForwarder replaces the whole list: "This cmdlet overwrites existing server level forwarders."

The client asks the wrong server (Step 2). Fix the DHCP scope's DNS server option, or the static setting. Microsoft's command for a static client (interface name and addresses are examples):

# Change. On ONE client: point the adapter at the internal DNS servers (Microsoft's form, two addresses)
Set-DnsClientServerAddress -InterfaceAlias 'Ethernet' -ServerAddresses ('10.20.0.10', '10.0.0.10')

Recursion off, or a policy refusing (Step 3). Turn recursion on only on servers that serve internal clients; Microsoft warns "Allow recursive queries only from trusted clients and networks, and don't expose recursive DNS service to untrusted networks." If selective recursion is by design, fix the policy's subnet or interface criteria instead.

# Change. On an internal-only DNS server: allow recursion
Set-DnsServerRecursion -Enable $true -PassThru

A root zone or an internal zone answering (Step 4). Delete the . zone in DNS Manager under Forward Lookup Zones, per Microsoft's DC guidance. For an external name under your internal zone name, add the record to the internal zone; don't create a conditional forwarder for a name you host.

A dead, slow, or refusing forwarder (Step 5). Replace the list with two or three forwarders you've tested from the DNS server, preferred one first:

# Change. On the DNS server: replace the whole forwarder list (it overwrites; addresses are examples)
Set-DnsServerForwarder -IPAddress 203.0.113.53, 198.51.100.53 -PassThru
# Or remove just the dead one
Remove-DnsServerForwarder -IPAddress 192.0.2.53 -PassThru

On Windows Server 2022 or later, if you want the older behavior of cycling through every forwarder after they've all failed, Microsoft's switch is "To make the server cycle through all configured forwarders as in earlier versions, disable dynamic reordering with Set-DnsServerForwarder -EnableReordering $false." Change timers last, and keep the recursion timeout above the forwarder timeout (Set-DnsServerRecursion -Timeout, Set-DnsServerForwarder -Timeout; both cap at 15 seconds per their references).

Root hints (Step 6). If they're missing, Microsoft's cmdlet copies them from another DNS server: "The root hints that you copy will not overwrite any existing root hints." If policy keeps all lookups inside the forwarders, make that explicit rather than leaving a fallback that can't work:

# Change. On the DNS server: copy root hints from a healthy DNS server (example name), or turn fallback off on purpose
Import-DnsServerRootHint -NameServer dc1.contoso.com -PassThru
Set-DnsServerForwarder -UseRootHint $false -PassThru

The conditional forwarder (Step 7). Store it in AD with a scope that covers every DC that needs it, and list only targets you've tested:

# Change. On one DC: create the forwarder in AD, replicated to every DNS server on a DC in the forest (addresses are examples)
Add-DnsServerConditionalForwarderZone -Name fabrikam.com -ReplicationScope Forest -MasterServers 192.0.2.10, 192.0.2.11 -PassThru
# Or fix an existing one: targets and per-zone timeout
Set-DnsServerConditionalForwarderZone -Name fabrikam.com -MasterServers 192.0.2.10, 192.0.2.11 -ForwarderTimeout 5 -PassThru

If a registry-stored copy already exists on one DC, remove that copy first (export its settings) so the AD-integrated one doesn't collide with it (our reading). Microsoft's forest scope means "Replicate the conditional forwarder to all DNS servers that run on domain controllers in the forest."

Cache (Step 8). Once, after the fix, on the server and then the client:

# Change (low risk). On the DNS server, then on the client that cached the failure
Clear-DnsServerCache -Force
Clear-DnsClientCache

Clear-DnsClientCache is documented as "equivalent to running ipconfig /flushdns."

Large UDP or TCP 53 dropped (Step 9). Fix the firewall. Microsoft's workaround, if you can't yet, turns off EDNS probes on the DNS server (a registry change; reverse it with 1 after the firewall is fixed):

:: Change. On the DNS server: Microsoft's EDNS0 workaround (zero, not the letter O)
dnscmd /config /enableednsprobes 0

Turn the analytic log back off when you're done (our habit).

Then prove it. From a failing client and from each DNS server: Resolve-DnsName -Name www.microsoft.com -Server <each DNS server> -DnsOnly answers in well under a second, the partner name resolves from the Dallas DC as well as head office, Test-DnsServer -Context Forwarder returns Success, and the analytic log shows no new event 262 for your forwarders (our reading). Then let the helpdesk retest the original sites.

When a change is justified

FAQ

Why is my Windows DNS server not resolving external names? Most often a forwarder is dead or refusing and the recursion timer runs out before root hints are tried, recursion is disabled or limited by a policy, or an internal zone answers for the name. Microsoft's first split: "Check whether the resolution fails from the DNS server itself." Then read Get-DnsServerForwarder and test each forwarder directly.

Should I use DNS forwarders or root hints? Either works; forwarders centralize lookups, root hints resolve without them. Microsoft keeps root hints as a fallback by default and says to turn it off "when policy requires all unresolved queries to remain within the forwarding boundary". Don't leave fallback on if the firewall blocks it.

Why is my DNS conditional forwarder not working? Usually it doesn't exist on the DC the client asked (registry-stored or the wrong replication scope), its targets don't answer from that site, or a target refuses or lacks the record. Run Step 7 on every DC, then query each target directly.

How many DNS forwarders should I configure? Two or three healthy ones. With default timers, Microsoft says the server can reach "at most three forwarders", and every dead one in front adds seconds to each lookup.

Why does nslookup work but ping doesn't? nslookup asks the DNS server directly; ping goes through the DNS Client service's cache, which also holds hosts file entries and earlier negative answers. Microsoft: "Because nslookup doesn't use the client's DNS cache, name resolution will use the client's configured DNS server." Check the client cache and hosts file (Steps 2 and 8).

Related posts

• Netlogon Event ID 5719 (when DCs can't find a domain controller through DNS at boot)
• DNS Scavenging Not Working (when internal records go missing instead)
• Active Directory Replication Error 1722 (DC DNS client settings and the ports between DCs)
• Windows Server DHCP Not Assigning IP Addresses (when the scope hands clients the wrong DNS servers)

Free checklist

Adding five forwarders and restarting DNS on every DC before anyone tested a single forwarder is a change made everywhere at once. So is a Conditional Access policy enforced before anyone read the report-only results. The free Conditional Access pre-enforcement checklist keeps break-glass accounts, pilot groups, and report-only review on one page:

👉 https://cashflow4375.gumroad.com/l/ujkqxb?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=windows-dns-server-not-resolving-external-names

If you want the rest of the Intune runway written down for the team (enrollment hygiene, compliance phasing, break/fix cards and read-only PowerShell snapshots), the Intune & M365 Admin Starter Pack pairs with this triage: https://cashflow4375.gumroad.com/l/joonf?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=windows-dns-server-not-resolving-external-names

Admin Pack Studio; not affiliated with Microsoft. Steps 1-9 favor read-only checks: Get-DnsClientServerAddress, Get-DnsClientCache, Resolve-DnsName, Get-Service, Get-DnsServerRecursion, Get-DnsServerRecursionScope, Get-DnsServerQueryResolutionPolicy, Get-DnsServerSetting, Get-DnsServerZone, Get-DnsServerForwarder, Get-DnsServerRootHint, Test-DnsServer, Show-DnsServerCache, Get-DnsServerCache, Get-DnsServerEDns, Get-WinEvent, dnscmd /enumzones, and nslookup. Enabling the analytic log, changing client DNS settings, recursion, forwarders, timeouts, reordering, root hints, conditional forwarders, or EDNS, and clearing caches are deliberate changes. Change one link at a time, test from the failing client and from each DNS server, and check current Microsoft Learn for version-specific behavior.

Not yet. "LDAPS not working" and "LDAP signing required" are two different breaks that show up in the same week. LDAPS on port 636 fails when the domain controller has no certificate it can use, serves a different certificate than you think, or serves one the client doesn't trust or can't match to the name it used. Signing and channel binding failures are binds the DC now rejects on purpose. Microsoft: "There's no user interface for configuring LDAPS. Installing a valid certificate on a domain controller permits the LDAP service to listen for, and automatically accept, SSL connections for both LDAP and global catalog traffic." On signing, Microsoft's own warning: "After you make this configuration change, clients that rely on unsigned SASL (Negotiate, Kerberos, NTLM, or Digest) LDAP binds or on LDAP simple binds over a non-SSL/TLS connection stop working." Turning signing off domain-wide, deleting certificates, or setting channel binding to Never each removes evidence you need, and the first and last reopen the attacks those settings exist to stop. Work through it as a chain: which client, which DC, which port, which bind → does the DC listen on 636 and does the client reach it? → does the DC have a certificate it can use (Event 1220)? → which certificate does it actually serve? → does the client trust that certificate? → does the name the client used match it? → who is binding unsigned (Events 2887 and 2889)? → which signing policy is actually in effect on each DC? → is channel binding rejecting a TLS bind? Test-NetConnection, Get-ChildItem Cert:\LocalMachine\My, certutil, Get-WinEvent, reg query, gpresult, and ldp.exe are the sources for that chain. Turn signing off everywhere and you'll find out later which client was sending passwords in clear text.

If the DC never got a certificate because enrollment fails, Certificate Autoenrollment Not Working covers the GPO, template, and CA side. If certutil on the client fails the revocation check, Revocation Server Was Offline 0x80092013 covers CRLs and CDPs. This post covers "LDAPS on port 636 (or 3269) to a domain controller fails, an application or appliance gets 'Strong Authentication Required' after LDAP signing changes or a new Windows Server 2025 DC, or Directory Service Events 1220, 2886, 2887, 2889, 3039, or 3075 appear, and you need a read-first path before you replace certificates, restart DCs, change signing or channel binding policy, or edit NTDS registry values."

TL;DR

• LDAPS is a certificate problem first. Microsoft's requirements include: "The Enhanced Key Usage extension includes the Server Authentication (1.3.6.1.5.5.7.3.1) object identifier (also known as OID)." and the DC's FQDN in the subject CN or a SAN DNS entry (Step 3).
• The DC may not serve the certificate you're looking at. Microsoft: "If there is a certificate in the NT Directory Services (NTDS) store, DC use the certificate in the NTDS store instead." (Step 4).
• The client has to trust it. Microsoft: "The certificate was issued by a CA that the domain controller and the LDAPS clients trust." (Step 5).
• Event 2889 names the unsigned clients. Microsoft: "The log entry displays the IP address of the client and the identity that the client tried to use to authenticate." It's off until you raise a diagnostic level (Step 7).
• Windows Server 2025 changes the default. Microsoft: "Windows Server 2025 or later requires LDAP signing by default for new Active Directory deployments." (Step 8).
• Channel binding only covers TLS plus SASL. Microsoft: "LDAP channel binding applies through the Domain controller: LDAP server channel binding token requirements policy only to TLS-secured sessions that use Simple Authentication and Security Layer (SASL) authentication." (Step 9).

Where in the chain did it break?

Step 1: Capture the symptom (read-only)

Write these on the ticket first:

Then branch on what you see:

Step 2: Does the DC listen on 636, and does the client reach it? (read-only)

Start at the network, because nothing else matters if the handshake never starts. From the failing client, or a Windows box on the same network segment:

# Read-only. From the client side: can it open TCP 636 (and 3269 for the global catalog) on the DC it uses?
Test-NetConnection -ComputerName DC2.contoso.com -Port 636
Test-NetConnection -ComputerName DC2.contoso.com -Port 3269

On the DC itself:

# Read-only. On the DC: is anything listening on 636 and 3269?
Get-NetTCPConnection -LocalPort 636, 3269 -State Listen -ErrorAction SilentlyContinue |
  Select-Object LocalAddress, LocalPort, State, OwningProcess | Format-Table -AutoSize

A port that answers isn't the same as LDAPS working. The TCP test only proves the socket opens; a DC with no certificate still accepts the connection and then fails the TLS handshake (our reading of Microsoft's Event 1220 text and the Schannel 36886 description).

Step 3: Does the DC have a certificate it can use? (read-only)

Microsoft's requirements for an LDAPS certificate, in short: it sits in the Local Computer's Personal store, "A private key that matches the certificate is present in the Local Computer's store and is correctly associated with the certificate," the private key doesn't have strong protection, the EKU includes Server Authentication, the DC's Active Directory FQDN is the subject CN or a SAN DNS entry, and it chains to a CA both the DC and the clients trust.

First, the DC's own log. Microsoft: "Event ID 1220 is logged on a domain controller when client computers attempt to make an LDAP-over-SSL connection to the directory when SSL connections are not enabled on the directory." Schannel logs its side in the System log; Microsoft's Schannel reference describes Event 36886 as occurring "when a server attempt to make an SSL connection but no server certificate is found."

# Read-only. On the DC: Event 1220 (Directory Service) and Schannel 36886 (System), last 7 days
$since = (Get-Date).AddDays(-7)
Get-WinEvent -FilterHashtable @{ LogName = 'Directory Service'; Id = 1220; StartTime = $since } -ErrorAction SilentlyContinue |
  Select-Object TimeCreated, Id, Message | Format-List
Get-WinEvent -FilterHashtable @{ LogName = 'System'; ProviderName = 'Schannel'; Id = 36886; StartTime = $since } -ErrorAction SilentlyContinue |
  Select-Object TimeCreated, Id, Message | Format-List

Then the certificates in the computer's Personal store that could serve LDAPS:

# Read-only. On the DC: Server Authentication certificates in the computer's Personal (MY) store
Get-ChildItem -Path Cert:\LocalMachine\My -SSLServerAuthentication |
  Select-Object Subject, Issuer, NotBefore, NotAfter, HasPrivateKey, Thumbprint, DnsNameList, EnhancedKeyUsageList |
  Format-List

And Microsoft's validity check from the Directory Services team, which also reports the private key:

:: Read-only. On the DC: verify each certificate in the computer's Personal store (Microsoft's AskDS form)
certutil -verifystore MY

Per that post, a good certificate shows "Encryption test passed", the Server Authentication policy under "Verified Application Policies", and "Certificate is valid" at the end. The missing-key case reads "No key provider information" and "Missing stored keyset".

If you see no 1220 and no 36886 but LDAPS still fails, the DC probably has a certificate; the question moves to which one it serves and whether the client accepts it (our reading).

Step 4: Which certificate does the DC actually serve? (read-only)

This is where renewals go wrong. Three Microsoft statements together explain it. The NTDS store wins: "AD DS preferentially looks for certificates in this store over the Local Machine's store." In the computer store, the pick isn't predictable: "Schannel, the Microsoft SSL provider, selects the first valid certificate that it finds in the local computer store." And on newer versions: "Finally, if a Windows Server 2008 or a later version domain controller finds multiple certificates in its store, it will random chose one of these certificates." Microsoft's LDAPS troubleshooting article adds: "A conflict with a certification authority (CA) certificate may occur if the CA is installed on a domain controller that you are trying to access through LDAPS."

So read what the DC presents on the wire, not what's in the store. This asks the DC for its certificate the same way an LDAPS client would and prints it, without trusting it:

# Read-only. From the client side: which certificate does DC2 present on 636? (our form, using .NET SslStream)
# The validation callback returns $true so you can see an untrusted or wrong certificate instead of an exception
$dc  = 'DC2.contoso.com'
$tcp = New-Object System.Net.Sockets.TcpClient($dc, 636)
$ssl = New-Object System.Net.Security.SslStream($tcp.GetStream(), $false, { $true })
try {
  $ssl.AuthenticateAsClient($dc)
  $served = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($ssl.RemoteCertificate)
  $served | Select-Object Subject, Issuer, NotBefore, NotAfter, Thumbprint, DnsNameList | Format-List
  # Save it for Step 5 (public certificate only)
  [System.IO.File]::WriteAllBytes("$env:TEMP\dc2-ldaps.cer", $served.Export('Cert'))
} finally { $ssl.Dispose(); $tcp.Dispose() }

Change the port to 3269 to read the global catalog side. If AuthenticateAsClient throws before any certificate comes back, the DC had nothing to offer, which matches Event 1220 (our reading).

Compare the served Thumbprint with Step 3's list. Then look in the NTDS store, which the computer-store listing doesn't show. Microsoft's route is the Certificates snap-in: Service account → Local computer → Active Directory Domain Services, then NTDS\Personal. A command-line read (our form; the -service option is documented as accessing "a machine service store"):

:: Read-only. On the DC: certificates in the AD DS service's own Personal store (our form; use the MMC route if it errors)
certutil -service -store NTDS\My

Step 5: Does the client trust that certificate? (read-only)

Microsoft's LDAPS troubleshooting article checks the chain from the client, with the certificate exported from the server: certutil -v -urlfetch -verify serverssl.cer > output.txt, then "Open the Output.txt file, and then search for errors." The file saved in Step 4 is the certificate the DC actually served, so use that one:

:: Read-only. On the CLIENT (or a Windows box with the same trust store): build the chain and fetch AIA and CDP URLs
certutil -v -urlfetch -verify %TEMP%\dc2-ldaps.cer > %TEMP%\dc2-ldaps-verify.txt
findstr /i "error expired revoked untrusted incomplete offline" %TEMP%\dc2-ldaps-verify.txt

-urlfetch is documented as "Retrieve and verify AIA Certs and CDP CRLs." Microsoft's AskDS post lists what the bottom of the output usually says when it fails:

Non-Windows clients (appliances, Java, Linux) don't use the Windows trust store. A clean certutil run on a Windows box proves the chain is buildable; the appliance still needs your root and intermediate imported into its own store (our reading). Microsoft's own requirement covers both sides: "Trust is established by configuring the clients and the server to trust the root CA to which the issuing CA chains."

The same server-certificate pattern (right EKU, private key present, chain the client trusts) is what breaks PEAP on NPS; NPS Event 6273 Reason Code 16 or 22 walks that version.

Step 6: Does the name the client used match the certificate? (read-only)

Microsoft's name rule: "The Active Directory fully qualified domain name of the domain controller (for example, dc01.contoso.com) must appear in one of the following places:" the subject CN or a DNS entry in the SAN. A client that connects by the domain name (contoso.com), a load-balancer or alias name (ldap.contoso.com), or an IP address checks that name instead, and a certificate issued only to DC2.contoso.com fails that check (our reading).

# Read-only. On a Windows client: does the served certificate pass SSL policy for the name the app uses?
# (Microsoft's Test-Certificate example uses -Policy SSL -DNSName 'dns=contoso.com'; the file is the one saved in Step 4)
$served = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2("$env:TEMP\dc2-ldaps.cer")
$served.DnsNameList
Test-Certificate -Cert $served -Policy SSL -DNSName 'dns=DC2.contoso.com'
Test-Certificate -Cert $served -Policy SSL -DNSName 'dns=ldap.contoso.com'

Microsoft's Test-Certificate reference: "If the DNSName parameter is used, then the DNS subject alternative name is used to verify SSL policy." Then read the application's LDAP server setting and the name it uses, and check that name in DNS:

# Read-only. Where does the name the app uses resolve? (names are examples)
Resolve-DnsName -Name ldap.contoso.com -DnsOnly
Resolve-DnsName -Name contoso.com -Type A -DnsOnly

The Directory Services team's older post adds a caveat for multi-name certificates: "The first name in the SAN has to match the FQDN of the server." Keep the DC's own FQDN first when you reissue (our habit).

Step 7: Who is binding unsigned or in clear text? (read-only, then one low-risk logging change)

This is the 389 side. Microsoft's definition of what the DC counts, from the Event 2889 text: "The following client performed a SASL (Negotiate/Kerberos/NTLM/Digest) LDAP bind without requesting signing (integrity verification), or performed a simple bind over a clear text (non-SSL/TLS-encrypted) LDAP connection."

The DC already logs summaries without any change:

Read the summaries on every DC first:

# Read-only. Every DC: the latest LDAP signing summaries (2886, 2887, 2888) from the Directory Service log
foreach ($dc in (Get-ADDomainController -Filter *).HostName) {
  Get-WinEvent -ComputerName $dc -FilterHashtable @{ LogName = 'Directory Service'; Id = 2886, 2887, 2888 } -MaxEvents 3 -ErrorAction SilentlyContinue |
    Select-Object @{ n = 'DC'; e = { $dc } }, TimeCreated, Id, Message | Format-List
}

The 2887 message ends with the two counts that matter: "Number of simple binds performed without SSL/TLS" and "Number of Negotiate/Kerberos/NTLM/Digest binds performed without signing". A DC that logs 2888 instead is already rejecting them, which is the "Strong Authentication Required" side.

To get names, raise the LDAP Interface Events level. Microsoft: "You can enable this additional logging by setting the 16 LDAP Interface Events diagnostic setting to 2 (Basic)." and, for the cost, "it has a negative effect on domain controller performance, and it should be disabled when it is no longer needed." Microsoft's command (a change, but a reversible logging change; do it on one DC at a time, our habit):

:: Low-risk change, on ONE DC. Raise LDAP Interface Events to 2 (Basic) to log Event 2889 per bind (Microsoft's form)
reg query HKLM\SYSTEM\CurrentControlSet\Services\NTDS\Diagnostics /v "16 LDAP Interface Events"
Reg Add HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTDS\Diagnostics /v "16 LDAP Interface Events" /t REG_DWORD /d 2

Leave it long enough to cover the clients' schedules (a nightly sync, a weekly report), then group what it logged:

# Read-only. On that DC: unsigned or clear-text binds in the last 24 hours, grouped by client, identity, and binding type
# (the property order IP:port, identity, binding type is our reading of the event text; check one event with Format-List first)
Get-WinEvent -FilterHashtable @{ LogName = 'Directory Service'; Id = 2889; StartTime = (Get-Date).AddDays(-1) } -ErrorAction SilentlyContinue |
  ForEach-Object {
    [pscustomobject]@{
      Client   = $_.Properties[0].Value -replace ':\d+$', ''
      Identity = $_.Properties[1].Value
      BindType = $_.Properties[2].Value
    }
  } |
  Group-Object -Property Client, Identity, BindType | Sort-Object -Property Count -Descending |
  Select-Object Count, Name | Format-Table -AutoSize

Microsoft's binding types in the 2889 text: "0 - SASL Bind that does not use signing" and "1 - Simple Bind that does not support signing".

Then turn it back down. Microsoft's revert: Reg Add HKLM\SYSTEM\CurrentControlSet\Services\NTDS\Diagnostics /v "16 LDAP Interface Events" /t REG_DWORD /d 0. The diagnostic logging article: "Make sure that you reset the entries to 0 after you finish investigating the problem."

Step 8: Which signing policy is actually in effect on each DC? (read-only)

Microsoft maps the Domain controller: LDAP server signing requirements policy to one registry value, LDAPServerIntegrity under NTDS\Parameters, with None = 1 and Require Signing = 2. Channel binding is LdapEnforceChannelBinding in the same key (Never = 0, When Supported = 1, Always = 2). Read both on every DC, plus the policy result:

:: Read-only. On each DC: the signing and channel binding values it holds (an error means the value isn't set)
reg query HKLM\SYSTEM\CurrentControlSet\Services\NTDS\Parameters /v LDAPServerIntegrity
reg query HKLM\SYSTEM\CurrentControlSet\Services\NTDS\Parameters /v LdapEnforceChannelBinding
:: Which GPO set them (open the report and search Security Options for "LDAP")
gpresult /scope computer /h %TEMP%\dc-rsop.html

Windows Server 2025 adds a second policy. Microsoft: "The Domain controller: LDAP server signing requirements enforcement policy takes precedence over the Domain controller: LDAP server signing requirements policy." Its Not Configured state isn't neutral: "Default: Not Configured, which has the same effect as Enabled (default for new deployments)." And for upgrades and mixed domains: "During an upgrade, Windows Server preserves existing signing policies to prevent disruption; if no LDAP server signing policy exists, Windows Server 2025 requires signing by default."

On the client side, Microsoft's default is fine for Windows: "All currently supported versions of Windows client use LDAP signing by default as LDAP clients." And its warning for the strict setting: "Not setting one of the sides will prevent client computers from communicating with the server."

LDAPS doesn't care about this policy. Microsoft's policy reference: "This setting doesn't have any impact on LDAP simple bind through SSL (LDAP TCP/636)." So "Strong Authentication Required" on 389 doesn't mean LDAPS is broken, and fixing the certificate won't fix a clear-text bind on 389 (our reading).

Step 9: Is channel binding rejecting a TLS bind? (read-only)

Channel binding is narrower than people expect. Microsoft: it doesn't apply to "Simple bind over TLS", to "Certificate bind over TLS", or to "Non-TLS SASL sessions". It only covers SASL (usually NTLM or Kerberos through Negotiate) inside LDAPS or STARTTLS. Microsoft's settings:

Defaults, per Microsoft: on Windows Server 2025 new deployments the policy "Defaults to When supported." On Windows Server 2022 or earlier it's "Set to Never" and "The 2023-03 security updates enable CBT auditing by default on domain controllers."

# Read-only. On each DC: channel binding events from the last 7 days (3039 fail, 3040/3041 summary or reminder, 3074/3075 audit)
Get-WinEvent -FilterHashtable @{ LogName = 'Directory Service'; Id = 3039, 3040, 3041, 3074, 3075; StartTime = (Get-Date).AddDays(-7) } -ErrorAction SilentlyContinue |
  Select-Object TimeCreated, Id, Message | Format-List

Microsoft: "Events 3039, 3074, and 3075 can only be generated when Channel Binding is set to When Supported or Always." Per-client events also need the same "16 LDAP Interface Events" level of 2 from Step 7.

Common false starts

Step 10: Change one link, then prove it

Only after Steps 1-9 agree on which link failed. Write down the current state first: the Step 3 certificate list with thumbprints, the Step 4 served thumbprint, the NTDS store contents, and the Step 8 reg query output from every DC (our habit). Export any certificate before you remove it.

No usable certificate, an expired one, or a missing key (Step 3). With a Microsoft Enterprise CA, Microsoft's route is certlm.msc → Personal → Certificates → All Tasks → Request New Certificate, then the Domain Controller template. Microsoft: "The Domain Controller template comes preconfigured with the correct properties for LDAPS, including:" Server Authentication, the FQDN in the SAN, and "Automatic renewal before expiration". If that enrollment fails, Certificate Autoenrollment Not Working is the next read. For a missing private key, the AskDS fix runs on the computer that generated the request, not the DC that received the certificate:

:: Change (low risk). On the computer where the request was generated: re-link the certificate to its private key (Microsoft's AskDS form; 0 is the store index)
certutil -repairstore MY 0

AskDS: "If we can't locate the private key container then we will need to request a new certificate."

A non-Microsoft CA, or a name the template doesn't cover (Steps 3 and 6). Microsoft's certreq request file, with a SAN section added from the certreq reference (DC FQDN first, our habit; names are examples):

;----------------- request.inf (Microsoft's LDAPS template, plus a SAN section) -----------------
[Version]
Signature="$Windows NT$"

[NewRequest]
Subject = "CN=DC2.contoso.com"
KeySpec = 1
KeyLength = 2048
Exportable = TRUE
MachineKeySet = TRUE
SMIME = False
PrivateKeyArchive = FALSE
UserProtected = FALSE
UseExistingKeySet = FALSE
ProviderName = "Microsoft RSA SChannel Cryptographic Provider"
ProviderType = 12
RequestType = PKCS10
KeyUsage = 0xa0

[EnhancedKeyUsageExtension]
OID=1.3.6.1.5.5.7.3.1 ; this is for Server Authentication

[Extensions]
2.5.29.17 = "{text}"
_continue_ = "DNS=DC2.contoso.com&"
_continue_ = "DNS=ldap.contoso.com"

:: Change. On the DC: create the request, submit it to the CA, then accept the issued certificate (Microsoft's commands)
certreq -new request.inf request.req
certreq -accept certnew.cer

The DC serves the wrong certificate (Step 4). Put the certificate you want into the AD DS service's store, which wins over the computer store. Microsoft's Event 1220 steps: Certificates snap-in → Service account → Local computer → Active Directory Domain Services → right-click NTDS\Personal → All Tasks → Import. Then make the DC reload without a restart. Microsoft: "A new rootDse operation that's named renewServerCertificate can be used to manually trigger AD DS to update its SSL certificates without having to restart AD DS or restart the domain controller." The LDIF from Microsoft's protocol spec, saved as renew.ldf:

dn:
changetype: modify
add: renewServerCertificate
renewServerCertificate: 1
-

:: Change (low risk). Import the rootDSE modify against the DC that should reload (-s is ours; the LDIF is Microsoft's)
ldifde -i -f renew.ldf -s DC2.contoso.com

Per the protocol spec, the account running it needs the Reload-SSL-Certificate control access right on that DC's nTDSDSA object. Microsoft also notes AD DS "detects when a new certificate is dropped into its certificate store and then triggers an SSL certificate update" by itself; the LDIF just forces it now.

The client doesn't trust the chain (Step 5). Add your root to the client's Trusted Root store and the issuing CA to Intermediate (Group Policy for Windows clients; the vendor's certificate import for an appliance). For one Windows test box, certutil can do it directly:

:: Change. On ONE Windows test client: add the root and the issuing CA (file names are examples)
certutil -addstore Root ContosoRootCA.cer
certutil -addstore CA ContosoIssuingCA.cer

If the chain is fine and only revocation fails, fix the CDP instead (Revocation Server Was Offline).

Unsigned or clear-text binds (Steps 7 and 8). Fix the client from Step 7's list, not the DC. Microsoft: "It's also quite possible that a solution for both topics consists in a single change. For example, by moving from simple bind to SASL using Kerberos or TLS with simple bind." For the badge server, that's its LDAP setting moved to port 636 with your CA imported (Steps 3-6 for the DC it uses). Then require signing on purpose, on every DC: Domain controller: LDAP server signing requirements = Require signing in the Default Domain Controllers Policy. On Windows Server 2025, leave Domain controller: LDAP server signing requirements enforcement at its default. If one application can't be fixed this week, a time-boxed exception is the enforcement policy set to Disabled with the older policy at None. Microsoft: "Disabled: The Domain controller: LDAP server signing requirements policy determines the LDAP signing requirement." Put an end date on the ticket (our habit).

Channel binding (Step 9). Start at When supported. Microsoft: "To maximize compatibility with older operating system versions (Windows Server 2008 and earlier versions), we recommend that you enable this setting with a value of 1." Move to Always only after 3039, 3074, and 3075 stay quiet on every DC. Use the Domain controller: LDAP server channel binding token requirements policy. If you test with the registry value on one DC first, Microsoft says the change is live without a restart: "LDAP server responds dynamically to changes to this registry entry."

Then prove it. For LDAPS, from the client side: re-run Step 4's served-certificate read (the new thumbprint), Step 5's certutil -verify, then Microsoft's ldp test. Microsoft: "Ensure that Port is set to 636, the Connectionless check box is cleared, and the SSL check box is selected, and then click OK." Then "RootDSE information should print in the right pane, indicating a successful connection." For signing, Microsoft's ldp simple-bind test on 389 carries its own warning: "Run it only in an isolated lab, use a nonprivileged disposable test account, and never enter production credentials." In production, Microsoft's check: "For production validation, monitor Event Viewer for Event ID 2888." The pass is the application working against every DC it can reach, no new Event 2889 from its IP, and Event 1220 gone from the DC (our reading). Then set "16 LDAP Interface Events" back to 0.

When a change is justified

FAQ

Why is LDAPS not working on my domain controller? Most often the DC has no certificate it can use (Event 1220), serves a different certificate than the one you renewed, or serves one the client doesn't trust or can't match to the name it used. Microsoft: "There's no user interface for configuring LDAPS." A valid certificate is what turns it on.

What is Event ID 2889? A per-bind record of an unsigned SASL bind or a clear-text simple bind on a DC, with the client IP, the identity, and the binding type (0 for SASL without signing, 1 for a simple bind). It only appears after "16 LDAP Interface Events" is set to 2; Event 2887 is the daily count that's logged without it.

What's the difference between LDAP signing and LDAPS? Microsoft: "LDAP signing provides integrity and authenticity but not confidentiality." LDAPS (TLS on 636) encrypts the whole session, and Microsoft says signing "applies only to non-TLS SASL sessions." A simple bind needs TLS to protect the password.

Do I need to restart the domain controller after replacing the LDAPS certificate? Usually not on current versions. Microsoft: AD DS detects a new certificate in its store, and renewServerCertificate triggers the update "without having to restart AD DS or restart the domain controller." Microsoft's enrollment steps still say to restart after the first certificate, so a restart in a maintenance window is the fallback.

What does "Strong Authentication Required" mean? The DC requires signing and the client sent a simple bind in clear text or an unsigned SASL bind. Microsoft's ldp test shows it as "Ldap_simple_bind_s() failed: Strong Authentication Required". Find the client in Event 2889 (Step 7) and move it to LDAPS or a signed bind.

Related posts

• Certificate Autoenrollment Not Working (when the DC never gets its Domain Controller certificate)
• Revocation Server Was Offline 0x80092013 (when the client can't check the DC certificate's CRL)
• NPS Event 6273 Reason Code 16 or 22 (the same server-certificate checks for PEAP on NPS)
• Group Policy Not Applying (when the signing policy reaches some DCs and not others)
• Windows Time Not Syncing With Domain Controller (a wrong clock makes a valid certificate look expired)

Free checklist

Requiring LDAP signing before anyone read Event 2889 is enforcement before audit, and so is turning it back off domain-wide when one badge server breaks. A Conditional Access policy enforced before anyone read the report-only results is the same mistake. The free Conditional Access pre-enforcement checklist keeps break-glass accounts, pilot groups, and report-only review on one page:

👉 https://cashflow4375.gumroad.com/l/ujkqxb?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=ldaps-not-working-domain-controller

If you want the rest of the Intune runway written down for the team (enrollment hygiene, compliance phasing, break/fix cards and read-only PowerShell snapshots), the Intune & M365 Admin Starter Pack pairs with this triage: https://cashflow4375.gumroad.com/l/joonf?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=ldaps-not-working-domain-controller

Admin Pack Studio; not affiliated with Microsoft. Steps 1-9 favor read-only checks: Test-NetConnection, Get-NetTCPConnection, Get-WinEvent, Get-ChildItem Cert:\LocalMachine\My, certutil -verifystore, -store, and -verify -urlfetch, the .NET SslStream read of the served certificate, Test-Certificate, Resolve-DnsName, Get-ADDomainController, reg query, gpresult, and ldp.exe connects. Raising and lowering "16 LDAP Interface Events", requesting, accepting, importing, or repairing certificates, renewServerCertificate, adding CA certificates to client stores, and changing LDAP signing, signing enforcement, or channel binding policy are deliberate changes. Change one link at a time, test from the failing client against every DC it can reach, and check current Microsoft Learn for version-specific behavior.

Not yet. "The target principal name is incorrect" doesn't mean Active Directory is corrupt. It means a Kerberos ticket reached a computer that couldn't decrypt it: the wrong computer, the right computer holding a different password than the KDC used, or a name (SPN) registered on the wrong account. Microsoft's replication article says the error "occurs when the source domain controller doesn't decrypt the service ticket provided by the destination (target) domain controller", and that the -2146893022 (0x80090322, SEC_E_WRONG_PRINCIPAL) "error code isn't an Active Directory error." Its top cause: "the KDC has an old version of the password for the source domain controller." Stopping the KDC on every DC, forcing a demotion, or rejoining a server can each hide which of those three it was, and a forced demotion leaves metadata behind that you then have to clean up by hand. Work through it as a chain: which computers, which name, since when → where is the error reported (repadmin, dcdiag, Kerberos Event 4, KDC Event 11)? → did the traffic reach the computer you meant (DNS, hosts file, NTDS Settings GUID)? → which KDC issued the ticket, and does it have the current password for the target? → are the target's SPNs present, and on only one account? → what does the client's Event 4 name as the target and the server that answered? → is an alias (CNAME) involved? → are time and the secure channel in order? → is this a demotion? repadmin, dcdiag, nltest, setspn, Get-WinEvent, Resolve-DnsName, and Get-ADComputer are the sources for that chain. Stop the KDC everywhere and you'll be finding out later which DC was handing out the stale ticket.

If replication fails with 1722 (the RPC server is unavailable) instead, no connection was made yet, and AD Replication Error 1722 starts earlier in the chain. If the message ends in "Cannot generate SSPI context" from SQL Server, SQL Server Error 18456 covers SQL SPNs in depth. This post covers "Active Directory replication, a domain controller, a demotion, or a file server fails with 'The target principal name is incorrect' (-2146893022, 0x80090322), 'target account name is incorrect', or Kerberos Event 4 KRB_AP_ERR_MODIFIED, and you need a read-first path before you stop KDCs, reset DC passwords, delete SPNs, force a demotion, or rejoin servers."

TL;DR

• It's a decryption failure, not a network failure. Microsoft: "Kerberos errors that are mapped by Windows code to -2146893022" include KRB_AP_ERR_MODIFIED, KRB_AP_ERR_BADMATCH, and KRB_AP_ERR_NOT_US. The connection worked; the ticket didn't (Step 2).
• Check you reached the right computer first. Microsoft's first specific cause: "A bad name-to-IP mapping in DNS, WINS, HOST, or LMHOST file." (Step 3).
• Then the password the KDC used. Microsoft: "Replication failures can prevent domain controllers from having a current password value for domain controllers in a given domain." (Step 4).
• SPNs have to be on one account. Microsoft: "Unless a service uses the computer account and the HOST SPN, SPNs must be unique in the AD DS forest." (Step 5).
• Event 4 names both sides. Microsoft's Event 4 text: "Please ensure that the target SPN is registered on, and only registered on, the account used by the server." (Step 6).
• Forced demotion is a last resort. Microsoft: "Forced demotion leaves orphaned metadata in Active Directory on the remaining domain controllers in the forest." (Step 9).

Where in the chain did it break?

Step 1: Capture the symptom (read-only)

Write these on the ticket first:

Then branch on what you see:

Step 2: Find where the error is reported (read-only)

For replication, Microsoft lists the commands that show it: REPADMIN /REPLSUM, /SHOWREPL, /SHOWREPS, and /SYNCALL, plus DCDIAG's replications test. Start forest-wide:

:: Read-only. From a DC or an admin box with RSAT: every DC's inbound replication, errors only (Microsoft's repadmin forms)
repadmin /showrepl * /errorsonly
repadmin /showrepl * /csv > showrepl.csv

Microsoft's own shortcut for reading the CSV is repadmin /showrepl * /csv | convertfrom-csv | out-gridview. To pull out only the rows with this error:

# Read-only. Which source and destination pairs report -2146893022 (the Select-String filter is ours)
repadmin /showrepl * /csv | Select-String -Pattern '2146893022'

List every pair. If one source DC appears in every row, that DC is where Step 4 starts. Microsoft's event table for this error includes NTDS KCC 1925 ("The attempt to establish a replication link for the following writable directory partition failed.") and 1308 in the Directory Service log on the destination.

For member servers and clients, the evidence is Kerberos Event 4. Microsoft: "The client then puts Kerberos event 4 (example below) in its System event log."

# Read-only. On the client that can't connect: Kerberos Event 4 in the System log, last 7 days
# (other providers also log an Id 4, so the provider filter is ours)
Get-WinEvent -FilterHashtable @{ LogName = 'System'; Id = 4; StartTime = (Get-Date).AddDays(-7) } -ErrorAction SilentlyContinue |
  Where-Object { $_.ProviderName -match 'Kerberos' } |
  Select-Object TimeCreated, ProviderName, Message | Format-List

On the DCs, look for duplicate names. Microsoft's archived KDC Event 11 message: "There are multiple accounts with name %1 of type %2." On Windows Server 2012 R2 and later DCs, a blocked attempt to add a duplicate is Event 2974 in the Directory Service log. Microsoft: "The event 2974 lists the value that was blocked and a list of one or more objects (up to 10) that already contain that value."

# Read-only. On each DC: KDC Event 11 (System log) and Event 2974 (Directory Service log), last 7 days
# (the provider match for the KDC is ours)
$since = (Get-Date).AddDays(-7)
Get-WinEvent -FilterHashtable @{ LogName = 'System'; Id = 11; StartTime = $since } -ErrorAction SilentlyContinue |
  Where-Object { $_.ProviderName -match 'KDC|Key-Distribution' } |
  Select-Object TimeCreated, Message | Format-List
Get-WinEvent -FilterHashtable @{ LogName = 'Directory Service'; Id = 2974; StartTime = $since } -ErrorAction SilentlyContinue |
  Select-Object TimeCreated, Message | Format-List

Microsoft's Kerberos guidance says to check the Security and System logs on "The authenticating client", "The target server or service", and "The domain controller". Do all three before deciding which one is at fault.

Step 3: Did the traffic reach the computer you meant? (read-only)

Microsoft: "The following situations can all cause a destination domain controller to submit Kerberos-encrypted traffic to the wrong Kerberos target:" stale NTDS Settings objects, bad name-to-IP mappings in DNS and WINS host records, and stale entries in HOST files. If DC1 asks for a ticket for DC2 but the packets land on DC3, DC3 can't decrypt a ticket made for DC2. Microsoft notes the realm in the error can even match: "Bad host-to-IP mappings could cause DC1 to connect to DC3 in the same domain."

How a destination DC finds its source, per Microsoft: it looks up the source's NTDS Settings objectGUID, queries DNS for "a matching DC GUIDED CNAME record", and follows that to the source's host A or AAAA record. Microsoft's check without a network trace, on the source DC first:

:: Read-only. On the SOURCE DC: its IP addresses, and its own DSA object GUID (top of the output; Microsoft's article uses /showreps)
ipconfig /all
repadmin /showreps

Then on the destination DC (Microsoft's sequence; DC1, the GUID, and the DNS server IP are examples):

:: Read-only. On the DESTINATION DC: which DNS servers it asks, and the source's GUID as this DC sees it
ipconfig /all
repadmin /showreps
:: Read-only. What each of those DNS servers returns for the source's GUID CNAME and host record (repeat per DNS server IP)
nslookup -type=cname 6f3f96d3-dfbf-4daf-9236-4d6da6909dd2._msdcs.contoso.com 10.0.0.10
nslookup -type=A+AAAA DC1.contoso.com 10.0.0.10

Microsoft: "The DSA object GUID listed for the source DC in the replication status section of the report should match the object GUID listed in the /showreps header when run on the console of the source DC." For the host records: "Check for duplicate host A records on all DNS Server IPs configured on the destination DC."

The same reads in PowerShell, one pass per DNS server (our form of Microsoft's nslookup steps):

# Read-only. On the destination DC. Fill in the DNS server IPs from ipconfig /all and the source's DSA object GUID
$dnsServers = '10.0.0.10', '10.0.0.11'
$guidName   = '6f3f96d3-dfbf-4daf-9236-4d6da6909dd2._msdcs.contoso.com'
foreach ($d in $dnsServers) {
  "--- DNS server $d"
  Resolve-DnsName -Name $guidName -Type CNAME -Server $d -DnsOnly -ErrorAction SilentlyContinue | Format-Table -AutoSize
  Resolve-DnsName -Name 'DC1.contoso.com' -Type A -Server $d -DnsOnly -ErrorAction SilentlyContinue | Format-Table -AutoSize
}
# The hosts file: any line naming the source DC?
Select-String -Path "$env:SystemRoot\System32\drivers\etc\hosts" -Pattern 'DC1'

For a member server or client, the same question applies to the server name it used: does Resolve-DnsName for that name, against the client's DNS servers, return the server you meant? If the client can't find a DC at all, Netlogon Event ID 5719 covers DC location.

Step 4: Which KDC issued the ticket, and does it have the current password? (read-only)

This is Microsoft's top cause. Every DC runs a KDC, and Microsoft: "For same realm transactions, a destination domain controller favors getting Kerberos tickets from itself. However, it may get a ticket from a remote domain controller." If whichever KDC issued the ticket holds an older password for the source DC than the source itself does, the source can't decrypt it.

Why the passwords drift: Microsoft's netdom article says "Each Windows-based computer maintains a machine account password history that contains the current and previous passwords that are used for the account," and "If the sequence of password changes exceeds two changes, the computers involved may not be able to communicate, and you may receive error messages." A DC that was offline, or couldn't replicate, while its partners changed passwords is the usual way to get there (our reading). Microsoft's replication article: "Replication failures can prevent domain controllers from having a current password value for domain controllers in a given domain."

Find the KDC, close to the time of the failure:

:: Read-only. On the destination (or the client): which KDC does it find? /FORCE skips the cache
nltest /dsgetdc:contoso.com /kdc /force

Microsoft: the nltest /dsgetdc command "that's run at an elevated command prompt in close time proximity to a SEC_E_WRONG_PRINCIPAL error can be used to quickly identify which KDC a Kerberos client is targeting." The definitive answer is a network trace; Microsoft: "The definitive way to determine which domain controller a Kerberos client got a ticket from is to take a network trace."

Then compare the source DC's password attributes as the source sees them and as that KDC sees them:

:: Read-only. The source DC's computer account: on the source (DC1), then on the KDC the destination used (DC3). Names are examples
repadmin /showobjmeta DC1 "CN=DC1,OU=Domain Controllers,DC=contoso,DC=com"
repadmin /showobjmeta DC3 "CN=DC1,OU=Domain Controllers,DC=contoso,DC=com"

Microsoft: look "for obvious version number differences in password-related attributes (dBCSPwd, UnicodePWD, NtPwdHistory, PwdLastSet, lmPwdHistory)". The same comparison as a quick read across every DC:

# Read-only. PasswordLastSet for the source DC's account (DC1) as each DC currently holds it (the loop is ours)
$source = 'DC1'
foreach ($dc in (Get-ADDomainController -Filter *)) {
  Get-ADComputer -Identity $source -Server $dc.HostName -Properties PasswordLastSet |
    Select-Object @{ n = 'AskedDC'; e = { $dc.HostName } }, Name, PasswordLastSet
}

Step 5: Are the target's SPNs present, and on only one account? (read-only)

Microsoft: "SPNs may be missing, invalid, or duplicated due to simple replication latency, especially following promotion, or replication failures." And: "Duplicate SPNs may cause bad SPN to name mappings." Microsoft's first resolution step for the replication error is a DCDIAG test that isn't run by default:

:: Read-only. On the console of each source DC that fails outbound replication with this error
dcdiag /test:checksecurityerror
:: Read-only. Add the partner check (DC2 is an example)
dcdiag /test:checksecurityerror /ReplSource:DC2.contoso.com

Microsoft: "Run this command on the console of all source domain controllers that fail outbound replication with the SEC_E_WRONG_PRINCIPAL error." Per the dcdiag reference, the test checks that "at least one KDC is online for each domain and is reachable", that the DC's computer object has replicated to other DCs, and that the account "has the minimum required Service Principal Names (SPN) configured." With /ReplSource, it also checks time skew: "The time skew calculated between the servers to verify it's less than 300 seconds (5 minutes) for Kerberos."

Then read the SPNs directly:

:: Read-only. The SPNs registered on the DC's own computer account (DC1 is an example)
setspn -L DC1
:: Read-only. Which account holds a given SPN
setspn -Q HOST/DC1.contoso.com
:: Read-only. Search for duplicate SPNs, forest-wide (can take a while)
setspn -X -F

From Microsoft's setspn reference: -L "Lists the currently registered SPN for the computer.", -Q "Queries for any existing SPNs.", -X "Performs a search of duplicate SPNs.", and -F "Performs queries at the forest, rather than domain level." (combining -X with -F is our form). Microsoft warns: "Searching for duplicates, especially forest-wide, can take a long time and lots of memory." Run it from one admin box, not every DC at once (our habit).

The replication SPN in a DC's ticket request is the replication service UUID plus the source's NTDS Settings GUID, as in Microsoft's trace: E3514235-4B06-11D1-AB04-00C04FC2DCD2/6f3f96d3-dfbf-4daf-9236-4d6da6909dd2/contoso.com. Microsoft: "If the source domain controller's SPN doesn't exist in the realm, you receive a KDC_ERR_S_PRINCIPAL_UNKNOWN error."

Since Windows Server 2012 R2, Microsoft says DCs "block the creation of duplicate service principal names (SPN) and user principal names (UPN)", so new duplicates usually come from writes to older DCs or from before that (our reading). Microsoft's blocked-write error 8647 reads "The operation failed because SPN value provided for addition/modification isn't unique forest-wide."

Step 6: Read Kerberos Event 4 on the client (read-only)

On member servers and clients, the error behind "target principal name" and "target account name" messages is usually a Kerberos Event 4 with KRB_AP_ERR_MODIFIED (our reading of Microsoft's Event 4 articles). Microsoft's Kerberos guidance: "Because more than one issue can cause this error, check for related events. For example, this string might mean that the client and target server clocks are out of sync, or that the SPN isn't unique." The current event text, from Microsoft's file-share article, names three causes:

• The SPN on the wrong account: "This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using."
• A password mismatch: "This error can also happen when the target service is using a different password for the target service account than what the Kerberos Key Distribution Center (KDC) has for the target service account."
• Two accounts with the same name: "If the server name is not fully qualified, and the target domain (DNS_prefix.dns_suffix) is different from the client domain (DNS_prefix.dns_suffix), check if there are identically named server accounts in these two domains, or use the fully-qualified name to identify the server."

Read the two names in the message. Microsoft's example: "The kerberos client received a KRB_AP_ERR_MODIFIED error from the server host/machinename.childdomain.rootdomain.com. The target name used was cifs/machinename.domain.com." The first is the server that answered; the second is the SPN the client asked a ticket for (our reading).

:: Read-only on the directory. On the client: who holds the SPN from "The target name used was", then ask for that ticket
setspn -Q cifs/FS01.contoso.com
klist get cifs/FS01.contoso.com
klist

klist get "Allows you to request a ticket to the target that is specified by the SPN." It adds a ticket to your cache; nothing changes in AD.

Step 7: Is an alias (CNAME) involved? (read-only)

The member-server version of this error often comes from a name the server doesn't own. Microsoft's SMB article: with a CNAME, \\CNAME\share_name can fail with "Logon Failure: The target account name is incorrect." while the NetBIOS name and FQDN work. Microsoft's causes: "Configuration issue or SMB server service hardening issue." and "The service principal name (SPN) for the DNS alias (CNAME) record for the file share server is missing."

:: Read-only. On the server (or with RSAT): which names does it own, and does each have a DNS record and an SPN? (FS01 is an example)
netdom computername FS01.contoso.com /enumerate:allnames
netdom computername FS01.contoso.com /verify
setspn -L FS01

# Read-only. On the client: what the alias resolves to (files.contoso.com is an example)
Resolve-DnsName -Name 'files.contoso.com' -DnsOnly | Format-Table -AutoSize

From Microsoft's netdom reference, /verify "Checks if there's a DNS record and a service principal name (SPN) for each computer name." If the alias resolves through a CNAME and isn't in the /enumerate list, and no SPN for it appears in setspn -L, that's this case. Microsoft's fix is in Step 10. If the alias points at a cluster name or a NAS, the SPN belongs on that object's account instead (our reading).

Step 8: Are time and the DC secure channel in order? (read-only)

Time. Microsoft's Kerberos guidance: "The difference in time between the domain controller and the client computer or the target server must be less than five minutes." A skew failure is logged as Event 4 with KERB_AP_ERR_SKEW (Microsoft's table), not MODIFIED, but check it while you're here:

:: Read-only. Three samples of the offset from this DC to its partner (DC1 is an example)
w32tm /stripchart /computer:DC1.contoso.com /samples:3 /dataonly

If the offset is minutes, read Windows Time Not Syncing With Domain Controller before you change any passwords.

The DC's secure channel. Microsoft's article on this error during manual replication lists Netlogon events alongside it: 3210 ("Failed to authenticate with \\DOMAINDC, a Windows NT domain controller for domain DOMAIN.") and 5722 ("The session setup from the computer 1 failed to authenticate."). On a DC, don't use Test-ComputerSecureChannel. Microsoft: "This cmdlet only works on Domain Member computers. When you run it on Domain Controllers, it returns false positive errors. To verify and reset the secure channels for Domain Controllers, use netdom.exe or nltest.exe."

:: Read-only. On the DC: state of its secure channel the last time it was used, and the PDC emulator
nltest /sc_query:contoso.com
netdom query fsmo

/sc_query "Reports on the state of the secure channel the last time that you used it." Leave nltest /sc_verify for later: Microsoft says if the channel doesn't work, "this parameter removes the existing channel, and then builds a new one." That's a change. For member servers and workstations, Trust Relationship Between This Workstation and the Primary Domain Failed covers the member-side secure channel and machine password.

The machine password itself. Microsoft's Directory Services blog: "Machine account passwords as such do not expire in Active Directory." And: "It is important to remember that machine account password changes are driven by the CLIENT (computer), and not the AD." So a DC that was offline for a month doesn't come back with an expired password; it comes back possibly behind on its partners' passwords, or with its own change not yet replicated (our reading, Step 4).

SQL Server. If the message is "The target principal name is incorrect. Cannot generate SSPI context.", that's SQL Server's version. Microsoft: "In most cases, a misconfigured Service Principal Name (SPN) causes this error." The SQL side (service account, MSSQLSvc SPNs, Kerberos Configuration Manager) is in SQL Server Error 18456; Steps 3, 5, and 8 here still apply to the DCs behind it.

Step 9: Is this a demotion? (read-only)

A demotion has to replicate its last changes out and talk to a partner, so a DC that can't authenticate to its partners fails here too (our reading). Microsoft's DC deployment troubleshooting: "The built-in logs are the most important instrument for troubleshooting issues with domain controller promotion and demotion." Read them first:

# Read-only. On the DC that won't demote: the last 60 lines of each demotion log, then search them for the error (the Select-String filter is ours)
Get-Content -Path "$env:SystemRoot\debug\dcpromoui.log" -Tail 60
Get-Content -Path "$env:SystemRoot\debug\dcpromo.log" -Tail 60 -ErrorAction SilentlyContinue
Select-String -Path "$env:SystemRoot\debug\dcpromo*.log" -Pattern 'target principal|2146893022|0x80090322'

Microsoft: "Examine the Dcpromoui.log and dcpromo.log for the errors shown in the output, then work backwards from them to see indications of why the failure occurred." Then run Steps 2-5 for this DC as a replication source: its outbound replication, its DNS records, its password versions, and dcdiag /test:checksecurityerror on its console. If SYSVOL (DFS Replication) is what's failing during the demotion, DFS Replication Not Working covers the DFSR side.

Forced removal. Microsoft is direct about it: "Don't select this option unless the domain controller can't contact other domain controllers and there's no reasonable way to resolve that network issue." Also: "all unreplicated changes on that domain controller, such as passwords or new user accounts, are lost forever." And: "If you forcibly demote a domain controller, you must manually perform metadata cleanup immediately."

A community answer in an archived TechNet forum thread on this exact failure: "I was able to get my DC to demote after stopping the KDC service. Then I reran the demotion with force removal and it completed successfully." That's a forced removal, with the metadata cleanup it requires (community, labeled). If the DC still replicates with anyone, fix the password or DNS link first (Step 10) and demote gracefully.

Common false starts

Step 10: Change one link, then prove it

Only after Steps 1-9 agree on which link failed. Write down the current state first: the repadmin /showrepl * /csv output from Step 2, the setspn -L output for every account you're about to touch, and the repadmin /showobjmeta pair from Step 4 (our habit). An SPN you delete by mistake is easy to put back only if you saved the list.

Wrong name-to-IP (Step 3). Fix the record, not the client. Delete the stale or duplicate A record on the DNS server, or the hosts entry, then clear the caches on the destination. Microsoft's steps: IPCONFIG /FLUSHDNS to "Clear the DNS Client cache", and "Run Nbtstat -R (upper case R) to refresh the NetBIOS name cache." If scavenging should have removed the record, DNS Scavenging Not Working covers why it didn't.

:: Change (low risk). On the destination DC, after the record or hosts entry is fixed
ipconfig /flushdns
nbtstat -R
klist -li 0x3e7 purge

Microsoft: "Kerberos tickets for the system account that are used by Active Directory replication can be purged without a restart by using KLIST -li 0x3e7 purge."

KDC holding an old password for the source (Step 4). Microsoft's top resolution, on the destination DC: disable its KDC so it gets a ticket from another DC, restart, replicate from the source, then put the KDC back. Microsoft: "Domain controllers can be made to use other domain controllers by stopping the KDC service on a local or remote domain controller."

# Change, on the DESTINATION DC (BR-DC2). Microsoft's sequence; dry-run the startup change first
Set-Service -Name KDC -StartupType Disabled -WhatIf
# ...restart the destination DC, then replicate from the source. Microsoft's form, run against the destination:
#    repadmin /replicate BR-DC2.contoso.com DC1.contoso.com "DC=contoso,DC=com"
Set-Service -Name KDC -StartupType Automatic -WhatIf   # Microsoft's article writes Auto; Set-Service documents Automatic
Start-Service -Name KDC -WhatIf

Remove -WhatIf when the dry run shows the service you meant. Microsoft: "If it doesn't resolve the issue, see the Resolution section for an alternative solution in which you use the netdom resetpwd command to reset the computer account password of the source domain controller."

The source DC's own password (Step 4). Run this on the DC whose password is wrong; Microsoft: "This command must be run on the DC whose machine account password you wish to reset." Microsoft's manual-replication article resets it against the PDC emulator, so find that first. Microsoft: "If you want to reset the password for a Windows domain controller, you must stop the Kerberos Key Distribution Center service and set its startup type to Manual."

:: Change, on the DC whose password is wrong (DC1). PDC01 is the PDC emulator from netdom query fsmo; names are examples
netdom query fsmo
sc config KDC start= demand
net stop KDC
klist -li 0x3e7 purge
netdom resetpwd /server:PDC01 /userd:contoso\admin.user /passwordd:*

demand is the sc config value for a manual start, from Microsoft's sc config reference. Then Microsoft: "Restart the server whose password was changed." After the restart, start the KDC and set it back to Automatic (sc config KDC start= auto, then net start KDC). If you run netdom resetpwd on the PDC emulator itself, point /server: at another healthy DC (our reading of Microsoft's "the name of the domain controller to use for setting the machine account password").

SPN on the wrong account, or duplicated (Steps 5 and 6). Microsoft's three commands, in order:

:: Change. Find, remove from the wrong account, add to the right one (SPN and account names are examples)
setspn -Q cifs/FS01.contoso.com
setspn -D cifs/FS01.contoso.com OLD-FS01
setspn -S cifs/FS01.contoso.com FS01

Microsoft's Event 4 article runs these "using Enterprise Administrator credentials". Use -S, not -A; Microsoft says Setspn.exe has had "duplicate SPN detection built-in to it since the Windows Server 2008 release" with the -S option. If the right account is a service account rather than the computer, add it there instead; Microsoft: "If the SPN is assigned to any account other than the correct service account (custom account or computer account), remove the SPN from those accounts."

Alias without an SPN (Step 7). Microsoft: "avoid using the DNS CNAME to configure the file server's alternative name." Add the alias to the server's own account:

:: Change, on the file server (elevated). Then verify; remove the old CNAME only after clients work by the alias (our habit)
netdom computername FS01.contoso.com /add:files.contoso.com
netdom computername FS01.contoso.com /verify
setspn -L FS01

Per Microsoft's Core Infrastructure blog post on aliases, the command will "register the DNS A record, register additional SPNs, and add OptionalNames registry key." If the old CNAME stays in DNS, clients may keep using it, so plan its removal (our reading).

The demotion (Step 9). Fix the password or DNS link above, confirm the DC replicates outbound, then demote normally. A forced removal (Uninstall-ADDSDomainController -ForceRemoval -DemoteOperationMasterRole) is only for a DC that truly can't reach a partner. Microsoft: "Metadata cleanup is a required procedure after a forced removal of Active Directory Domain Services (AD DS)." Microsoft's console route deletes the old DC's computer object in Active Directory Users and Computers with This Domain Controller is permanently offline and can no longer be demoted using the Active Directory Domain Services Installation Wizard (DCPROMO) selected. Then delete its stale DNS records.

Then prove it. For replication, one destination, one source, one partition. Microsoft's article on diagnosing replication lists "Time accuracy within 5 minutes to support Kerberos authentication" among its dependencies, so run the Step 8 time read again if you changed anything near the clocks.

:: Read-mostly. Trigger one inbound replication, then re-read status (names and partition are examples)
repadmin /replicate BR-DC2.contoso.com DC1.contoso.com "DC=contoso,DC=com"
repadmin /showrepl BR-DC2.contoso.com
dcdiag /test:checksecurityerror /ReplSource:DC1.contoso.com

For a member server, purge only your own tickets, connect by the same name that failed, and look for the new service ticket:

:: Low-risk. On the client, in the user's own session
klist purge
dir \\files.contoso.com\share
klist

The pass is a successful replication with no new -2146893022 in /showrepl, or a share that opens by the alias with a cifs/ ticket listed by klist and no new Event 4 (Step 2's read). Then the next pair, or the next client, since a destination that pulled from one source can still hold a bad ticket for another (our reading).

When a change is justified

FAQ

What does "The target principal name is incorrect" mean? A computer received a Kerberos ticket it couldn't decrypt. Microsoft's top cause in AD replication: "the KDC has an old version of the password for the source domain controller." The other causes Microsoft lists are a bad name-to-IP mapping (the ticket went to the wrong computer), an SPN the KDC couldn't find or found on the wrong account, and frames modified in transit.

What is error -2146893022 (0x80090322)? It's SEC_E_WRONG_PRINCIPAL, the code behind "The target principal name is incorrect." Microsoft: it "isn't an Active Directory error" and can come from RPC, Kerberos, SSL, LSA, or NTLM. In replication, Microsoft's article shows it remapped from the Kerberos errors KRB_AP_ERR_MODIFIED or KRB_AP_ERR_NOT_US.

How do I find duplicate SPNs? setspn -Q <SPN> shows which account holds one SPN; setspn -X searches for duplicates (add -F for the whole forest). On DCs, KDC Event 11 reports multiple accounts with the same name, and Event 2974 reports a blocked attempt to add a duplicate.

Should I stop the KDC on every domain controller? No. Microsoft's top resolution disables the KDC on the destination DC only, restarts it, replicates, and turns it back on. Its netdom article allows stopping the KDC "on all domain controllers except one" only if needed. With none running, no one in the domain can get a ticket.

Is this the same as SQL Server's "Cannot generate SSPI context"? Same family. SQL Server shows "The target principal name is incorrect. Cannot generate SSPI context." and Microsoft says a misconfigured SPN causes it in most cases. Start with the SQL Server post for the service account side; the DNS, SPN, and time steps here still apply.

Related posts

• AD Replication Error 1722 (the sibling when no connection was made: DNS, RPC ports, firewalls, NICs)
• SQL Server Error 18456 (SQL SPNs, service accounts, and "Cannot generate SSPI context")
• DNS Scavenging Not Working (the stale A records that send a ticket to the wrong computer)
• Netlogon Event ID 5719 (when a computer can't find a domain controller at all)
• Windows Time Not Syncing With Domain Controller (the five-minute Kerberos clock limit)
• Trust Relationship Between This Workstation and the Primary Domain Failed (the member-side secure channel and machine password)
• DFS Replication Not Working (SYSVOL replication, including during a demotion)

Free checklist

Stopping the KDC on every domain controller before anyone read which DC issued the bad ticket is a change made everywhere at once. So is a Conditional Access policy enforced before anyone read the report-only results. The free Conditional Access pre-enforcement checklist keeps break-glass accounts, pilot groups, and report-only review on one page:

👉 https://cashflow4375.gumroad.com/l/ujkqxb?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=target-principal-name-is-incorrect

If you want the rest of the Intune runway written down for the team (enrollment hygiene, compliance phasing, break/fix cards and read-only PowerShell snapshots), the Intune & M365 Admin Starter Pack pairs with this triage: https://cashflow4375.gumroad.com/l/joonf?utm_source=hashnode&utm_medium=article&utm_campaign=aps_first_sale&utm_content=target-principal-name-is-incorrect

Admin Pack Studio; not affiliated with Microsoft. Steps 1-9 favor read-only checks: repadmin /showrepl, /showreps, and /showobjmeta, dcdiag /test:checksecurityerror, Get-WinEvent, ipconfig /all, nslookup, Resolve-DnsName, Select-String, nltest /dsgetdc and /sc_query, Get-ADDomainController, Get-ADComputer, setspn -L, -Q, and -X, klist, netdom computername /enumerate and /verify, netdom query fsmo, w32tm /stripchart, and Get-Content on the demotion logs. klist get adds a ticket to your own cache. Fixing DNS records and hosts entries, flushing caches, purging tickets, disabling or stopping the KDC, netdom resetpwd, setspn -D and -S, netdom computername /add, nltest /sc_verify and /sc_reset, demotion, forced removal, and metadata cleanup are deliberate changes. Change one link at a time, test from the failing pair or client, and check current Microsoft Learn for version-specific behavior.

Not yet. Event 6273 doesn't mean NPS is broken. It means NPS received the request, ran it through its policies, and refused it, and the event itself records which policy, which EAP type, and which reason. Microsoft's Event 6273 message is "Network Policy Server denied access to a user." Reason code 16 is "Authentication failed due to a user credentials mismatch. Either the user name provided does not match an existing user account or the password was incorrect." Reason code 22 is "Network Policy Server was unable to negotiate the use of an Extensible Authentication Protocol (EAP) type with the client computer." Neither one says the password was wrong for certain. Reason 16 also shows up when a certificate can't be mapped to an account, and reason 22 shows up when the client and the policy never agreed on a method, often because of the NPS server's own certificate (our reading, Steps 4 and 6). Ignoring dial-in properties doesn't touch either of those, and an accept-without-credentials policy opens the network to everyone. Work through it as a chain: which network, which users or devices, since when → is NPS auditing on, and is it 6273 or something else? → read one 6273 end to end (policy names, authentication type, EAP type, account, reason) → reason 22: did the client and the policy agree on an EAP type, and is the NPS server certificate valid and bound? → reason 16 with passwords: is the account real, unlocked, and reachable on a DC? → reason 16 with certificates: does the certificate map to the account (strong mapping since February 2025)? → does the request reach the policy you think it does? → is the NPS server itself registered, logging, and healthy? auditpol, Get-WinEvent, the NPS accounting log, Get-ADUser, the certificate store, and certutil are the sources for that chain. Accept everyone and you'll be finding out later which devices got on while it was open.

If the Wi-Fi or VPN profile never reached the device, Intune Wi-Fi Profile Not Applying starts earlier in the chain. If the rejections began when a CRL expired or a CA changed, Revocation Server Was Offline reads the chain first. This post covers "NPS (Network Policy Server) on Windows Server rejects 802.1X Wi-Fi, wired 802.1X, or VPN connections with Event 6273 and reason code 16 or 22, and you need a read-first path before you loosen policies, accept unauthenticated users, roll back security updates, or reissue every certificate."

TL;DR

• The event names the policy that rejected the request. Microsoft's 802.1X guide: "NPS event log entries contain information about the connection attempt, including the name of the connection request policy that matched the connection attempt and the network policy that accepted or rejected the connection attempt." (Step 3).
• Password or certificate decides which reason-16 path you take. Microsoft's 6273 page: "Use the information provided in Event Viewer to determine whether the authentication method that applies to the user connection is password- or certificate-based." (Steps 5 and 6).
• Reason 22 is often the NPS server certificate. Microsoft: "In Network Policy Server (NPS) on Windows Server, the server certificate that you bind to a network policy isn't updated automatically when it expires." (Step 4).
• Certificate users rejected since early 2025 need a strong mapping. Microsoft's KB5014754 on Full Enforcement mode: "Authentication will be denied if a certificate cannot be strongly mapped." (Step 6).
• Policies are an ordered list. Microsoft: "NPS checks each connection request against the first rule in the list, then the second, and so on, until a match is found." (Step 7).
• Certificates are the usual 802.1X break. Microsoft: "Most 802.1X authentication issues are because of problems with the certificate that's used for client or server authentication." (Steps 4, 6, 9).

Where in the chain did it break?

Step 1: Capture the symptom (read-only)

Write these on the ticket first:

Then branch on what you see:

Step 2: Confirm auditing, then find the events (read-only)

Microsoft's NPS troubleshooting checklist starts with the audit policy:

:: Read-only. On each NPS server: is the Network Policy Server audit subcategory on?
auditpol /get /subcategory:"Network Policy Server"

Microsoft says auditing is on when that shows Success and Failure, or Failure. If it shows No Auditing, the fix is Microsoft's auditpol /set line in Step 10, and Microsoft adds: "Even if audit policy appears to be fully enabled, it sometimes helps to disable and then re-enable this setting." Microsoft's audit page notes "This subcategory generates events only if NAS or IAS role is installed on the server."

The events live in the Security log on the NPS server, from Microsoft-Windows-Security-Auditing (Microsoft's 6273 event details). Microsoft's checklist: "Check for events that have Event ID 6273 or 6274. Most authentication failures produce these events." And: "Check the reason codes of the authentication failure events. The reason code indicates the cause of the failure."

# Read-only. On the NPS server: the last 24 hours of NPS denials, grouped by reason code, policy, and EAP type
# (field labels from Microsoft's 6273 event template; the regex parsing is ours)
$since = (Get-Date).AddHours(-24)
$denied = Get-WinEvent -FilterHashtable @{ LogName = 'Security'; Id = 6273; StartTime = $since } -ErrorAction SilentlyContinue |
  ForEach-Object {
    $m = $_.Message
    [pscustomobject]@{
      Time          = $_.TimeCreated
      Account       = if ($m -match 'Account Name:\s+(.+)') { $Matches[1].Trim() }
      NetworkPolicy = if ($m -match 'Network Policy Name:\s+(.+)') { $Matches[1].Trim() }
      ProxyPolicy   = if ($m -match 'Proxy Policy Name:\s+(.+)') { $Matches[1].Trim() }
      AuthType      = if ($m -match 'Authentication Type:\s+(.+)') { $Matches[1].Trim() }
      EapType       = if ($m -match 'EAP Type:\s+(.+)') { $Matches[1].Trim() }
      RadiusClient  = if ($m -match 'Client Friendly Name:\s+(.+)') { $Matches[1].Trim() }
      ReasonCode    = if ($m -match 'Reason Code:\s+(\d+)') { [int]$Matches[1] }
    }
  }
$denied | Group-Object ReasonCode, NetworkPolicy, EapType -NoElement | Sort-Object Count -Descending | Format-Table -AutoSize

Account Name appears twice in the event (user and client machine); the first match is the user section (our reading of Microsoft's template). If one reason and one policy dominate, that's your chain. If the codes are spread out, take one user at a time.

The neighbors of 6273, each from Microsoft's audit page or NPS troubleshooting guidance:

Events 13 and 18 are in the System log (Microsoft's NPS troubleshooting guidance lists them as NPS events; the log is our reading). If neither 6273 nor 13 nor 18 appears while users fail, the request may not be reaching this server. Microsoft: "By default, NPS sends and receives RADIUS traffic by using User Datagram Protocol (UDP) ports 1812, 1813, 1645, and 1646."

# Read-only. On the NPS server: is NPS listening on the RADIUS ports, and which NPS servers does the NAS point at?
Get-NetUDPEndpoint -LocalPort 1812, 1813, 1645, 1646 -ErrorAction SilentlyContinue |
  Select-Object LocalAddress, LocalPort, OwningProcess
Get-NpsRadiusClient | Select-Object Name, Address, Enabled

Get-NpsRadiusClient "gets Remote Authentication Dial-In User Service (RADIUS) clients." Check that the access point, controller, switch, or VPN server sending the request is in the list with the address it actually uses (Microsoft's checklist item: "The RADIUS Clients and Servers list includes the radius client in question.").

Step 3: Read one 6273 end to end (read-only)

The reason code is one field out of more than twenty. Microsoft's 802.1X guide: "In the event message, scroll to the bottom, and then check the Reason Code field and the text that's associated with it." Read the rest of the event first, for one user who is failing right now:

# Read-only. On the NPS server: the newest 6273 for one account (the name is an example), full message
Get-WinEvent -FilterHashtable @{ LogName = 'Security'; Id = 6273 } -MaxEvents 500 |
  Where-Object { $_.Message -match 'jdoe' } |
  Select-Object -First 1 -Property TimeCreated, Message | Format-List

The fields, in the order Microsoft's event template lists them, and what each one decides:

Two things trip people up here. The reason text in current events can differ from Microsoft's 2008-era reason-code table: a 2024 Microsoft Q&A post quotes the reason 22 text as "The client could not be authenticated because the Extensible Authentication Protocol (EAP) Type cannot be processed by the server." Same code, same meaning. And a 6273 that names an unexpected network policy is a policy-order problem before it's a credential problem, whatever the code says (our reading).

Then the accounting log, which records every request and response, not only failures. Microsoft's Always On VPN guide: "NPS creates and stores the NPS accounting logs." Microsoft's accounting page: "The default location is the systemroot\System32\LogFiles folder." The file names start with IN (Microsoft's Always On VPN page shows the naming).

# Read-only. On the NPS server: the newest accounting log files, then every line for one account
$logs = Get-ChildItem -Path "$env:windir\System32\LogFiles" -Filter 'IN*' | Sort-Object LastWriteTime | Select-Object -Last 2
$logs | Select-Object Name, Length, LastWriteTime
$logs | Select-String -Pattern 'jdoe' | Select-Object -Last 20 | ForEach-Object { $_.Line }

Microsoft's Always On VPN page lists the database-format fields, including Packet-Type, User-Name, Calling-Station-ID, NAS-IP-Address, Client-Friendly-Name, Authentication-Type, Policy-Name, and Reason-Code. If your server writes the DTS Compliant (XML) format, the same names appear as elements (our reading of Microsoft's accounting page). Microsoft on the log: "The NPS logs can help you diagnose policy-related issues." A request with no response line at all, or nothing in the log for that user, sends you back to Step 2 (our reading).

Step 4: For reason 22, did the client and the policy agree on an EAP type? (read-only)

Reason 22 means the EAP conversation never got as far as the account. Three things decide it: which method the client offers, which methods the matching policy allows, and whether the NPS server can present a usable certificate for that method (our reading of Microsoft's reason-code and certificate pages).

The policy side. Microsoft's checklist item: "The network policy constraints list the correct authentication method." In the NPS console that's the policy's Constraints tab, Authentication Methods, EAP Types. A Microsoft-moderated answer on a 2018 reason 22 thread (community): "Please make sure you have selected PEAP-MSCHAPv2; not MSCHAPv2 as a supported authentication method in your NPS policy." Microsoft's Always On VPN page describes the same mismatch from the client side, for error 812: "You typically encounter this error when the NPS specified an authentication condition the client can't meet."

The client side. On a failing Windows client, the WLAN or Wired AutoConfig log shows the method and the reason Windows gave up. Microsoft: "On the client side, go to Event Viewer (Local)\Applications and Services Logs\Microsoft\Windows\WLAN-AutoConfig/Operational for wireless issues."

# Read-only. On a failing Windows client: the newest WLAN AutoConfig events (wired: Microsoft-Windows-Wired-AutoConfig/Operational)
Get-WinEvent -LogName 'Microsoft-Windows-WLAN-AutoConfig/Operational' -MaxEvents 30 |
  Select-Object TimeCreated, Id, LevelDisplayName, Message | Format-List

Read the EAP method the client used and compare it with the 6273's Authentication Type and EAP Type and the policy's list. PEAP on the client with only "Smart card or other certificate" in the policy, or EAP-TLS on the client with only PEAP in the policy, is a reason 22 by design (our reading). Microsoft also warns not to offer the same inner method both tunneled and untunneled: "When you deploy both a tunneled EAP method and EAP (which isn't protected), don't use the same authentication type."

The NPS server certificate. PEAP-MS-CHAP v2 needs one too. Microsoft: "With PEAP-MS-CHAP v2, PEAP-TLS, or EAP-TLS as the authentication method, the NPS must use a server certificate that meets the minimum server certificate requirements." And the part that breaks things on renewal day: "When the certificate applied to a policy expires, an administrator must manually update each policy that uses certificate-based authentication to reference a new, non-expired certificate. Until you update the policy, clients that rely on that policy can't successfully authenticate." Microsoft adds that this manual step "isn't required on NPS servers configured for certificate autoenrollment."

Microsoft publishes a read-only way to see which certificate a policy is bound to: export the configuration and pull the thumbprint from the policy's EAP blob. Microsoft's snippet, with the policy name as the only change:

# Read-only (writes one XML file). On the NPS server: which certificate thumbprint is bound to this network policy?
# Microsoft's snippet from "Manage certificates used with NPS"; the export contains RADIUS shared secrets in clear text, so delete it after reading
$PolicyName = "CORP-SECURE Wireless" # Change this to your policy name (case-insensitive)
$NPSConfigPath = "$env:TEMP\nps-config.xml"
Export-NpsConfiguration -Path $NPSConfigPath
$blob = (([xml](Get-Content "$env:TEMP\nps-config.xml" -Raw)).SelectNodes("//RadiusProfiles//*[@name]") | Where-Object { $_.name -ieq $PolicyName }).Properties.msEAPConfiguration.'#text'
$found = [regex]::Matches($blob, '14000000([0-9a-f]{40})') | ForEach-Object { $_.Groups[1].Value.ToUpper() } | Select-Object -Unique
if (-not $found) { "No certificate thumbprint found in policy '$PolicyName'" } elseif ($found.Count -eq 1) { "Bound certificate thumbprint:$found" } else { "WARNING: multiple candidate thumbprints found, manual review of $NPSConfigPath needed:`n$($found -join "`n")" }

Microsoft's caution on that file: "The exported NPS configuration file contains unencrypted shared secrets for RADIUS clients and members of remote RADIUS server groups." Then compare the thumbprint with what's actually in the machine store:

# Read-only. On the NPS server: certificates that can do server authentication, with expiry and names
Get-ChildItem -Path Cert:\LocalMachine\My -SSLServerAuthentication |
  Select-Object Thumbprint, Subject, NotAfter, @{ n = 'DnsNames'; e = { $_.DnsNameList -join ', ' } }, @{ n = 'EKU'; e = { $_.EnhancedKeyUsageList.FriendlyName -join ', ' } } |
  Format-List
# Anything expiring in the next 30 days
Get-ChildItem -Path Cert:\LocalMachine\My -ExpiringInDays 30 | Select-Object Thumbprint, Subject, NotAfter

The bound thumbprint missing from that list, or present with a past NotAfter, is your reason 22. Microsoft's server certificate requirements, each checkable in that output: "The Subject name contains a value."; the certificate includes "the Server Authentication purpose in EKU extensions (the object identifier (OID) for Server Authentication is 1.3.6.1.5.5.7.3.1)"; "The Subject Alternative Name (SubjectAltName) extension, if used, must contain the DNS name of the server." And it has to be trusted by the clients: it must "Be issued by a certification authority (CA) that is trusted by client computers." A new certificate from a different CA, or a client profile that pins the old server name or root, fails here even though the certificate itself is fine (our reading of Microsoft's client-side validation rules: "The name in the Subject line of the server certificate matches the name that's configured on the client for the connection.").

TLS settings. In the same 2018 thread, the poster found the cause in a hardening change: "I followed a document from microsoft increasing security by turning off SSL 2.0/3.0 and TLS 1.0/1.1 and only allowing TLS 1.2." That was Windows Server 2012 (community, old). On current builds, don't re-enable old protocols on that basis; read what changed under SCHANNEL and when, and compare with a server that works (our reading):

:: Read-only. On the NPS server: SCHANNEL protocol overrides (no output under Protocols = OS defaults)
reg query "HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols" /s

Clocks matter at this step too: reason 264 is "The Security Support Provider Interface (SSPI) called by EAP reports that the system clocks on the NPS server and the access client are not synchronized." See Windows Time Not Syncing With Domain Controller for that read.

Step 5: For reason 16 with passwords, is the account real, usable, and reachable? (read-only)

This step is for PEAP-MS-CHAP v2 (the event's EAP Type says so). Microsoft's NPS troubleshooting page lists three checks for 6273 reason 16: "Check that the username and password for the user are valid.", "Check to see whether the user account is locked in Active Directory.", and "Check that the request is targeted to the correct domain controller and that the user account exists."

# Read-only. Any domain-joined machine with RSAT: the account NPS was asked about (the name is an example)
Get-ADUser -Identity jdoe -Properties Enabled, LockedOut, PasswordExpired, PasswordLastSet, BadLogonCount, LastBadPasswordAttempt, AccountExpirationDate, UserPrincipalName |
  Select-Object SamAccountName, UserPrincipalName, Enabled, LockedOut, PasswordExpired, PasswordLastSet, BadLogonCount, LastBadPasswordAttempt, AccountExpirationDate

BadLogonCount and LastBadPasswordAttempt come from the DC that answered, so read them on the DC that NPS uses (our reading). Disabled, expired, and locked accounts usually get their own codes (34, 35, 36), so a reason 16 with a healthy-looking account points at the password the client sent or the name it sent it under (our reading of Microsoft's reason-code table).

The DC's side of the same attempt. MS-CHAP v2 is validated by the DC, so the DC's Security log should show the attempt as Event 4776 (our reading; Microsoft: "This event generates every time that a credential validation occurs using NTLM authentication." and "For domain accounts, the domain controller is authoritative.").

# Read-only. On a DC (or remotely with -ComputerName): recent 4776 entries for the account
Get-WinEvent -ComputerName DC01 -FilterHashtable @{ LogName = 'Security'; Id = 4776; StartTime = (Get-Date).AddHours(-4) } -MaxEvents 2000 |
  Where-Object { $_.Message -match 'jdoe' } |
  Select-Object -First 10 -Property TimeCreated, Message | Format-List

The Error Code field, from Microsoft's 4776 table:

No 4776 at all on any DC for that attempt means NPS didn't get as far as a DC (Step 8) or the name went somewhere else, a trusted domain or a different realm (our reading).

Which name was sent. The 6273's Account Name and Account Domain show what the client sent. A UPN suffix NPS can't resolve, a saved credential from before a rename, or a personal device sending only jdoe with no domain all look like reason 16 (our reading; Microsoft's codes 7 and 8 are the explicit versions: the domain in the User-Name attribute doesn't exist, or the user account doesn't). Microsoft's best-practices page has one more: "Network Policy Server (NPS) does not support the use of the Extended ASCII characters within passwords." A few users with accented characters in new passwords is a pattern worth checking.

Computer authentication. If the Account Name is host/LAPTOP-123 or CONTOSO\LAPTOP-123$, the computer is authenticating with its machine account (our reading). Read the computer object:

# Read-only. The computer account behind a host/ authentication (the name is an example)
Get-ADComputer -Identity 'LAPTOP-123' -Properties Enabled, PasswordLastSet, whenCreated |
  Select-Object Name, Enabled, PasswordLastSet, whenCreated

A disabled or recently re-created computer object, or one restored from an old image, can fail where users on the same laptop succeed (our reading). AD Account Keeps Getting Locked Out covers the reverse case: NPS as the caller that keeps locking an account with a saved old password.

Windows 11 and Credential Guard. Microsoft: "Upgrades to Windows 11, version 22H2, and Windows Server 2025 have Credential Guard enabled by default unless explicitly disabled." And for Wi-Fi and VPN: "Devices that use 802.1x wireless or wired network, RDP, or VPN connections that rely on insecure protocols with password-based authentication are unable to use SSO to sign in and are forced to manually reauthenticate in every new Windows session when Credential Guard is running." Users who used to connect silently with "Automatically use my Windows logon name and password" now get a prompt, type something, and fail (our reading). Microsoft's check on the client:

# Read-only. On a failing Windows client: 1 in the output = Credential Guard is running
(Get-CimInstance -ClassName Win32_DeviceGuard -Namespace root\Microsoft\Windows\DeviceGuard).SecurityServicesRunning

Microsoft's direction is clear: "Microsoft recommends moving away from MSCHAPv2-based connections (for example, PEAP-MSCHAPv2 and EAP-MSCHAPv2), to certificate-based authentication (for example, PEAP-TLS or EAP-TLS). Credential Guard doesn't block certificate-based authentication."

A pattern from the field (community). On a Microsoft Q&A thread about intermittent 6273 reason 16 for wired and Wi-Fi users, one admin traced it to a read-only DC in the NPS server's site: "Appears the root cause of this problem is related to NPS trying to authenticate to an RODC in the same site." Their fix was moving the RODC to its own site. If reason 16 comes and goes for healthy accounts, check which DCs share the NPS server's site (our reading of that thread; not Microsoft documentation).

Step 6: For reason 16 with certificates, does the certificate map to the account? (read-only)

This step is for EAP-TLS or PEAP-TLS. Microsoft's 6273 page: "If a certificate-based authentication method is used, examine the user or computer certificate to confirm that the user is providing the correct certificate for authentication." Reason 16's text says the user name "does not match an existing user account". With a certificate, the name comes from the certificate, so that's a mapping question (our reading).

The certificate itself. Microsoft's client certificate requirements: "The client certificate is issued by an enterprise CA or mapped to a user or computer account in Active Directory Domain Services (AD DS)."; it carries "the Client Authentication purpose in EKU extensions (the OID for Client Authentication is 1.3.6.1.5.5.7.3.2)"; "For user certificates, the Subject Alternative Name (SubjectAltName) extension in the certificate contains the user principal name (UPN)."; and "For computer certificates, the Subject Alternative Name (SubjectAltName) extension in the certificate must contain the fully qualified domain name (FQDN) of the client, which is also called the DNS name."

# Read-only. On a failing Windows client: client-auth certificates in the user and computer stores,
# with expiry, names, and whether the AD CS SID extension (1.3.6.1.4.1.311.25.2) is present
foreach ($store in 'Cert:\CurrentUser\My', 'Cert:\LocalMachine\My') {
  Get-ChildItem -Path $store |
    Where-Object { $_.EnhancedKeyUsageList.ObjectId -contains '1.3.6.1.5.5.7.3.2' } |
    Select-Object @{ n = 'Store'; e = { $store } }, Thumbprint, Subject, Issuer, NotBefore, NotAfter,
      @{ n = 'SAN'; e = { ($_.Extensions | Where-Object { $_.Oid.Value -eq '2.5.29.17' }).Format($false) } },
      @{ n = 'SidExtension'; e = { [bool]($_.Extensions | Where-Object { $_.Oid.Value -eq '1.3.6.1.4.1.311.25.2' }) } }
}

The store filter, Format($false) and the calculated properties are our forms. More than one client-auth certificate in a store is worth noting: Richard Hicks (Microsoft MVP, his Always On VPN blog, community) answers a reader that if an older certificate that isn't strongly mapped is still in the store, "Windows could be selecting the wrong one when it connects."