Some time ago one of my friends at MCS had come across a problem while promoting RODC in a multi domain forest. This problem was "access denied" error when new RODC was promoted in one of child domains. This problem was solved thanks to discussion and involvement of few different individuals and I had a pleasure also to involved in this discussion. While this problem was then also reported by few other people I took some time to reproduce it in my mobile lab (thank's to some magic 🙂 my machine was upgraded to 4GB so I can run VMs and write it at the same time 🙂 ) and present here both problem and its resolution.
I thin we will get KB describing this issue but in the mean time lets Google and Live consume this blog entry and bring this information to those who are seeking for solution :).
Enjoy ….
Symptoms
When promoting Read-Only Domain Controller (RODC) in a domain promotion process fails during directory replication data process. When examining DCPROMO.LOG following entry can be found:
<date and time> [INFO] Error – Active Directory Domain Services could not replicate the directory partition <Domain DN> from the remote Active Directory Domain Controller <DC FQDN>. (8453)
<date and time> [INFO] EVENTLOG (Error): NTDS General / Internal Processing : 1168
Internal error: An Active Directory Domain Services error has occurred.Additional Data
Error value (decimal):
-1073741790Error value (hex):
c0000022Internal ID:
30014c7
In addition event log entry with Event ID 1168 is being logged in Directory Services event log which holds following information:
Log Name: Directory Service
Source: Microsoft-Windows-ActiveDirectory_DomainService
Date: <date and time>
Event ID: 1168
Task Category: Internal Processing
Level: Error
Keywords: Classic
User: ANONYMOUS LOGON
Computer: <DC name>
Description:
Internal error: An Active Directory Domain Services error has occurred.
Additional Data
Error value (decimal):
-1073741790
Error value (hex):
c0000022
Internal ID:
30014c7
Error code c00000022 is translated to Access Denied error message:
C:\>err c0000022
# for hex 0xc0000022 / decimal -1073741790
STATUS_ACCESS_DENIED ntstatus.h
# {Access Denied}
# A process has requested access to an object, but has not
# been granted those access rights.
Cause
During the adprep /rodcprep portion of domain preparation set of ACE entries is being added to NC head of domain in which this process was executed. List of these ACEs can be found in following TechNET article: Read-Only Domain Controller Updates. Two of these new One of entries being added is grant of "Replicate Directory Changes" right for "Enterprise Read-Only Domain Controller" (ERODC) group . This group is being identified by well known SID with RID 498. Full SID for ERODC has following value:
<FOREST ROOT DOMAIN SID>-498
For forest root domain with SID S-1-5-21-329151704-1384884650-1385766050 it will have following value:
S-1-5-21-329151704-1384884650-1385766050-498
Note: We have to use forest root domain SID as ERODC is a universal group which exists in forest root domain.
ERODC group is reflected by well known SID because it is not created automatically during /rodcprep operation. This group is being created afterwards by one of two events:
- Transfer of PDC Emulator role to a Windows Server 2008 based DC in a forest root domain. After creation of ERODC group this role can be transferred back to its original holder. For information how to transfer PDC Emulator FSMO role to other DC please reefer to article KB 255504.
- Promotion of first RODC in a forest root domain.
Lack of this permission granted to ERDC group will prevent directory information to occur and will cause error described in Symptoms section.
Presence of this ACE entry can be checked using any tool which allows to view permissions on a directory services object. For example using adfind.exe (http://www.joeware.net/freetools/tools/adfind/) with following syntax:
adfind -b DC=W2k,DC=PL -s base ntsecuritydescriptor -sddl+
will result on a DC with proper permissions after /rodcprep in following result:
Using server: rootdc.w2k.pl:389
Directory: Windows Server 2003dn:DC=W2k,DC=PL
>nTSecurityDescriptor: [OWNER] BA
>nTSecurityDescriptor: [GROUP] BA
>nTSecurityDescriptor: [DACL] AI(…)
>nTSecurityDescriptor: [DACL] OA;;CR;Replicating Directory Changes;;S-1-5-21-329151704-1384884650-1385766050-498
(…)
>nTSecurityDescriptor: [SACL] OU;CISA;WP;gPOptions;organizationalUnit;WD
1 Objects returned
Note that SID of ERODC group was returned in this ACE as creation of the group didn't took place yet when this tool was executed. Similar information can be obtained using dsacls.exe (standard tool included Support Tool package or shipped with operating system in Windows Server 2008 ):
C:\>>dsacls DC=w2k,dc=pl
Owner: BUILTIN\Administrators
Group: BUILTIN\AdministratorsAccess list:
Allow Everyone SPECIAL ACCESS
READ PROPERTY(…)
Allow W2K\Enterprise Read-only Domain Controllers
Replicating Directory Changes(…)
The command completed successfully
Note that in this case, ERODC group was specified with a name, as this command was executed after this group was created.
Solution
To correct this situation proper ACE has to be added to security descriptor of NC head on which it is missing and is causing an error. This can be achieved in two ways:
- Using GUI ACLs editor in ADU&C or any other tool which allows directory permission management.
- Using command line dsacls.exe tool or any other CLI tool which allows to manage directory services permissions, using SID:
dsacls DC=w2k,dc=pl /G "S-1-5-21-329151704-1384884650-1385766050-498:CA;Replicating Directory Changes"
or ERODC group name
dsacls DC=w2k,dc=pl /G "W2k\Enterprise Read-only Domain Controllers:CA;Replicating Directory Changes"
Note: ability to use SID of a group may depend on the tool used to set this ACE on directory. In real environment change NC head DN and domain name \ SID to proper value.
After this modification RODC promotion process should be allowed to be finished.
In proper configuration there should be two ACEs for ERODC group present. One was described above, second is simply "List contents" and "Read all properties" on a NC Head applied to "This object only". In case this ACE is also missing it can be re-created using UI:
or from command line using for example dsacls.exe:
dsacls DC=w2k,dc=pl /G "S-1-5-21-329151704-1384884650-1385766050-498:LCRPLO;;"
So that is basically it regarding this particular issue.
Additional information
In case of multi domain forests such operation might be required to perform not only on a NC of a domain in which new RODC is being promoted but also on other NCs like forest root domain.
Credits
Credits for finding and providing solution to this problem goes to Matjaz Ladava from MCS Slovenia and some great guys at MCS and PSS who were involved in resolution of this problem (Herbert, Roberts and others – kudos to You).