In the first part of this series, we discussed why we want to virtualize Domain Controllers. The first question people ask is:
How do I properly size Domain Controllers on my virtualization platform?
Specifically, for VMware vSphere, this is a good question, because there are a couple of areas of attention, beyond the recommended practices from Microsoft:
Microsoft recommended practices
For sizing Domain Controllers, Microsoft recommends to:
- Deploy at least two Domain Controllers per Active Directory domain.
- Create exceptions for antimalware solutions for the folders containing Active Directory files.
- Deploy the Local Administrator Password Solution (LAPS).
- Do not install additional software or Server Roles on Domain Controllers.
- Install Windows updates on Domain Controllers.
- Keep information security measures on Domain Controllers, like antimalware, backup, restore, monitoring, auditing, bad password blocking and SIEM solutions, up to date.
- Have a recovery plan available for Active Directory.
Areas of Attention
A good rule of thumb for the number of virtual Central Processing Units (vCPUs) for is to size virtual Domain Controllers with 1 vCPU, when the environment has 10,000 users, or less. When the environment has more than 10,000 users, add another vCPU to the Domain Controllers.
When in doubt, start with 2 vCPUs in virtual Domain Controllers and add vCPUs as needed. The Domain Controller holding the Primary Domain Controller emulator (PDCe) Flexible Single Master Operations (FSMO) role will be the most burdened Domain Controller of all. It performs these additional tasks, when compared to all the other Domain Controllers in the Active Directory domain:
- Password changes performed by other Domain Controllers in the Active Directory domain are replicated preferentially to the PDC emulator.
- If a logon authentication fails at a given Domain Controller in an Active Directory domain due to a bad password, the Domain Controller will forward the authentication request to the PDC emulator to validate the request against the most current password. If the PDC reports an invalid password to the Domain Controller, the Domain Controller will send back a bad password failure message to the user.
- Account lockout is processed on the PDC emulator.
- The Domain Controller with the PDC emulator FSMO role, by default, functions as the authoritative source of time in the Active Directory domain.
- The Domain Controller with the PDC emulator FSMO role fulfills the role of the PDC in the NetLogon Remote Protocol methods. Therefore, the Domain Controller with the PDC emulator FSMO role must support and perform all PDC specific functionality specified in that section. Every other Domain Controller must not perform this functionality.
In VMware vSphere-based VMs with more than one vCPU, make sure to look at the Networking section below to avoid the unavailability of receive-side scaling.
If you want highly-performing Domain Controllers, provide them with a sufficient amount of Random Access Memory (RAM) to be able to cache the Active Directory database (ntds.dit).
A good metric to monitor is the Database/Database Cache %Hit counter for the LSASS process on Domain Controllers. A low hit rate indicates that the Domain Controller would benefit from more RAM.
Microsoft recommends to use a 40GB system volume (C:\) to store the Windows Operating System. However, Active Directory requires additional storage. You may or may not place these files on the system volume, depending on your view on dynamic files. Independent of this choice, you need to take into account the following storage needs for a Domain Controller:
|Active Directory role||250 MB|
|Active Directory database||4 KB per object (excluding photos)|
|Active Directory logs||22MB|
|Active Directory System Volume||Any files you store in the SYSVOL share|
|Active Directory System Volume||2MB per Group Policy object|
|Active Directory System Volume||20MB for the Central Policy Store|
|Active Directory System Volume||1GB for the SYSVOL replication staging area|
For a typical organization of 100,000 persons, this would lead to 5GB of additional storage requirements. Of course, when such an organization would decide to store user photos in Active Directory, storage requirements could potentially triple.
Ideally, configure virtual Domain Controllers with one virtual Network Interface Card (vNIC). Use VMXNET3 for best performance.
Additionally, upgrade the VMware Tools of existing virtual machines to version 10.2.5. The Windows Receive Side Scaling (RSS) feature is not functional on virtual machines running VMware Tools versions 9.10.0 up to 10.1.5. On virtual Domain Controllers with multiple vNICs, under heavy network load, this may cause a situation where CPU0 is overloaded, as depicted in the below screenshot:
Since VMXNET3 driver version 22.214.171.124 (as part of VMware Tools 10.2.5), this driver version enables Receive-side Scaling (RSS) and Receive Throttle settings, by default – but only for new VMware Tools installations on new virtual machines. If you upgrade an existing VMware Tools install, these settings will remain as is.
On existing virtual Domain Controllers, make sure to enable Receive Side Scaling (RSS) and set the Receive Throttle to 30, using the following line of Windows PowerShell on virtual Domain Controllers running VMware Tools 10.2.5, or up:
Enable-NetAdapterrss –Name "*"
Agents and add-ons
On top of the above requirements for the Active Directory role, make sure you provide sufficient resources for the typical agents and add-ons your organization would typically install on Domain Controllers, like antimalware, backup, restore, monitoring, auditing, bad password blocking and SIEM solutions.
Because of this last area of attention, we see organizations typically deploy virtual Domain Controllers with 1 vCPU, 4GB RAM, a 60GB system volume and 1 vNIC.
These dimension are a far cry from the other sizing we often encounter in organization where, according to the design all hardware factors are blades and a physical Domain Controller was deemed necessary. The typical dimension of these Domain Controller blades depend on the phase in the project where its importance was understood. The largest Domain Controller we’ve come across was a blade with 2 CPUs, 24 cores, 512GB RAM and 2 300GB hard disks in RAID1.
Sizing is often the first hurdle to cross when virtualizing Domain Controllers. Join me for the next parts were we drill deeper into the integrity, confidentiality and availability of Domain Controllers.
Just out of curiosity, what was the largest physical Domain Controller you’ve ever come across in production?