Because Active Directory Federation Services (AD FS) rely heavily on certificates, you’ll want the most straightforward SSL/TLS certificate as the Service Communications Certificate throughout your Active Directory Federation Services (AD FS) implementation.

Notice however, that I’m not recommending to use the strongest certificates for your Active Directory Federation Services (AD FS) implementation? You won’t hear me recommending to use certificates with the longest key size available, the most impressive hashing algorithm or even the most advanced private key generation…

There’s a couple of reasons for that, and, in this series, I’ll try to give you a couple of best practices on the certificates to use as the Active Directory Federation Services (AD FS) Service Communication Certificate.

In this part I’ll discuss key size.
(also referred to as key length)

In the first part of this series, we’ve come to the conclusion that SHA-256 is the ideal hashing algorithm for the Service Communications Certificate throughout your Active Directory Federation Services (AD FS) implementation.

These kind of certificates can be requested with a couple of key sizes:

• 512 bits
• 1024 bits (default)
• 2048 bits
• 4096 bits
• 8192 bits
• 16384 bits

Minimum key size calculations can be done on keylength.com. The sizes provided there are designed to resist mathematic attacks. As the key size increases, so does the complexity of brute forcing to the point where it becomes impracticable to crack the encryption directly.

Note:
They do not take algorithmic attacks, hardware flaws, etc. into account.

The current consensus is to pick 2048 bits as the key size.

Why Going smaller is asking for problems

When you use a key size smaller than the recommended values, you might risk being targeted for attacks, because the rest of the industry has moved on to SSL/TLS certificates with 2048bit key size (and up).

Going smaller than 512 bits for the key size even results in the Service Communications Certificate throughout your Active Directory Federation Services (AD FS) implementation being marked as insecure on systems installed with KB2661254 (August 2012), regardless of the other validity options.

Why going larger doesn’t make much sense, too

Going with a larger key size poses its own set of challenges.

Certificates with 16384bit key size (the maximum you may specify when requesting a certificate in Windows) can be used, but may be a recipe to Denial of Service (DoS) towards your Active Directory Federation Services (AD FS) implementation.

Upon the first contact with your Active Directory Federation Services (AD FS) implementation, this will trigger an encryption process that is ‘expensive’ in cert speak. This means it will require a lot of CPU. The longer the key size, the more CPU this requires.

Concluding

When requesting a certificate for the Service Communications Certificate throughout your Active Directory Federation Services (AD FS) implementation, opt for a certificate with a 2048 bit key size.

Further reading

Recommendations for PKI Key Lengths and Validity Periods with Configuration Manager
Selecting a Key Size for Your Root Certificate Server in Windows Server 2012 AD CS
RSA keys under 1024 bits are blocked
Blocking RSA Keys less than 1024 bits (part 2)
Blocking RSA keys less than 1024 bits (part 3)