On this blog, and in several other places, I’ve shared my experiences with Azure Multi-Factor Authentication. In the early days of Azure MFA, a lot of organizations, a lot of client applications and a lot of 3rd party services were not able to perform multi-factor authentication. For these situations, Microsoft provided the App Passwords functionality.

In IT, temporary solutions tend to become the most permanent solutions. As you’d guess, most of the people, devices and services that were initially configured with an app password, and not with true multi-factor authentication, still use these app passwords.

The question now is:

Who still uses App Passwords, and where?

About App Passwords

If you have multi-factor authentication turned on and a less secure client application or device isn't prompting you for multi-factor authentication, you may be able to sign in with an app password instead.

An app password is a long, randomly generated password that you provide only once instead of your regular password when signing in to an app or device that doesn't support multi-factor authentication.

Getting to know the devices and people

To answer the question on who (or what service account) still uses App Passwords, I look at the Sign-in logs for Azure AD in Azure Log Analytics. The below three steps provide a means to get the information on App Password usage.

STEP 1: SET UP A LOG ANALYTICS WORKSPACE

As the first step, set up a Log Analytics Workspace. As Azure subscriptions, by default, do not get configured with a Log Analytics workspace, the first step is to create a Log Analytics Workspace.

If you already have an Azure Log Analytics workspace, you can skip this step.

Perform these steps:

• Sign into the Azure Portal with an account that has one or more of the roles mentioned in the above requirements paragraph.
• In the Azure portal, click All services. In the list of resources, type Log Analytics. As you begin typing, the list filters based on your input. Select Log Analytics workspaces from the list.
• Click + Add.
  The Log Analytics workspace blade appears.
• Fill in the required information to add a Log Analytics workspace.
• Click OK on the bottom of the blade to create the Log Analytics workspace.

STEP 2: INTEGRATE AZURE AD Sign-in LOGS INTO LOG ANALYTICS

If you’ve already integrated your Azure AD sign-in logs with an Azure Log Analytics workspace, you can skip this step.

Perform the following steps to route audit activity logs and sign-in activity logs from Azure Active Directory to the Log Analytics Workspace:

• While still logged on in the Azure AD Portal, click on Azure Active Directory in the left navigation menu.
• Select Diagnostic settings in Azure AD’s navigation menu.
• In the main pane, click Add diagnostic setting.
  The Diagnostic settings blade appears.
• On the Diagnostic settings blade, provide a name for the diagnostic settings.
• Select the Send to Log Analytics workspace check box.
• Select the Log Analytics workspace you want to send the logs to, or create a new workspace in the provided dialog box.
• To send sign-in logs to the Log Analytics workspace, select the SignInLogs check box.
• Select Save on top of the blade to save the diagnostic settings.

Allow for ample time for the diagnostic settings to apply and the data to be streamed to the Log Analytics workspace.

Step 3: Get information on App Password Usage

Now, let’s get the information out of the Azure Log Analytics workspace to report on the people still using App Passwords and the information that is available on the devices they use these App Passwords on:

• While still logged on in the Azure AD Portal, search for the Azure Log Analytics workspace where you stream the sign-in logs to.  Click on the name of the workspace from the search results.
• In the left navigation menu of the Log Analytics workspace, click Logs.
• Close the Example queries dialog window, if it appears.
• In the main pane, on the New Query 1 tab, in the field underneath the blue Run button, type the following query:

SigninLogs
| where AuthenticationDetails contains "MFA requirement skipped due to app password"
| distinct Identity, tostring(split(UserAgent, "/", 0))
| sort by Identity asc

• Select an appropriate time range in the Time range : field next to the Run button.
• Run the query by pressing the blue Run button. The names of the people and their device types appear in the Results field underneath.

Tip!
You might be tempted to pick a large time range, but what you’ll notice is that this makes little sense. Your results may include devices that are no longer in use, devices that have been upgraded to a newer version or configuration and services that have been converted to use API access since the beginning of the time frame.

Concluding

App Passwords are remnants of old ways of dealing with multi-factor authentication. While you might still encounter some client applications, devices and services that are not capable of using multi-factor authentication, these distinct exceptions can be excluded from multi-factor authentication, if the organization’s security measures allow for it.

At least now, you have a list of exceptions to work with and minimalize.

Further reading

App Passwords are only available with a non-Conditional Access MFA requirement
HOWTO: Set an alert to notify when an Azure AD emergency access account is used
HOWTO: Set an alert to notify when an additional person is assigned the Azure AD Global Administrator role