In this post I will show you how to track Conditional Access Policy change using email alert as a solution. Conditional Access is one of the most important aspect of Azure where we can use to block / allow certain applications and other settings based upon conditions. However, we also want to make sure that no unauthorized changes are applied to Conditional access policies. Using Log Analytics alert feature we can get the email whenever any changes are applied in the environment.

This will help analysing who made the changes to Conditional Access policy.

Need of tracking Conditional Access Policy

In a big environment there might be multiple Conditional access administrator which might create, update and delete conditional access policies. It’s a good practice to act on these policies through Change Order or request as any unauthorized / wrong policies implemented can change the user behaviour and can have adverse impact in your environment related to cloud based apps such as Office 365 and other azure related components.

If we can get an email whenever someone is making a change, that will be awesome. We can monitor the changes made along with the date, name of the user.

Steps to perform for tracking Conditional Access policy

There are certain steps to be performed to start getting the alert in form of email which includes:

  • Setting up Log Analytics workspace
  • Enable AuditLogs – SiginLogs under Diagnostics Settings
  • Create alert under Log Analytics

Setup Log Analytics workspace

This is the very first step to be done. The chances are, log Analytics has been already configured in your environment. If not, login to Azure Portal, search for Log Analytics workspaces.

Log Analytics workspaces

Click on Create and specify the Instance details by specifying subscription, Resource group, Name and Region.

Create Log Analytics workspace

Allow some time to get the deployment created for Log Analytics workspace. You may check keep track who modified WUfB Intune policies for more details on how to create Log Analytics Workspace.

Enable AuditLogs and SiginLogs under Diagnostics Settings

Under Azure Active Directory, navigate to Diagnostic settings and click on Add diagnostic setting.

Add diagnostic setting

Under Diagnostic setting name, provide the name as “Signin Attempt” and provide Log\Categories as:

  • AuditLogs
  • SignInLogs

Select Destination details as:

Send to Log Analytics workspace along with subscription and Log Analytics workspace name selected.

Add diagnostic setting

Once done, you will be able to see the Diagnostic setting name.

TrackCAPolicy 05

Create Kusto Query Language to get details of Conditional Access Policy

Before creating an alert, let’s explore the KQL to get Conditional Access policy details.

Open Log Analytics Workspace page and select the workspace we created.

Log Analytics

Under workspace, click on Logs which will load KQL Editor.

Run the following query:

// Query to get details of Conditional Access policy
AuditLogs
| where Category == "Policy"
and AdditionalDetails[0].value == "Conditional Access"
and (OperationName == "Add conditional access policy"
or OperationName == "Update conditional access policy"
or OperationName == "Delete conditional access policy")
| project ActivityDateTime, AdditionalDetails[0].value, ActivityDisplayName, TargetResources[0].displayName, InitiatedBy.user.userPrincipalName
KQL query to track Conditional Access policy

We can see the results of all Conditional Access policy for Add, Update and Delete of Conditional Access policy.

I have provided the content for all three condition, you may modify / delete the lines based upon your requirement.

The query used was for the testing purpose to see the results we will get. We are ready to create alert now under Log Analytics workspace.

Create alert for Conditional Access Policy

Under Log Analytics workspace, scroll down and click on Alerts.

Under Right Pane, click on Create > Alert rule

Create Alert Log Analytics

Under Select a signal blade, click on Custom log search.

custom log search

Under Log query > Search query, provide the query as (the same one which I used previously):

// Query to get details of Conditional Access policy
AuditLogs
| where Category == "Policy"
and AdditionalDetails[0].value == "Conditional Access"
and (OperationName == "Add conditional access policy"
or OperationName == "Update conditional access policy"
or OperationName == "Delete conditional access policy")
| project ActivityDateTime, AdditionalDetails[0].value, ActivityDisplayName, TargetResources[0].displayName, InitiatedBy.user.userPrincipalName
Create alert rule log query

Scroll down further to specify:
Measurement > Aggregation granularity: 5 minutes
(The interval over which datapoints are grouped by the aggregation type)

Specify Alert logic:

Operator: Greater than
Threshold value: 0
Frequency of Evaluation: 5 minutes
(Choose how often the alert rule should run)

TrackCAPolicy 12

Click on Next: Actions

Under Actions, click on Create action group.

Create action group

Under Project details, specify Action group name as “CAPolicy” and click Next.

TrackCAPolicy 14

Under Notifications, specify:

  • Notification type: Email/SMS message/Push/Voice
  • Name: TrackCAPolicy
  • Email: specify email id where you wanted to send the alert
TrackCAPolicy 15

Click Next for Action and Tags as we are not going to specify anything.

Click on Review + Create

Under Create an action group, again click on Create.

TrackCAPolicy 16

The action group is created now. Click on Next: Details to continue with further settings.

TrackCAPolicy 17

Under Details, specify:

Severity: 2 – Warning

Alert rule name: Track CA Policy

TrackCAPolicy 18

Click on Review + create.

Verify the settings, and click on Create to setup the alert.

TrackCAPolicy 19

Alert has been created, now if anyone create, update or delete the conditional access policy. This will be alerted within 5 minutes( though this setting is aggressive and you need to make the adjustment as per your need) and you will receive email notification.

Verify Alert through Email

As I made few changes with Conditional Access policy, alert was expected.

I received the alert with Alert name, Severity and Description

Conditional Access Alert Email

Upon clicking the Filtered search results, it took me to Log Analytics workspace KQL editor window and ran the command which showed me few changes made in last 5 minutes such as:

  • Delete conditional Access policy
  • Add conditional access policy

The alerts can be monitored through Log Analytics > Alerts blade as well, which will show you how many alerts have been and you can take the action by clicking on Change user response once you verify the alert.

Track alert

You can then response to the alert with Acknowledge or Closed based upon the actions you will take.

Track alert Change user response

Conclusion

I hope this post will help you creating your own policies for not only tracking the Conditional access policy but can be used for few other kind of tracking if you thoroughly understand what inside is in various tables and how to explore it using Kusto Query language (KQL).