In this post I will show you how to track Conditional Access Policy change using email alert as a solution. Conditional Access is one of the most important aspect of Azure where we can use to block / allow certain applications and other settings based upon conditions. However, we also want to make sure that no unauthorized changes are applied to Conditional access policies. Using Log Analytics alert feature we can get the email whenever any changes are applied in the environment.
This will help analysing who made the changes to Conditional Access policy.
Need of tracking Conditional Access Policy
In a big environment there might be multiple Conditional access administrator which might create, update and delete conditional access policies. It’s a good practice to act on these policies through Change Order or request as any unauthorized / wrong policies implemented can change the user behaviour and can have adverse impact in your environment related to cloud based apps such as Office 365 and other azure related components.
If we can get an email whenever someone is making a change, that will be awesome. We can monitor the changes made along with the date, name of the user.
Steps to perform for tracking Conditional Access policy
There are certain steps to be performed to start getting the alert in form of email which includes:
- Setting up Log Analytics workspace
- Enable AuditLogs – SiginLogs under Diagnostics Settings
- Create alert under Log Analytics
Setup Log Analytics workspace
This is the very first step to be done. The chances are, log Analytics has been already configured in your environment. If not, login to Azure Portal, search for Log Analytics workspaces.
Click on Create and specify the Instance details by specifying subscription, Resource group, Name and Region.
Allow some time to get the deployment created for Log Analytics workspace. You may check keep track who modified WUfB Intune policies for more details on how to create Log Analytics Workspace.
Enable AuditLogs and SiginLogs under Diagnostics Settings
Under Azure Active Directory, navigate to Diagnostic settings and click on Add diagnostic setting.
Under Diagnostic setting name, provide the name as “Signin Attempt” and provide Log\Categories as:
- AuditLogs
- SignInLogs
Select Destination details as:
Send to Log Analytics workspace along with subscription and Log Analytics workspace name selected.
Once done, you will be able to see the Diagnostic setting name.
Create Kusto Query Language to get details of Conditional Access Policy
Before creating an alert, let’s explore the KQL to get Conditional Access policy details.
Open Log Analytics Workspace page and select the workspace we created.
Under workspace, click on Logs which will load KQL Editor.
Run the following query:
// Query to get details of Conditional Access policy
AuditLogs
| where Category == "Policy"
and AdditionalDetails[0].value == "Conditional Access"
and (OperationName == "Add conditional access policy"
or OperationName == "Update conditional access policy"
or OperationName == "Delete conditional access policy")
| project ActivityDateTime, AdditionalDetails[0].value, ActivityDisplayName, TargetResources[0].displayName, InitiatedBy.user.userPrincipalName
We can see the results of all Conditional Access policy for Add, Update and Delete of Conditional Access policy.
I have provided the content for all three condition, you may modify / delete the lines based upon your requirement.
The query used was for the testing purpose to see the results we will get. We are ready to create alert now under Log Analytics workspace.
Create alert for Conditional Access Policy
Under Log Analytics workspace, scroll down and click on Alerts.
Under Right Pane, click on Create > Alert rule
Under Select a signal blade, click on Custom log search.
Under Log query > Search query, provide the query as (the same one which I used previously):
// Query to get details of Conditional Access policy
AuditLogs
| where Category == "Policy"
and AdditionalDetails[0].value == "Conditional Access"
and (OperationName == "Add conditional access policy"
or OperationName == "Update conditional access policy"
or OperationName == "Delete conditional access policy")
| project ActivityDateTime, AdditionalDetails[0].value, ActivityDisplayName, TargetResources[0].displayName, InitiatedBy.user.userPrincipalName
Scroll down further to specify:
Measurement > Aggregation granularity: 5 minutes
(The interval over which datapoints are grouped by the aggregation type)
Specify Alert logic:
Operator: Greater than
Threshold value: 0
Frequency of Evaluation: 5 minutes
(Choose how often the alert rule should run)
Click on Next: Actions
Under Actions, click on Create action group.
Under Project details, specify Action group name as “CAPolicy” and click Next.
Under Notifications, specify:
- Notification type: Email/SMS message/Push/Voice
- Name: TrackCAPolicy
- Email: specify email id where you wanted to send the alert
Click Next for Action and Tags as we are not going to specify anything.
Click on Review + Create
Under Create an action group, again click on Create.
The action group is created now. Click on Next: Details to continue with further settings.
Under Details, specify:
Severity: 2 – Warning
Alert rule name: Track CA Policy
Click on Review + create.
Verify the settings, and click on Create to setup the alert.
Alert has been created, now if anyone create, update or delete the conditional access policy. This will be alerted within 5 minutes( though this setting is aggressive and you need to make the adjustment as per your need) and you will receive email notification.
Verify Alert through Email
As I made few changes with Conditional Access policy, alert was expected.
I received the alert with Alert name, Severity and Description
Upon clicking the Filtered search results, it took me to Log Analytics workspace KQL editor window and ran the command which showed me few changes made in last 5 minutes such as:
- Delete conditional Access policy
- Add conditional access policy
The alerts can be monitored through Log Analytics > Alerts blade as well, which will show you how many alerts have been and you can take the action by clicking on Change user response once you verify the alert.
You can then response to the alert with Acknowledge or Closed based upon the actions you will take.
Conclusion
I hope this post will help you creating your own policies for not only tracking the Conditional access policy but can be used for few other kind of tracking if you thoroughly understand what inside is in various tables and how to explore it using Kusto Query language (KQL).
