Errors When Accessing Sentinel

[harperaa]

Describe the bug
I am following the tutorial in the following blog.
https://medium.com/@iknowjason/sentinel-for-purple-teaming-183b7df7a2f4

To Reproduce
Steps to reproduce the behavior:
When I open sentinel, it does not appear to be linked to the LAW. I get this error in the dashboard overview.

Under incidents (all the panels have similar error)
Error occurred while fetching the data
{"error":{"code":"BadRequest","message":"{"error":{"code":"BadRequest","message":"Workspace 'pc-non9r' is not onboarded to Microsoft Sentinel. Please onboard through the portal (https://learn.microsoft.com/en-us/azure/sentinel/quickstart-onboard) or use the OnboardingStates ARM api to onboard to Sentinel (https://learn.microsoft.com/en-us/rest/api/securityinsights/sentinel-onboarding-states/create?view=rest-securityinsights-2024-03-01).\"}}"}}

Expected behavior
I was expecting to see log data in sentinel, it is in LAW.

Screenshots

Desktop (please complete the following information):

• OS: mac os, m1
• Browser: brave
• Version: 1.76.82

Additional context
I ran the terraform scripts and they completed without error, I also see log data in LAW. Is there some step missing, to link the LAW and Sentinel?

[iknowjason]

Hey there @harperaa thanks for reporting this in!

Good and bad news here. The good news is there is a fix and I'm going to share it just below. The bad news is unfortunately this is the way cloud providers like Azure are. They make some kind of change to their backend or frontend that impacts stuff like this. I stumbled across your exact issue a few weeks ago. Major pain in the ass as I had to spend a lot of time debugging why it was going on. Further, I haven't had a chance to update the documentation.

The other bad piece of news is now this lab is not 100% automated just with terraform. You have to run an extra step. What a pain.

Now to the good news. Microsoft now requires you to add an "Onboarding State" to Sentinel. When you create Sentinel manually in the Azure Portal, this happens naturally when you hit an "Ok" button. But somehow when you use terraform, their frontend doesn't show the same button to onboard Sentinel for the first time.

So you have to "Onboard Sentinel" using the Azure REST API. Here are the steps:

1. Login to Azure portal with your credentials
2. Click on this link for Sentinel Onboarding API: https://learn.microsoft.com/en-us/rest/api/securityinsights/sentinel-onboarding-states?view=rest-securityinsights-2024-09-01
3. Click on the "Create" link
4. Click on the green "Try it" button
5. Click on Continue with Account (if prompted)
6. You should see an example REST API request. You will need to fill in all of the values that are shown blank. These values are unique to your environment. You can either browse the Azure portal or use the terraform output command in your bash shell that just ran terraform.
7. Fill in your law and resource group name. For sentinelOnboardingStateName, enter default
8. This is what mine looks like:

9. Scroll down to body and copy and paste the following json into the body:

{
  "properties": {
    "customerManagedKey": false
  }
}

10. Hit Run at the bottom
11. You should see a response code with 201. You have now onboarded Sentinel, allowing new analytics rules to be created.
12. Go back into your Sentinel and all of the errors should be clear.

Please test this out and report it back to me if you don't mind. I do need to update my blog and instructions on the PurpleCloud website for sentinel generator.

What a pain, no?

[iknowjason]

Apparently there is a new terraform resource for this new Senintel onboarding state. I will research getting this added so that everything will be automated. Thanks for your patience.

[harperaa]

Thanks. Another question, what happened to prior config with ELK, velociraptor, etc... I don't see that in the generators any more... Btw, I am lead author of gray hat hacking and used purpleCloud in book, and am glad to see you update it. I am happy to send PR, if I extend it.

…

On Fri, Apr 4, 2025 at 9:00 AM Jason Ostrom ***@***.***> wrote: Apparently there is a new terraform resource for this new Senintel onboarding state. I will research getting this added so that everything will be automated. Thanks for your patience. — Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKE6AFYUY3K5RPNZIBOWBD2XZ65ZAVCNFSM6AAAAAB2NTGU5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONZYGY3DSNJXGY> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: iknowjason]*iknowjason* left a comment (iknowjason/PurpleCloud#42) <#42 (comment)> Apparently there is a new terraform resource for this new Senintel onboarding state. I will research getting this added so that everything will be automated. Thanks for your patience. — Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKE6AFYUY3K5RPNZIBOWBD2XZ65ZAVCNFSM6AAAAAB2NTGU5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONZYGY3DSNJXGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[iknowjason]

@harperaa If you want to submit a PR for the new terraform for the Sentinel Onboarding, go for it. I would be happy to add you as a contributor. That would make it more automated and we wouldn't have to add manual steps for the Azure REST API. Let me know.

That's nice to know about it being in Gray Hat! Thanks for including and recommending PurpleCloud.

[iknowjason]

@harperaa The Velociraptor + Elastic is still in PurpleCloud under the ad.py generator. See here:
https://www.purplecloud.network/tools/ad/

Check out example number 4.

[iknowjason]

@harperaa Looks like this is the terraform resource:

https://learn.microsoft.com/en-us/azure/templates/microsoft.securityinsights/2024-03-01/onboardingstates?pivots=deployment-language-terraform

[harperaa]

Thanks so much for your efforts, I will look into the above soon and report back! Thanks Jason!

[harperaa]

I have my own good news and bad news... ha.

I was able to adjust the sentinel.py code to add the functionality to onboard sentinel, by adding the following code at line 1390:

# Onboard the Log Analytics Workspace to Sentinel
resource "azurerm_sentinel_log_analytics_workspace_onboarding" "pc" {
  workspace_id = azurerm_log_analytics_workspace.pc.id
  
  # Ensure the Log Analytics Solution is created first
  depends_on = [azurerm_log_analytics_solution.pc]
}

But then, the fun began, it worked fine, but I started to chase my tail with two issues:

1. on subsequent runs, I was getting an error that the threat_monitoring resource already existed:

Microsoft.Insights/dataCollectionRuleAssociations/dcra-azurerm-vm-win10-1]
╷
│ Error: A resource with the ID "/providers/Microsoft.AADIAM/diagnosticSettings/threat-monitoring" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_monitor_aad_diagnostic_setting" for more information.
│ 
│   with azurerm_monitor_aad_diagnostic_setting.threat_monitoring,
│   on sentinel.tf line 76, in resource "azurerm_monitor_aad_diagnostic_setting" "threat_monitoring":
│   76: resource "azurerm_monitor_aad_diagnostic_setting" "threat_monitoring" {
│ 
╵

After some research, I learned how to remove it from Azure Entra ID diagnostics settings screen... that sort of worked, but then I started getting another issue, on destroy, I was getting:

╷
│ Error: deleting Solution (Subscription: "0407b4b9-2455-4e9c-895e-f1e5b4297e82"
│ Resource Group Name: "PurpleCloud-bvtkf"
│ Solution Name: "SecurityInsights(pc-bvtkf)"): performing Delete: unexpected status 204 (204 No Content) received with no body
│ 
│ deleting Solution (Subscription: "0407b4b9-2455-4e9c-895e-f1e5b4297e82"
│ Resource Group Name: "PurpleCloud-bvtkf"
│ Solution Name: "SecurityInsights(pc-bvtkf)"): performing Delete: unexpected status 204 (204 No Content) received with no body
╵

That appeared to be harmless, but the resource group was not fully deleted and securityInsights (LAW) was still there, so I had to manually delete the resource group...

Then, on retry, I got that first error above, about existing threat_monitoring resource... even though it was no longer in the Azure Entra ID diagnostic settings... uggghh....

hours later, even after a failed (cancelled PR), I resolved that I suck at HCL/Terraform... so, for now, I am punting...

Sorry, but have partial solution above with code snippet, so perhaps you understand other issues and can make quick work of them...

Do see Linkedin Message, I want to chat...

[iknowjason]

Hey Allen,

Good and bad news is good overall. That's how we learn :-). Sounds like you were really close to getting this working. Do you still want to submit your PR and I'll take a look and let you know where it needs to be corrected? That way you can get credit for this as a contributor? It definitely works with that terraform resource. You might have just ran into a local issue with your terraform state with all of the changes you were making. Try deleting out all of the .terraform directory, terraform.tfstate and the backup file. I can go ahead and fix everything but just wanted to check one last time if you wanted to submit what you had done and I can take a look at it. That's the nice thing about git. It allows collaboration for stuff like this.

Other questions:

1. Did you get Sentinel Onboarded manually using those API steps?
2. Were you able to find the velociraptor + ELK stuff in ad.py?

Responding to your LI.

[harperaa]

You can leave issue open, I will try again in a couple weeks when I free up. No hurry until then. I didn’t try to replicate the manual step, as I had success with the terraform for onboarding step. That part was working with snippet shown. I did not check ELK script, but will. Thanks.

…

On Sun, Apr 6, 2025 at 9:13 AM Jason Ostrom ***@***.***> wrote: Hey Allen, Good and bad news is good overall. That's how we learn :-). Sounds like you were really close to getting this working. Do you still want to submit your PR and I'll take a look and let you know where it needs to be corrected? That way you can get credit for this as a contributor? It definitely works with that terraform resource. You might have just ran into a local issue with your terraform state with all of the changes you were making. Try deleting out all of the .terraform directory, terraform.tfstate and the backup file. I can go ahead and fix everything but just wanted to check one last time if you wanted to submit what you had done and I can take a look at it. That's the nice thing about git. It allows collaboration for stuff like this. Other questions: 1. Did you get Sentinel Onboarded manually using those API steps? 2. Were you able to find the velociraptor + ELK stuff in ad.py? Responding to your LI. — Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKE6AF6MLOCTSY4L3WHIS32YER73AVCNFSM6AAAAAB2NTGU5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOBRGQYTOOBTGA> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: iknowjason]*iknowjason* left a comment (iknowjason/PurpleCloud#42) <#42 (comment)> Hey Allen, Good and bad news is good overall. That's how we learn :-). Sounds like you were really close to getting this working. Do you still want to submit your PR and I'll take a look and let you know where it needs to be corrected? That way you can get credit for this as a contributor? It definitely works with that terraform resource. You might have just ran into a local issue with your terraform state with all of the changes you were making. Try deleting out all of the .terraform directory, terraform.tfstate and the backup file. I can go ahead and fix everything but just wanted to check one last time if you wanted to submit what you had done and I can take a look at it. That's the nice thing about git. It allows collaboration for stuff like this. Other questions: 1. Did you get Sentinel Onboarded manually using those API steps? 2. Were you able to find the velociraptor + ELK stuff in ad.py? Responding to your LI. — Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKE6AF6MLOCTSY4L3WHIS32YER73AVCNFSM6AAAAAB2NTGU5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOBRGQYTOOBTGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[pir8g33k]

I have my own good news and bad news... ha.

I was able to adjust the sentinel.py code to add the functionality to onboard sentinel, by adding the following code at line 1390:

# Onboard the Log Analytics Workspace to Sentinel
resource "azurerm_sentinel_log_analytics_workspace_onboarding" "pc" {
  workspace_id = azurerm_log_analytics_workspace.pc.id
  
  # Ensure the Log Analytics Solution is created first
  depends_on = [azurerm_log_analytics_solution.pc]
}

But then, the fun began, it worked fine, but I started to chase my tail with two issues:

1. on subsequent runs, I was getting an error that the threat_monitoring resource already existed:

Microsoft.Insights/dataCollectionRuleAssociations/dcra-azurerm-vm-win10-1]
╷
│ Error: A resource with the ID "/providers/Microsoft.AADIAM/diagnosticSettings/threat-monitoring" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_monitor_aad_diagnostic_setting" for more information.
│ 
│   with azurerm_monitor_aad_diagnostic_setting.threat_monitoring,
│   on sentinel.tf line 76, in resource "azurerm_monitor_aad_diagnostic_setting" "threat_monitoring":
│   76: resource "azurerm_monitor_aad_diagnostic_setting" "threat_monitoring" {
│ 
╵

After some research, I learned how to remove it from Azure Entra ID diagnostics settings screen... that sort of worked, but then I started getting another issue, on destroy, I was getting:

╷
│ Error: deleting Solution (Subscription: "0407b4b9-2455-4e9c-895e-f1e5b4297e82"
│ Resource Group Name: "PurpleCloud-bvtkf"
│ Solution Name: "SecurityInsights(pc-bvtkf)"): performing Delete: unexpected status 204 (204 No Content) received with no body
│ 
│ deleting Solution (Subscription: "0407b4b9-2455-4e9c-895e-f1e5b4297e82"
│ Resource Group Name: "PurpleCloud-bvtkf"
│ Solution Name: "SecurityInsights(pc-bvtkf)"): performing Delete: unexpected status 204 (204 No Content) received with no body
╵

That appeared to be harmless, but the resource group was not fully deleted and securityInsights (LAW) was still there, so I had to manually delete the resource group...

Then, on retry, I got that first error above, about existing threat_monitoring resource... even though it was no longer in the Azure Entra ID diagnostic settings... uggghh....

hours later, even after a failed (cancelled PR), I resolved that I suck at HCL/Terraform... so, for now, I am punting...

Sorry, but have partial solution above with code snippet, so perhaps you understand other issues and can make quick work of them...

Do see Linkedin Message, I want to chat...

Hi i have the same issue but i was able to fixed those due to some update in azurern for sentinel will request for PR later

[iknowjason]

@pir8g33k @harperaa
Feel free to submit a PR for my review on this. I fixed the issue right after the issue was opened but kept it local. I thought @harperaa wanted to submit the pR for the fix. But it has been a while, so go ahead and submit it @pir8g33k .
It's an easy fix and it works great to automatically onboard Sentinel via terraform :-)