Cannot install Qualys extension to Azure-Arc enabled Windows servers (Microsoft Defender for Cloud)

Development Programming & Scripting
,
1

Several machines are in an unhelathy state, but when I try to fix them via “Deploy the integrated vulnerability scanner powered by Qualys (included with Microsoft Defender for servers)”.

This keeps failing again and again - “Installation of the scanner failed or timed out. Try removing the extension and deploying again.”.

I went through several articles and we should have all prerequisites in place already - extension currently installed on VM:

  • MicrosoftMonitoringAgent
  • MDE.Windows

Also machine(s) is able to reach (are whitelisted) over port 443:

Machine(s) is in ‘Connected’ state under Azure Arc/Servers and Log Analytics agent is also healthy.

Any suggestions/help much appreciated!

ERROR:

Remediation failure Failed remediating the selected resources.

Error details

{
  "status": "Failed",
  "error": {
    "code": "ResourceDeploymentFailure",
    "message": "The resource operation completed with terminal provisioning state 'Failed'."
  }
}

ERROR DETAILS:

{
    "authorization": {
        "action": "Microsoft.Security/serverVulnerabilityAssessments/write",
        "scope": "/subscriptions/%SubscriptionId%/resourcegroups/ResourceGroup/providers/microsoft.hybridcompute/machines/%ComputerName%/providers/Microsoft.Security/serverVulnerabilityAssessments/Default"
    },
    "caller": "%User/Caller%",
    "channels": "Operation",
    "claims": {
        "aud": "https://management.core.windows.net/",
        "iss": "https://sts.windows.net/%TenantId%/",
        "iat": "1655901465",
        "nbf": "1655901465",
        "exp": "1655906623",
        "http://schemas.microsoft.com/claims/authnclassreference": "1",
        "aio": "AeQAG/8TAAAANUzd1RjrUyWOS2io+01dvLIi44vh+gBvGvFImnsITiE48MYOdttdDYdgFt2i/uMewlR0m0kjufyDC5umbwrtUJsSDFjLEUXtb0wCMDidtKQAe7eZo+nrupl6EvWy0e6HUFKUnGq/vZRbdVmFQ6Do4Tv8sLwhLKFx8evEYNikaC+2Apm/+rStbTS2Z/EFxayPaSM8S0mAWMZpX7G5B2iM9OJNH5VyIJQwZOVUg6TDiIcPrcQk0d91Wg3pXtBrN1xvp00AnCG/tpoyqi0cqzVcSeClqNz1U4T7ycoLxvJpgtg=",
        "altsecid": "5::100320004BA7C285",
        "http://schemas.microsoft.com/claims/authnmethodsreferences": "pwd,rsa,mfa",
        "appid": "c44b4083-3bb0-49c1-b47d-974e53cbdf3c",
        "appidacr": "2",
        "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress": "%User/Caller%",
        "groups": "5ff04599-3403-441b-bd34-6f925da043fa,7019c2b2-ddd0-4d61-abe1-201b21eb79b0,6ff2cb1b-f1c3-4daf-8609-9ad31c788665,eb28fcda-c821-499b-acbf-1b3aef97a50a,c422057f-4aa6-4f2e-8cb2-d494f8528fee,1236438a-f436-4413-9f3e-08ae06bd7a51,2ffe5c49-a10b-409c-8ec0-8c4f33720551,bd32a1c9-19fb-429d-928f-41edc49f0d6c",
        "http://schemas.microsoft.com/identity/claims/identityprovider": "https://sts.windows.net/fed95e69-8d73-43fe-affb-a7d85ede36fb/",
        "ipaddr": "109.183.33.251",
        "name": "%User/Caller%",
        "http://schemas.microsoft.com/identity/claims/objectidentifier": "53d41d63-2434-4cd6-9b8f-5b120213e9c4",
        "puid": "10032001686C41AD",
        "rh": "0.AUcApXuB_4XCYUG5dCXpi1ZCSEZIf3kAutdPukPawfj2MBNHAOI.",
        "http://schemas.microsoft.com/identity/claims/scope": "user_impersonation",
        "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier": "Z9X2hMrO8HeSaEc39ruVNW2GqLU-Gdaguaja6BW91mk",
        "http://schemas.microsoft.com/identity/claims/tenantid": "%TenantId%",
        "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name": "%User/Caller%",
        "uti": "Ve6SzqPM6EiALk1ZHqGAAA",
        "ver": "1.0",
        "xms_tcdt": "1576483460"
    },
    "correlationId": "4c518592-e2c5-4d85-80a5-c7eac74ef1eb",
    "description": "",
    "eventDataId": "f1c8d8cb-88ae-4b83-922b-94e2ae96b6a5",
    "eventName": {
        "value": "EndRequest",
        "localizedValue": "End request"
    },
    "category": {
        "value": "Administrative",
        "localizedValue": "Administrative"
    },
    "eventTimestamp": "2022-06-22T13:15:38.1236836Z",
    "id": "/subscriptions/%SubscriptionId%/resourcegroups/ResourceGroup/providers/microsoft.hybridcompute/machines/%ComputerName%/providers/Microsoft.Security/serverVulnerabilityAssessments/Default/events/f1c8d8cb-88ae-4b83-922b-94e2ae96b6a5/ticks/637915005381236836",
    "level": "Error",
    "operationId": "004d96f4-a255-49dc-8585-6d3903cbbeb1",
    "operationName": {
        "value": "Microsoft.Security/serverVulnerabilityAssessments/write",
        "localizedValue": "Create or Update server vulnerability assessments"
    },
    "resourceGroupName": "ResourceGroup",
    "resourceProviderName": {
        "value": "microsoft.hybridcompute",
        "localizedValue": "microsoft.hybridcompute"
    },
    "resourceType": {
        "value": "Microsoft.Security/serverVulnerabilityAssessments",
        "localizedValue": "Microsoft.Security/serverVulnerabilityAssessments"
    },
    "resourceId": "/subscriptions/%SubscriptionId%/resourcegroups/ResourceGroup/providers/microsoft.hybridcompute/machines/%ComputerName%/providers/Microsoft.Security/serverVulnerabilityAssessments/Default",
    "status": {
        "value": "Failed",
        "localizedValue": "Failed"
    },
    "subStatus": {
        "value": "",
        "localizedValue": ""
    },
    "submissionTimestamp": "2022-06-22T13:16:55.1640903Z",
    "subscriptionId": "%SubscriptionId%",
    "tenantId": "%TenantId%",
    "properties": {
        "statusCode": "Conflict",
        "statusMessage": "{\"status\":\"Failed\",\"error\":{\"code\":\"ResourceDeploymentFailure\",\"message\":\"The resource operation completed with terminal provisioning state 'Failed'.\"}}",
        "eventCategory": "Administrative",
        "entity": "/subscriptions/%SubscriptionId%/resourcegroups/ResourceGroup/providers/microsoft.hybridcompute/machines/%ComputerName%/providers/Microsoft.Security/serverVulnerabilityAssessments/Default",
        "message": "Microsoft.Security/serverVulnerabilityAssessments/write",
        "hierarchy": "%TenantId%/35fd4cf9-024a-4adf-95bf-c7b918208e1c/%SubscriptionId%"
    },
    "relatedEvents": []
}
``` Several machines are in an unhelathy state, but when I try to fix them via "Deploy the integrated vulnerability scanner powered by Qualys (included with Microsoft Defender for servers)".

This keeps failing again and again - "Installation of the scanner failed or timed out. Try removing the extension and deploying again.".

I went through several articles and we should have all prerequisites in place already - extension currently installed on VM:

MicrosoftMonitoringAgent
MDE.Windows
Also machine(s) is able to reach (are whitelisted) over port 443:

https://qagpublic.qg1.apps.qualys.com
http://qagpublic.qg2.apps.qualys.com
http://qagpublic.qg3.apps.qualys.com
http://qagpublic.qg1.apps.qualys.eu
http://qagpublic.qg2.apps.qualys.eu
Machine(s) is in 'Connected' state under Azure Arc/Servers and Log Analytics agent is also healthy.

Any suggestions/help much appreciated!

ERROR:

Remediation failure Failed remediating the selected resources.

Error details

{
  "status": "Failed",
  "error": {
    "code": "ResourceDeploymentFailure",
    "message": "The resource operation completed with terminal provisioning state 'Failed'."
  }
}
ERROR DETAILS:

{
    "authorization": {
        "action": "Microsoft.Security/serverVulnerabilityAssessments/write",
        "scope": "/subscriptions/%SubscriptionId%/resourcegroups/ResourceGroup/providers/microsoft.hybridcompute/machines/%ComputerName%/providers/Microsoft.Security/serverVulnerabilityAssessments/Default"
    },
    "caller": "%User/Caller%",
    "channels": "Operation",
    "claims": {
        "aud": "https://management.core.windows.net/",
        "iss": "https://sts.windows.net/%TenantId%/",
        "iat": "1655901465",
        "nbf": "1655901465",
        "exp": "1655906623",
        "http://schemas.microsoft.com/claims/authnclassreference": "1",
        "aio": "AeQAG/8TAAAANUzd1RjrUyWOS2io+01dvLIi44vh+gBvGvFImnsITiE48MYOdttdDYdgFt2i/uMewlR0m0kjufyDC5umbwrtUJsSDFjLEUXtb0wCMDidtKQAe7eZo+nrupl6EvWy0e6HUFKUnGq/vZRbdVmFQ6Do4Tv8sLwhLKFx8evEYNikaC+2Apm/+rStbTS2Z/EFxayPaSM8S0mAWMZpX7G5B2iM9OJNH5VyIJQwZOVUg6TDiIcPrcQk0d91Wg3pXtBrN1xvp00AnCG/tpoyqi0cqzVcSeClqNz1U4T7ycoLxvJpgtg=",
        "altsecid": "5::100320004BA7C285",
        "http://schemas.microsoft.com/claims/authnmethodsreferences": "pwd,rsa,mfa",
        "appid": "c44b4083-3bb0-49c1-b47d-974e53cbdf3c",
        "appidacr": "2",
        "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress": "%User/Caller%",
        "groups": "5ff04599-3403-441b-bd34-6f925da043fa,7019c2b2-ddd0-4d61-abe1-201b21eb79b0,6ff2cb1b-f1c3-4daf-8609-9ad31c788665,eb28fcda-c821-499b-acbf-1b3aef97a50a,c422057f-4aa6-4f2e-8cb2-d494f8528fee,1236438a-f436-4413-9f3e-08ae06bd7a51,2ffe5c49-a10b-409c-8ec0-8c4f33720551,bd32a1c9-19fb-429d-928f-41edc49f0d6c",
        "http://schemas.microsoft.com/identity/claims/identityprovider": "https://sts.windows.net/fed95e69-8d73-43fe-affb-a7d85ede36fb/",
        "ipaddr": "109.183.33.251",
        "name": "%User/Caller%",
        "http://schemas.microsoft.com/identity/claims/objectidentifier": "53d41d63-2434-4cd6-9b8f-5b120213e9c4",
        "puid": "10032001686C41AD",
        "rh": "0.AUcApXuB_4XCYUG5dCXpi1ZCSEZIf3kAutdPukPawfj2MBNHAOI.",
        "http://schemas.microsoft.com/identity/claims/scope": "user_impersonation",
        "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier": "Z9X2hMrO8HeSaEc39ruVNW2GqLU-Gdaguaja6BW91mk",
        "http://schemas.microsoft.com/identity/claims/tenantid": "%TenantId%",
        "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name": "%User/Caller%",
        "uti": "Ve6SzqPM6EiALk1ZHqGAAA",
        "ver": "1.0",
        "xms_tcdt": "1576483460"
    },
    "correlationId": "4c518592-e2c5-4d85-80a5-c7eac74ef1eb",
    "description": "",
    "eventDataId": "f1c8d8cb-88ae-4b83-922b-94e2ae96b6a5",
    "eventName": {
        "value": "EndRequest",
        "localizedValue": "End request"
    },
    "category": {
        "value": "Administrative",
        "localizedValue": "Administrative"
    },
    "eventTimestamp": "2022-06-22T13:15:38.1236836Z",
    "id": "/subscriptions/%SubscriptionId%/resourcegroups/ResourceGroup/providers/microsoft.hybridcompute/machines/%ComputerName%/providers/Microsoft.Security/serverVulnerabilityAssessments/Default/events/f1c8d8cb-88ae-4b83-922b-94e2ae96b6a5/ticks/637915005381236836",
    "level": "Error",
    "operationId": "004d96f4-a255-49dc-8585-6d3903cbbeb1",
    "operationName": {
        "value": "Microsoft.Security/serverVulnerabilityAssessments/write",
        "localizedValue": "Create or Update server vulnerability assessments"
    },
    "resourceGroupName": "ResourceGroup",
    "resourceProviderName": {
        "value": "microsoft.hybridcompute",
        "localizedValue": "microsoft.hybridcompute"
    },
    "resourceType": {
        "value": "Microsoft.Security/serverVulnerabilityAssessments",
        "localizedValue": "Microsoft.Security/serverVulnerabilityAssessments"
    },
    "resourceId": "/subscriptions/%SubscriptionId%/resourcegroups/ResourceGroup/providers/microsoft.hybridcompute/machines/%ComputerName%/providers/Microsoft.Security/serverVulnerabilityAssessments/Default",
    "status": {
        "value": "Failed",
        "localizedValue": "Failed"
    },
    "subStatus": {
        "value": "",
        "localizedValue": ""
    },
    "submissionTimestamp": "2022-06-22T13:16:55.1640903Z",
    "subscriptionId": "%SubscriptionId%",
    "tenantId": "%TenantId%",
    "properties": {
        "statusCode": "Conflict",
        "statusMessage": "{\"status\":\"Failed\",\"error\":{\"code\":\"ResourceDeploymentFailure\",\"message\":\"The resource operation completed with terminal provisioning state 'Failed'.\"}}",
        "eventCategory": "Administrative",
        "entity": "/subscriptions/%SubscriptionId%/resourcegroups/ResourceGroup/providers/microsoft.hybridcompute/machines/%ComputerName%/providers/Microsoft.Security/serverVulnerabilityAssessments/Default",
        "message": "Microsoft.Security/serverVulnerabilityAssessments/write",
        "hierarchy": "%TenantId%/35fd4cf9-024a-4adf-95bf-c7b918208e1c/%SubscriptionId%"
    },
    "relatedEvents": []
}