Effective Cloud Resource Monitoring with Azure Monitor: Hands-On Lab

Introduction

Azure Monitor is a comprehensive solution for collecting, analyzing, and acting on telemetry data from Azure and on-premises environments. This practical guide walks you through setting up a proof of concept to collect events and performance counters from virtual machines using the Azure Monitor Agent.

By the end of this workshop, you will:

• Understand the basics of Azure Monitor.

• Deploy an Azure virtual machine (VM).

• Create essential Azure resources such as a Log Analytics workspace and a storage account.

• Create and configure a Data Collection Rule (DCR).

Let’s dive into the hands-on lab!

Workshop Scenario

You have been asked to collect events and performance counters from virtual machines with Azure Monitor Agent.

For all the resources in this lab, we are using the East US region.

Lab objectives

In this lab, you will complete the following exercises:

• Exercise 1: Deploy an Azure virtual machine

• Exercise 2: Create a Log Analytics workspace

• Exercise 3: Create an Azure storage account

• Exercise 4: Create and configure a data collection rule (DCR).

Instructions

Pre-requisites:

• An active Azure subscription.

• Knowledge of Azure Monitor and PowerShell basics.

Exercise 1: Deploy an Azure virtual machine

Exercise timing: 10 minutes

Purpose:

This exercise focuses on deploying a virtual machine (VM) to serve as the monitored resource for collecting performance data. It ensures you have a foundational infrastructure to work with Azure Monitor.

In this exercise, you will complete the following tasks:

• Task 1: Deploy an Azure virtual machine.

Task 1: Deploy an Azure virtual machine

1. Sign-in to the Azure portal https://portal.azure.com/.

   Sign in to the Azure portal using an account that has the Owner or Contributor role in the Azure subscription you are using for this lab.

2. Open the Cloud Shell by clicking the first icon in the top right of the Azure Portal. If prompted, select PowerShell and Create Storage.

3. Ensure PowerShell is selected in the drop-down menu in the upper-left corner of the Cloud Shell pane.

4. In the PowerShell session within the Cloud Shell pane, run the following to create a resource group that will be used in this lab:

    powershellNew-AzResourceGroup -Name AZ500LAB131415 -Location 'EastUS'
   

5. In the PowerShell session within the Cloud Shell pane, run the following to enable encryption at host (EAH)

    powershellRegister-AzProviderFeature -FeatureName "EncryptionAtHost" -ProviderNamespace Microsoft.Compute
   

6. In the PowerShell session within the Cloud Shell pane, run the following to create a new Azure virtual machine.

    powershellNew-AzVm -ResourceGroupName "AZ500LAB131415" -Name "myVM" -Location 'EastUS' -VirtualNetworkName "myVnet" -SubnetName "mySubnet" -SecurityGroupName   "myNetworkSecurityGroup" -PublicIpAddressName "myPublicIpAddress" -PublicIpSku Standard -OpenPorts 80,3389 -Size Standard_DS1_v2
   

When prompted for credentials:

Wait for the deployment to complete.

In the PowerShell session within the Cloud Shell pane, please run the following to confirm that the virtual machine named myVM was created and its ProvisioningState has Succeeded.

powershellGet-AzVM -Name 'myVM' -ResourceGroupName 'AZ500LAB131415' | Format-Table

Result: You have successfully deployed an Azure virtual machine.

8. Close the Cloud Shell pane

Exercise 2: Create a Log Analytics workspace

Exercise timing: 10 minutes

Purpose:

A Log Analytics workspace acts as the central repository for monitoring data. It consolidates logs, performance metrics, and diagnostic data, enabling efficient analysis and troubleshooting. This step is essential for building a robust monitoring solution.

In this exercise, you will complete the following tasks:

• Task 1: Create a Log Analytics workspace.

Task 1: Create a Log Analytics workspace

In this task, you will create a Log Analytics workspace.

1. In the Azure portal, in the Search resources, services, and docs text box at the top of the Azure portal page, type Log Analytics workspaces and press the Enter key.

2. On the Log Analytics workspaces blade, click + Create.

3. On the Basics tab of the Create Log Analytics workspace blade, specify the following settings (leave others with their default values):

4. Select Review + Create.

5. On the Review + Create tab of the Create Log Analytics workspace blade, select Create.

Result: You have successfully created a Log Analytics workspace, which will serve as the hub for analyzing and visualizing your collected monitoring data.

Exercise 3: Create an Azure storage account

Estimated timing: 10 minutes

Purpose:

Storage accounts are used to store diagnostic data, metrics, and logs securely. In this context, the storage account acts as a repository for retaining log data and can also integrate seamlessly with monitoring tools such as Log Analytics or Azure Monitor. This ensures that data is available for long-term analysis and troubleshooting.

In this exercise, you will complete the following tasks:

• Task 1: Create an Azure storage account.

Task 1: Create an Azure storage account

In this task, you will create a storage account.

1. In the Azure portal, in the Search resources, services, and docs text box at the top of the Azure portal page, type Storage accounts and press the Enter key.

2. On the Storage accounts blade in the Azure portal, click the + Create button to create a new storage account.

   

3. On the Basics tab of the Create storage account blade, specify the following settings (leave others with their default values):

4. On the Basics tab of the Create storage account blade, click Review, wait for the validation process to complete, and click Create.

Wait for the Storage account to be created. This should take about 2 minutes.

Result: You have successfully created an Azure storage account that will serve as the backbone for storing logs, metrics, and other critical diagnostic data for the monitoring solution.

Exercise 4: Create a Data Collection Rule (DCR)

Estimated timing: 15 minutes

Purpose:

Data Collection Rules (DCRs) define how data is collected and sent to your Log Analytics workspace. This step configures what performance counters to monitor. The DCR acts as the bridge connecting your monitored resource (e.g., a virtual machine) to the Log Analytics workspace by specifying what data to collect and where to send it.

In this exercise, you will complete the following tasks:

• Task 1: Create a Data Collection Rule.

Task 1: Create a Data Collection Rule.

In this task, you will create a data collection rule.

1. In the Azure portal, in the Search resources, services, and docs text box at the top of the Azure portal page, type Monitor and press the Enter key.

2. On the Monitor Settings blade, click Data Collection Rules.

   

3. On the Basics tab of the Create Data Collection Rule blade, specify the following settings:

1. Click on the button labeled Next: Resources > to proceed.

2. On the Resources tab, select + Add resources, and check Enable Data Collection Endpoints. In the Select a scope template, check AZ500LAB131415, and click Apply.

   

3. Click on the button labeled Next: Collect and deliver > to proceed.

   

4. Click + Add data source, then on the Add data source page, change the Data source type drop-down menu to display Performance Counters. Leave the following default settings:

1. Click on the button labeled Next: Destination > to proceed.

2. Change the Destination type drop-down menu to display Azure Monitor Logs. In the Subscription window, ensure that your Subscription is displayed, then change the Account or namespace drop-down menu to reflect your previously created Log Analytics Workspace.

   

3. Click on Add data source at the bottom of the page.

   

4. Click Review + Create.

   

5. Click Create.

   Explanation of What Each Step Does:

   • Adding Resources: This step ensures the DCR is linked to the virtual machine that will generate the performance data.

   • Data Source Configuration: Specifies the types of performance data (e.g., CPU usage) to monitor and at what frequency.

   • Setting Destination: Ensures the collected data is directed to the appropriate Log Analytics workspace for analysis.

Results: You deployed an Azure virtual machine, Log Analytics workspace, Azure storage account, and a data collection rule to collect events and performance counters from virtual machines with Azure Monitor Agent.

Validating the Proof of Concept

Steps to Validate:

1. Navigate to your Log Analytics workspace in the Azure portal.

2. Use Logs (Query) to search for data being collected from your VM:

    Perf
    | where ObjectName == "Processor" and CounterName == "% Processor Time"
    | summarize AvgValue = avg(CounterValue) by bin(TimeGenerated, 1m)
   

3. Confirm that the performance data (CPU, Memory, etc.) is visible in the query results.

   Additional Validation and Troubleshooting Tips:

   • Validation: Cross-check whether the data collected aligns with the performance metrics expected for the deployed virtual machine.

   • Verify Connectivity: Ensure that the Azure Monitor Agent is properly installed and configured on the virtual machine.

   • Missing Data:

     • Navigate to the virtual machine settings and confirm that the correct Data Collection Rule (DCR) is associated.

     • Check the diagnostic settings to ensure they are correctly set up to forward data to the Log Analytics workspace.

   • Query Issues: If queries fail to return expected data, confirm that the syntax and field names in the query match the configuration of your data source.

   • Logs Not Updating: Check for any network or permission issues that might block the data flow from the VM to the Log Analytics workspace.

Clean Up Resources

To avoid unnecessary costs, remove all resources created during this lab:

1. Open the Cloud Shell and select PowerShell.

2. Run the following command to delete the resource group:

    Remove-AzResourceGroup -Name "AZ500LAB131415" -Force -AsJob
   

3. Close the Cloud Shell pane.

Conclusion

Congratulations! You have successfully completed this lab.

In this hands-on lab, you successfully learned how to deploy an Azure virtual machine, create and configure essential Azure resources, and use the Azure Monitor Agent to collect events and performance counters.

These skills are crucial for effectively managing and monitoring Azure resources. By understanding and implementing these processes, you are now better equipped to ensure the performance and reliability of your cloud infrastructure. If you have any feedback or questions, feel free to share them in the comments.

Challenges :

1. Multiple Regions: Deploy and monitor resources in multiple regions for redundancy.

2. Cost Optimization: Investigate Azure Cost Management to understand the expenses of monitoring and suggest ways to optimize costs.

Further Resources

• Azure Monitor Documentation

• Azure Storage Account Best Practices

• Log Analytics Workspace Overview

• Azure Monitor Pricing

• Azure Monitor Fundamentals Training

• Azure Monitor Vs Microsoft Sentinel