Create and configure a Virtual Machine (VM) for Windows and Linux
- May include but not limited to: Configure high availability; configure monitoring, networking, storage, and virtual machine size; deploy and configure scale sets
The following are the questions compiled under each of the above listed topics:
1) Create and configure a Virtual Machine (VM) for Windows:
What is Azure Virtual Machine?
Azure Virtual Machines (VM) is one of several types of on-demand, scalable computing resources that Azure offers. Typically, you choose a VM when you need more control over the computing environment than the other choices offer. It gives flexibility of virtualization without having to buy and maintain the physical hardware that runs it.
What do I need to think about before creating a VM?
There are always a multitude of design considerations when you build out an application infrastructure in Azure. These aspects of a VM are important to think about before you start:
- The names of your application resources
- The location where the resources are stored
- The size of the VM
- The maximum number of VMs that can be created
- The operating system that the VM runs
- The configuration of the VM after it starts
- The related resources that the VM needs
What are the considerations for VM naming?
A virtual machine has a name assigned to it and it has a computer name configured as part of the operating system. The name of a VM can be up to 15 characters.
What is VM Locations?
A region is called location when you create a VM. For a VM, the location specifies where the virtual hard disks are stored.
What is VM Size?
The size of the VM that you use is determined by the workload that you want to run. The size that you choose then determines factors such as processing power, memory, and storage capacity. Azure offers a wide variety of sizes to support many types of uses.
How is the VM pricing is calculated?
Azure charges an hourly price based on the VM’s size and operating system. For partial hours, Azure charges only for the minutes used. Storage is priced and charged separately.
What is VM Limits?
Your subscription has default quota limits in place that could impact the deployment of many VMs for your project. The current limit on a per subscription basis is 20 VMs per region. Limits can be raised by filing a support ticket requesting an increase.
What is Operating system disks and images?
Virtual machines use virtual hard disks (VHDs) to store their operating system (OS) and data. VHDs are also used for the images you can choose from to install an OS. Marketplace images are identified by image publisher, offer, sku, and version (typically version is specified as latest). Only 64-bit operating systems are supported.
What is Extensions?
VM extensions give your VM additional capabilities through post deployment configuration and automated tasks.
These common tasks can be accomplished using extensions:
Run custom scripts – The Custom Script Extension helps you configure workloads on the VM by running your script when the VM is provisioned.
Deploy and manage configurations – The PowerShell Desired State Configuration (DSC) Extension helps you set up DSC on a VM to manage configurations and environments.
Collect diagnostics data – The Azure Diagnostics Extension helps you configure the VM to collect diagnostics data that can be used to monitor the health of your application.
What are the related resources required for VM creation?
The following resources are required to create a VM:
What are the various options to create and manage a VM?
The following are the various options to create and manage a VM:
What is the SLA for Virtual Machines?
- 99.9% for premium storage
- 99.95% for two or more VM's in Availability Set
- 99.99% for two or more VM's in Availablity Zones in same Azure Region
what are the options to back up the VM?
A Recovery Services vault is used to protect data and assets in both Azure Backup and Azure Site Recovery services. You can use a Recovery Services vault to deploy and manage backups for Resource Manager-deployed VMs using PowerShell.
=======
How does Azure Backup work for Linux VM?
When the Azure Backup service initiates a backup, it triggers the backup extension to take a point-in-time snapshot. The Azure Backup service uses the VMSnapshotLinux extension in Linux. The extension is installed during the first VM backup if the VM is running. If the VM is not running, the Backup service takes a snapshot of the underlying storage (since no application writes occur while the VM is stopped).
By default, Azure Backup takes a file system consistent backup for Linux VM but it can be configured to take application consistent backup using pre-script and post-script framework. Once the Azure Backup service takes the snapshot, the data is transferred to the vault. To maximize efficiency, the service identifies and transfers only the blocks of data that have changed since the previous backup.
When the data transfer is complete, the snapshot is removed and a recovery point is created.
How to restore a file from backup vault for Linux VM?
If you accidentally delete or make changes to a file, you can use File Recovery to recover the file from your backup vault. File Recovery uses a script that runs on the VM, to mount the recovery point as a local drive. These drives remain mounted for 12 hours so that you can copy files from the recovery point and restore them to the VM.
Explain the steps to recover file from VM backup?
https://docs.microsoft.com/en-us/azure/virtual-machines/linux/tutorial-backup-vms
=========
=======
How does Azure Backup work for Linux VM?
When the Azure Backup service initiates a backup, it triggers the backup extension to take a point-in-time snapshot. The Azure Backup service uses the VMSnapshotLinux extension in Linux. The extension is installed during the first VM backup if the VM is running. If the VM is not running, the Backup service takes a snapshot of the underlying storage (since no application writes occur while the VM is stopped).
By default, Azure Backup takes a file system consistent backup for Linux VM but it can be configured to take application consistent backup using pre-script and post-script framework. Once the Azure Backup service takes the snapshot, the data is transferred to the vault. To maximize efficiency, the service identifies and transfers only the blocks of data that have changed since the previous backup.
When the data transfer is complete, the snapshot is removed and a recovery point is created.
How to restore a file from backup vault for Linux VM?
If you accidentally delete or make changes to a file, you can use File Recovery to recover the file from your backup vault. File Recovery uses a script that runs on the VM, to mount the recovery point as a local drive. These drives remain mounted for 12 hours so that you can copy files from the recovery point and restore them to the VM.
Explain the steps to recover file from VM backup?
https://docs.microsoft.com/en-us/azure/virtual-machines/linux/tutorial-backup-vms
=========
2) Create and configure a Virtual Machine (VM) for Linux:
3) Configure high availability for VM:
Regions and Availability:
What are Azure Regions?
Azure operates in multiple datacenters around the world. These datacenters are grouped into geographic regions, giving you flexibility in choosing where to build your applications. This approach gives you flexibility as you design applications to create VMs closest to your users and to meet any legal, compliance, or tax purposes.
What are Special Azure regions?
1) US Gov Virginia and US Gov Iowa (A physical and logical network-isolated instance for US Government)
2) China East and China North (21Vianet partnership)
3) Germany Central and Germany Northeast (under control of T-Systems, a Deutsche Telekom company)
What is Region Pairs?
Each Azure region is paired with another region within the same geography (such as US, Europe, or Asia). This approach allows for the replication of resources, such as VM storage, across a geography that should reduce the likelihood of natural disasters, civil unrest, power outages, or physical network outages affecting both regions at once.
What are the advantages of Regional Pairs?
- In the event of a wider Azure outage, one region is prioritized out of every pair to help reduce the time to restore for applications.
- Planned Azure updates are rolled out to paired regions one at a time to minimize downtime and risk of application outage.
- Data continues to reside within the same geography as its pair (except for Brazil South) for tax and law enforcement jurisdiction purposes.
| Primary | Secondary |
|---|---|
| West US | East US |
| North Europe | West Europe |
| Southeast Asia | East Asia |
What are the various Storage replication options available?
1) Azure Managed Disks
Locally redundant storage (LRS)
Replicates your data three times within the region in which you created your storage account.
2) Storage account-based disks
a) Locally redundant storage (LRS)
Replicates your data three times within the region in which you created your storage account.
b) Zone redundant storage (ZRS)
Replicates your data three times across two to three facilities, either within a single region or across two regions.
c) Geo-redundant storage (GRS)
Replicates your data to a secondary region that is hundreds of miles away from the primary region.
d) Read-access geo-redundant storage (RA-GRS)
Replicates your data to a secondary region, as with GRS, but also then provides read-only access to the data in the secondary location.
| Replication strategy | LRS | ZRS | GRS | RA-GRS |
|---|---|---|---|---|
| Data is replicated across multiple facilities. | No | Yes | Yes | Yes |
| Data can be read from the secondary location and from the primary location. | No | No | No | Yes |
| Number of copies of data maintained on separate nodes. | 3 | 3 | 6 | 6 |
What are various storage type and their backup options for VM's?
Azure Managed Disks
Premium Managed Disks are backed by Solid-State Drives (SSDs) and Standard Managed Disks are backed by regular spinning disks. Both Premium and Standard Managed Disks are charged based on the provisioned capacity for the disk.
Unmanaged disks
- Premium storage is backed by Solid-State Drives (SSDs) and is charged based on the capacity of the disk.
- Standard storage is backed by regular spinning disks and is charged based on the in-use capacity and desired storage availability.
- For RA-GRS, there is an additional Geo-Replication Data Transfer charge for the bandwidth of replicating that data to another Azure region
What is Availability Sets?
An availability set is a logical grouping of VMs within a datacenter that allows Azure to understand how your application is built to provide for redundancy and availability. We recommended that two or more VMs are created within an availability set to provide for a highly available application and to meet the 99.95% Azure SLA.
An availability set is composed of two additional groupings that protect against hardware failures and allow updates to safely be applied - fault domains (FDs) and update domains (UDs).
What is Fault Domain?
A fault domain is a logical group of underlying hardware that share a common power source and network switch, similar to a rack within an on-premises datacenter. As you create VMs within an availability set, the Azure platform automatically distributes your VMs across these fault domains. This approach limits the impact of potential physical hardware failures, network outages, or power interruptions.
What is Update Domain?
An update domain is a logical group of underlying hardware that can undergo maintenance or be rebooted at the same time. As you create VMs within an availability set, the Azure platform automatically distributes your VMs across these update domains. This approach ensures that at least one instance of your application always remains running as the Azure platform undergoes periodic maintenance. The order of update domains being rebooted may not proceed sequentially during planned maintenance, but only one update domain is rebooted at a time.
What is Manged Disk fault domains?
For VMs using Azure Managed Disks, VMs are aligned with managed disk fault domains when using a managed availability set. This alignment ensures that all the managed disks attached to a VM are within the same managed disk fault domain. Only VMs with managed disks can be created in a managed availability set. The number of managed disk fault domains varies by region - either two or three managed disk fault domains per region. You can read more about these managed disk fault domains for Linux VMs or Windows VMs.
What is Availability Zones?
Availability zones, an alternative to availability sets, expand the level of control you have to maintain the availability of the applications and data on your VMs. An Availability Zone is a physically separate zone within an Azure region. There are three Availability Zones per supported Azure region. Each Availability Zone has a distinct power source, network, and cooling. By architecting your solutions to use replicated VMs in zones, you can protect your apps and data from the loss of a datacenter. If one zone is compromised, then replicated apps and data are instantly available in another zone.
Maintenance of VM's
How Azure perform maintenance for the VM in Azure?
Azure periodically updates platform to improve the reliability, performance, and security of the host infrastructure for virtual machines. These updates range from patching software components in the hosting environment, upgrading networking components, to hardware decommissioning. However, there are cases where updates do have an impact and Azure chooses the least impactful method for updates:
- If a non-rebootful update is possible, the VM is paused while the host is updated or it is live migrated to an already updated host.
- If maintenance requires a reboot, you get a notice of when the maintenance is planned. Azure will also give a time window where you can start the maintenance yourself, at a time that works for you. Azure is investing in technologies to reduce the cases when the VMs have to be rebooted for planned platform maintenance.
What is Memory preserving maintenance?
The goal for most non-rebootful updates is less than 10 seconds pause for the VM. In certain cases memory preserving maintenance mechanisms are used, which pauses the VM for up to 30 seconds and preserves the memory in RAM. The virtual machine is then resumed and the clock of the virtual machine is automatically synchronized. Azure is increasingly using live migration technologies and improving memory preserving maintenance mechanism to reduce the pause duration.
These non-rebootful maintenance operations are applied fault domain by fault domain, and progress is stopped if any warning health signals are received.
The goal for most non-rebootful updates is less than 10 seconds pause for the VM. In certain cases memory preserving maintenance mechanisms are used, which pauses the VM for up to 30 seconds and preserves the memory in RAM. The virtual machine is then resumed and the clock of the virtual machine is automatically synchronized. Azure is increasingly using live migration technologies and improving memory preserving maintenance mechanism to reduce the pause duration.
These non-rebootful maintenance operations are applied fault domain by fault domain, and progress is stopped if any warning health signals are received.
What is the process when it requires a reboot of VM for planned maintenance?
In the rare case when VMs need to be rebooted for planned maintenance, you are notified in advance. Planned maintenance has two phases: the self-service window and a scheduled maintenance window.
The self-service window lets you start the maintenance on your VMs. During this time, you can query each VM to see their status and check the result of your last maintenance request.
When you start self-service maintenance, your VM is redeployed to an already updated node. Because the VM reboots, the temporary disk is lost and dynamic IP addresses associated with virtual network interface are updated.
If you start self-service maintenance and there is an error during the process, the operation is stopped, the VM is not updated and you get the option to retry the self-service maintenance.
When the self-service window has passed, the scheduled maintenance window begins. During this time window, you can still query for the maintenance window, but can't start the maintenance yourself.
4) Configure monitoring for VM:
What is Advanced monitoring options in Azure?
1. Azure Monitor is a service that monitors your cloud and on-premises environments to maintain their availability and performance. It delivers a comprehensive solution for collecting, analyzing, and acting on telemetry from your cloud and on-premises environments. It helps you understand how your applications are performing and proactively identifies issues affecting them and the resources they depend on. You can install an extension on a Linux VM or a Windows VM that installs the Log Analytics agent to collect log data and store in a Log Analytics workspace.
For Windows and Linux VMs, the recommended method for collecting logs is by installing the Log Analytics agent. The easiest way to install the Log Analytics agent on a VM is through the Log Analytics VM Extension. Using the extension simplifies the installation process and automatically configures the agent to send data to the Log Analytics workspace that you specify. The agent is also upgraded automatically, ensuring that you have the latest features and fixes.
2. Network Watcher enables you to monitor your VM and its associated resources as they relate to the network that they are in. You can install the Network Watcher Agent extension on a Linux VM or a Windows VM.
3. Azure Monitor for VMs monitors your Azure virtual machines (VM) at scale by analyzing the performance and health of your Windows and Linux VMs, including their different processes and interconnected dependencies on other resources and external processes.
What is Virtual machine lifecycle and its various states?
Azure Virtual Machines (VMs) go through different states that can be categorized into provisioning and power states.
Power states:
You can take advantage of many opportunities to monitor your VMs by collecting, viewing, and analyzing diagnostic and log data. To do simple monitoring of your VM, you can use the Overview screen for the VM in the Azure portal. You can use extensions to configure diagnostics on your VMs to collect additional metric data. You can also use more advanced monitoring options, such as Application Insights and Log Analytics.
What are the various tools available in Azure to monitor VM?
1) Simple monitoring of the VM from Overview screen in Azure Portal
2) Extensions to configure diagnostics on your VMs to collect additional metric data
3) Advanced monitoring options in Azure Monitor, such as Application Insights and Log Analytics
4) Diagnostics and metrics
5) Alerts
6) Azure Service Health
7) Azure Resource Health
8) Azure Activity Log
9) Network Watcher
How to collect Diagnostics and metrics?
You can set up and monitor the collection of diagnostics data using metrics in the Azure portal, the Azure CLI, Azure PowerShell, and programming Applications Programming Interfaces (APIs). For example, you can:
1) Observe basic metrics for the VM. On the Overview screen of the Azure portal, the basic metrics shown include CPU usage, network usage, total of disk bytes, and disk operations per second.
2) Enable the collection of boot diagnostics and view it using the Azure portal.
When bringing your own image to Azure or even booting one of the platform images, there can be many reasons why a VM gets into a non-bootable state. You can easily enable boot diagnostics when you create a VM by clicking Enabled for Boot Diagnostics under the Monitoring section of the Settings screen.
As VMs boot, the boot diagnostic agent captures boot output and stores it in Azure storage. This data can be used to troubleshoot VM boot issues. Boot diagnostics are not automatically enabled when you create a VM from command-line tools. Before enabling boot diagnostics, a storage account needs to be created for storing boot logs. If you enable boot diagnostics in the Azure portal, a storage account is automatically created for you.
3) Enable the collection of guest OS diagnostics data. When you create a VM, you have the opportunity on the settings screen to enable guest OS diagnostics. When you do enable the collection of diagnostics data, the IaaSDiagnostics extension for Linux or the IaaSDiagnostics extension for Windows is added to the VM, which enables you to collect additional disk, CPU, and memory data.
Using the collected diagnostics data, you can configure autoscaling for your VMs. You can also configure logs to store the data and set up alerts to let you know when performance isn't quite right.
How to create Alerts?
You can create alerts based on specific performance metrics. Examples of the issues you can be alerted about include when average CPU usage exceeds a certain threshold, or available free disk space drops below a certain amount.
What is Azure Service Health?
Azure Service Health provides personalized guidance and support when issues in Azure services affect you, and helps you prepare for upcoming planned maintenance. Azure Service Health alerts you and your teams using targeted and flexible notifications.
What is Azure Resource Health?
Azure Resource health helps you diagnose and get support when an Azure issue impacts your resources. It informs you about the current and past health of your resources and helps you mitigate issues. Resource health provides technical support when you need help with Azure service issues.
What is Azure Activity Log?
The Azure Activity Log is a subscription log that provides insight into subscription-level events that have occurred in Azure. The log includes a range of data, from Azure Resource Manager operational data to updates on Service Health events. You can click Activity Log in the Azure portal to view the log for your VM.
Some of the things you can do with the activity log include:
You can also access activity log data by using Azure PowerShell, the Azure CLI, or Monitor REST APIs.
What is Azure Diagnostic Logs?
Azure Diagnostic Logs are logs emitted by your VM that provide rich, frequent data about its operation. Diagnostic logs differ from the activity log by providing insight about operations that were performed within the VM.
Some of the things you can do with diagnostics logs include:
Azure Virtual Machines (VMs) go through different states that can be categorized into provisioning and power states.
Power states:
- Starting
- Running
- Stopping
- Stopped
- Deallocating
- Deallocated
- Create
- Update
- Delete
- Deallocate
You can take advantage of many opportunities to monitor your VMs by collecting, viewing, and analyzing diagnostic and log data. To do simple monitoring of your VM, you can use the Overview screen for the VM in the Azure portal. You can use extensions to configure diagnostics on your VMs to collect additional metric data. You can also use more advanced monitoring options, such as Application Insights and Log Analytics.
What are the various tools available in Azure to monitor VM?
1) Simple monitoring of the VM from Overview screen in Azure Portal
2) Extensions to configure diagnostics on your VMs to collect additional metric data
3) Advanced monitoring options in Azure Monitor, such as Application Insights and Log Analytics
4) Diagnostics and metrics
5) Alerts
6) Azure Service Health
7) Azure Resource Health
8) Azure Activity Log
9) Network Watcher
How to collect Diagnostics and metrics?
You can set up and monitor the collection of diagnostics data using metrics in the Azure portal, the Azure CLI, Azure PowerShell, and programming Applications Programming Interfaces (APIs). For example, you can:
1) Observe basic metrics for the VM. On the Overview screen of the Azure portal, the basic metrics shown include CPU usage, network usage, total of disk bytes, and disk operations per second.
2) Enable the collection of boot diagnostics and view it using the Azure portal.
When bringing your own image to Azure or even booting one of the platform images, there can be many reasons why a VM gets into a non-bootable state. You can easily enable boot diagnostics when you create a VM by clicking Enabled for Boot Diagnostics under the Monitoring section of the Settings screen.
As VMs boot, the boot diagnostic agent captures boot output and stores it in Azure storage. This data can be used to troubleshoot VM boot issues. Boot diagnostics are not automatically enabled when you create a VM from command-line tools. Before enabling boot diagnostics, a storage account needs to be created for storing boot logs. If you enable boot diagnostics in the Azure portal, a storage account is automatically created for you.
3) Enable the collection of guest OS diagnostics data. When you create a VM, you have the opportunity on the settings screen to enable guest OS diagnostics. When you do enable the collection of diagnostics data, the IaaSDiagnostics extension for Linux or the IaaSDiagnostics extension for Windows is added to the VM, which enables you to collect additional disk, CPU, and memory data.
Using the collected diagnostics data, you can configure autoscaling for your VMs. You can also configure logs to store the data and set up alerts to let you know when performance isn't quite right.
How to create Alerts?
You can create alerts based on specific performance metrics. Examples of the issues you can be alerted about include when average CPU usage exceeds a certain threshold, or available free disk space drops below a certain amount.
What is Azure Service Health?
Azure Service Health provides personalized guidance and support when issues in Azure services affect you, and helps you prepare for upcoming planned maintenance. Azure Service Health alerts you and your teams using targeted and flexible notifications.
What is Azure Resource Health?
Azure Resource health helps you diagnose and get support when an Azure issue impacts your resources. It informs you about the current and past health of your resources and helps you mitigate issues. Resource health provides technical support when you need help with Azure service issues.
What is Azure Activity Log?
The Azure Activity Log is a subscription log that provides insight into subscription-level events that have occurred in Azure. The log includes a range of data, from Azure Resource Manager operational data to updates on Service Health events. You can click Activity Log in the Azure portal to view the log for your VM.
Some of the things you can do with the activity log include:
- Create an alert on an Activity Log event.
- Stream it to an Event Hub for ingestion by a third-party service or custom analytics solution such as PowerBI.
- Analyze it in PowerBI using the PowerBI content pack.
- Save it to a storage account for archival or manual inspection. You can specify the retention time (in days) using the Log Profile.
You can also access activity log data by using Azure PowerShell, the Azure CLI, or Monitor REST APIs.
What is Azure Diagnostic Logs?
Azure Diagnostic Logs are logs emitted by your VM that provide rich, frequent data about its operation. Diagnostic logs differ from the activity log by providing insight about operations that were performed within the VM.
Some of the things you can do with diagnostics logs include:
- Save them to a storage account for auditing or manual inspection. You can specify the retention time (in days) using Resource Diagnostic Settings.
- Stream them to Event Hubs for ingestion by a third-party service or custom analytics solution such as PowerBI.
- Analyze them with Log Analytics.
1. Azure Monitor is a service that monitors your cloud and on-premises environments to maintain their availability and performance. It delivers a comprehensive solution for collecting, analyzing, and acting on telemetry from your cloud and on-premises environments. It helps you understand how your applications are performing and proactively identifies issues affecting them and the resources they depend on. You can install an extension on a Linux VM or a Windows VM that installs the Log Analytics agent to collect log data and store in a Log Analytics workspace.
For Windows and Linux VMs, the recommended method for collecting logs is by installing the Log Analytics agent. The easiest way to install the Log Analytics agent on a VM is through the Log Analytics VM Extension. Using the extension simplifies the installation process and automatically configures the agent to send data to the Log Analytics workspace that you specify. The agent is also upgraded automatically, ensuring that you have the latest features and fixes.
2. Network Watcher enables you to monitor your VM and its associated resources as they relate to the network that they are in. You can install the Network Watcher Agent extension on a Linux VM or a Windows VM.
3. Azure Monitor for VMs monitors your Azure virtual machines (VM) at scale by analyzing the performance and health of your Windows and Linux VMs, including their different processes and interconnected dependencies on other resources and external processes.
5) Configure networking for VM:
Networking:
What is Virtual Networks?
Azure Virtual Network enables many types of Azure resources, such as Azure Virtual Machines (VM), to securely communicate with each other, the internet, and on-premises networks. A virtual network is scoped to a single region; however, multiple virtual networks from different regions can be connected together using Virtual Network Peering.
What is common networking architecture for application having Front End (web server) and Backend (database server)?
You can create a VNet before you create a VM or you can as you create a VM. You create these resources to support communication with a VM:
In addition to those basic resources, you should also consider these optional resources:
What is Network interfaces?
A network interface (NIC) is the interconnection between a VM and a virtual network (VNet). A VM must have at least one NIC, but can have more than one, depending on the size of the VM you create.
You can create a VM with multiple NICs, and add or remove NICs through the lifecycle of a VM. Multiple NICs allow a VM to connect to different subnets and send or receive traffic over the most appropriate interface. VMs with any number of network interfaces can exist in the same availability set, up to the number supported by the VM size.
Each NIC attached to a VM must exist in the same location and subscription as the VM. Each NIC must be connected to a VNet that exists in the same Azure location and subscription as the NIC. You can change the subnet a VM is connected to after it's created, but you cannot change the VNet. Each NIC attached to a VM is assigned a MAC address that doesn’t change until the VM is deleted.
Note: When you create a VM in the Azure portal, a network interface is automatically created for you (you cannot use a NIC you create separately). The portal creates a VM with only one NIC. If you want to create a VM with more than one NIC, you must create it with a different method.
What is IP Addresses?
Public IP addresses - Used to communicate inbound and outbound (without network address translation (NAT)) with the Internet and other Azure resources not connected to a VNet. Assigning a public IP address to a NIC is optional. Public IP addresses have a nominal charge, and there's a maximum number that can be used per subscription.
Private IP addresses - Used for communication within a VNet, your on-premises network, and the Internet (with NAT). You must assign at least one private IP address to a VM.
WHat is dynamic and static IP's?
There are two methods in which an IP address is allocated to a resource - dynamic or static. The default allocation method is dynamic, where an IP address is not allocated when it's created. Instead, the IP address is allocated when you create a VM or start a stopped VM. The IP address is released when you stop or delete the VM.
To ensure the IP address for the VM remains the same, you can set the allocation method explicitly to static. In this case, an IP address is assigned immediately. It is released only when you delete the VM or change its allocation method to dynamic.
What is VNet(Virtual Network) and Subnet?
A subnet is a range of IP addresses in the VNet. You can divide a VNet into multiple subnets for organization and security. Each NIC in a VM is connected to one subnet in one VNet. NICs connected to subnets (same or different) within a VNet can communicate with each other without any extra configuration.
How the IP allocation happens?
When you set up a VNet, you specify the topology, including the available address spaces and subnets. If the VNet is to be connected to other VNets or on-premises networks, you must select address ranges that don't overlap. The IP addresses are private and can't be accessed from the Internet, which was true only for the non-routable IP addresses such as 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16. Now, Azure treats any address range as part of the private VNet IP address space that is only reachable within the VNet, within interconnected VNets, and from your on-premises location.
If you work within an organization in which someone else is responsible for the internal networks, you should talk to that person before selecting your address space. Make sure there is no overlap and let them know the space you want to use so they don’t try to use the same range of IP addresses.
By default, there is no security boundary between subnets, so VMs in each of these subnets can talk to one another. However, you can set up Network Security Groups (NSGs), which allow you to control the traffic flow to and from subnets and to and from VMs.
Note:
If you let Azure create a VNet when you create a VM, the name is a combination of the resource group name that contains the VNet and -vnet. The address space is 10.0.0.0/24, the required subnet name is default, and the subnet address range is 10.0.0.0/24.
What is Network Security Groups (NSGs)?
A network security group (NSG) contains a list of Access Control List (ACL) rules that allow or deny network traffic to subnets, NICs, or both. NSGs can be associated with either subnets or individual NICs connected to a subnet. When an NSG is associated with a subnet, the ACL rules apply to all the VMs in that subnet. In addition, traffic to an individual NIC can be restricted by associating an NSG directly to a NIC.
NSGs contain two sets of rules: inbound and outbound. The priority for a rule must be unique within each set. Each rule has properties of protocol, source and destination port ranges, address prefixes, direction of traffic, priority, and access type.
How rules are configured in NSGs?
All NSGs contain a set of default rules. The default rules cannot be deleted, but because they are assigned the lowest priority, they can be overridden by the rules that you create.
When you associate an NSG to a NIC, the network access rules in the NSG are applied only to that NIC. If an NSG is applied to a single NIC on a multi-NIC VM, it does not affect traffic to the other NICs. You can associate different NSGs to a NIC (or VM, depending on the deployment model) and the subnet that a NIC or VM is bound to. Priority is given based on the direction of traffic.
Note: When you create a VM in the Azure portal, an NSG is automatically created and associated to the NIC the portal creates. The name of the NSG is a combination of the name of the VM and -nsg. This NSG contains one inbound rule with a priority of 1000, service set to RDP, the protocol set to TCP, port set to 3389, and action set to Allow. If you want to allow any other inbound traffic to the VM, you must add additional rules to the NSG.
What is Load Balancer?
Azure Load Balancer delivers high availability and network performance to your applications. A load balancer can be configured to balance incoming Internet traffic to VMs or balance traffic between VMs in a VNet. A load balancer can also balance traffic between on-premises computers and VMs in a cross-premises network, or forward external traffic to a specific VM.
The load balancer maps incoming and outgoing traffic between the public IP address and port on the load balancer and the private IP address and port of the VM.
What configuration elements need to considered when you create a load balancer?
When you create a load balancer, you must also consider these configuration elements:
Note: You can't currently create an internet-facing load balancer using the Azure port
How to migrate an existing VM to VNet?
If you create a VM and later want to migrate it into a VNet, it is not a simple configuration change. You must redeploy the VM into the VNet. The easiest way to redeploy is to delete the VM, but not any disks attached to it, and then re-create the VM using the original disks in the VNet.
What is Azure Load Balancer?
An Azure load balancer is a Layer-4 (TCP, UDP) load balancer that provides high availability by distributing incoming traffic among healthy VMs. A load balancer health probe monitors a given port on each VM and only distributes traffic to an operational VM.
You define a front-end IP configuration that contains one or more public IP addresses. This front-end IP configuration allows your load balancer and applications to be accessible over the Internet.
Virtual machines connect to a load balancer using their virtual network interface card (NIC). To distribute traffic to the VMs, a back-end address pool contains the IP addresses of the virtual (NICs) connected to the load balancer.
To control the flow of traffic, you define load balancer rules for specific ports and protocols that map to your VMs.
What is NSG?
A network security group (NSG) contains a list of security rules that allow or deny network traffic to resources connected to Azure Virtual Networks (VNet).
NSG's can be associated at what levels?
NSGs can be associated to subnets or individual network interfaces. When an NSG is associated with a network interface, it applies only the associated VM. When an NSG is associated to a subnet, the rules apply to all resources connected to the subnet.
What are the default rules for NSG's?
All NSGs contain a set of default rules. The default rules cannot be deleted, but because they are assigned the lowest priority, they can be overridden by the rules that you create.
The default rules for NSGs are:
===========
Networking:
What is Virtual Networks?
Azure Virtual Network enables many types of Azure resources, such as Azure Virtual Machines (VM), to securely communicate with each other, the internet, and on-premises networks. A virtual network is scoped to a single region; however, multiple virtual networks from different regions can be connected together using Virtual Network Peering.
What is common networking architecture for application having Front End (web server) and Backend (database server)?
You can create a VNet before you create a VM or you can as you create a VM. You create these resources to support communication with a VM:
- Network interfaces
- IP addresses
- Virtual network and subnets
In addition to those basic resources, you should also consider these optional resources:
- Network security groups
- Load balancers
What is Network interfaces?
A network interface (NIC) is the interconnection between a VM and a virtual network (VNet). A VM must have at least one NIC, but can have more than one, depending on the size of the VM you create.
You can create a VM with multiple NICs, and add or remove NICs through the lifecycle of a VM. Multiple NICs allow a VM to connect to different subnets and send or receive traffic over the most appropriate interface. VMs with any number of network interfaces can exist in the same availability set, up to the number supported by the VM size.
Each NIC attached to a VM must exist in the same location and subscription as the VM. Each NIC must be connected to a VNet that exists in the same Azure location and subscription as the NIC. You can change the subnet a VM is connected to after it's created, but you cannot change the VNet. Each NIC attached to a VM is assigned a MAC address that doesn’t change until the VM is deleted.
Note: When you create a VM in the Azure portal, a network interface is automatically created for you (you cannot use a NIC you create separately). The portal creates a VM with only one NIC. If you want to create a VM with more than one NIC, you must create it with a different method.
What is IP Addresses?
Public IP addresses - Used to communicate inbound and outbound (without network address translation (NAT)) with the Internet and other Azure resources not connected to a VNet. Assigning a public IP address to a NIC is optional. Public IP addresses have a nominal charge, and there's a maximum number that can be used per subscription.
Private IP addresses - Used for communication within a VNet, your on-premises network, and the Internet (with NAT). You must assign at least one private IP address to a VM.
WHat is dynamic and static IP's?
There are two methods in which an IP address is allocated to a resource - dynamic or static. The default allocation method is dynamic, where an IP address is not allocated when it's created. Instead, the IP address is allocated when you create a VM or start a stopped VM. The IP address is released when you stop or delete the VM.
To ensure the IP address for the VM remains the same, you can set the allocation method explicitly to static. In this case, an IP address is assigned immediately. It is released only when you delete the VM or change its allocation method to dynamic.
What is VNet(Virtual Network) and Subnet?
A subnet is a range of IP addresses in the VNet. You can divide a VNet into multiple subnets for organization and security. Each NIC in a VM is connected to one subnet in one VNet. NICs connected to subnets (same or different) within a VNet can communicate with each other without any extra configuration.
How the IP allocation happens?
When you set up a VNet, you specify the topology, including the available address spaces and subnets. If the VNet is to be connected to other VNets or on-premises networks, you must select address ranges that don't overlap. The IP addresses are private and can't be accessed from the Internet, which was true only for the non-routable IP addresses such as 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16. Now, Azure treats any address range as part of the private VNet IP address space that is only reachable within the VNet, within interconnected VNets, and from your on-premises location.
If you work within an organization in which someone else is responsible for the internal networks, you should talk to that person before selecting your address space. Make sure there is no overlap and let them know the space you want to use so they don’t try to use the same range of IP addresses.
By default, there is no security boundary between subnets, so VMs in each of these subnets can talk to one another. However, you can set up Network Security Groups (NSGs), which allow you to control the traffic flow to and from subnets and to and from VMs.
Note:
If you let Azure create a VNet when you create a VM, the name is a combination of the resource group name that contains the VNet and -vnet. The address space is 10.0.0.0/24, the required subnet name is default, and the subnet address range is 10.0.0.0/24.
What is Network Security Groups (NSGs)?
A network security group (NSG) contains a list of Access Control List (ACL) rules that allow or deny network traffic to subnets, NICs, or both. NSGs can be associated with either subnets or individual NICs connected to a subnet. When an NSG is associated with a subnet, the ACL rules apply to all the VMs in that subnet. In addition, traffic to an individual NIC can be restricted by associating an NSG directly to a NIC.
NSGs contain two sets of rules: inbound and outbound. The priority for a rule must be unique within each set. Each rule has properties of protocol, source and destination port ranges, address prefixes, direction of traffic, priority, and access type.
How rules are configured in NSGs?
All NSGs contain a set of default rules. The default rules cannot be deleted, but because they are assigned the lowest priority, they can be overridden by the rules that you create.
When you associate an NSG to a NIC, the network access rules in the NSG are applied only to that NIC. If an NSG is applied to a single NIC on a multi-NIC VM, it does not affect traffic to the other NICs. You can associate different NSGs to a NIC (or VM, depending on the deployment model) and the subnet that a NIC or VM is bound to. Priority is given based on the direction of traffic.
Note: When you create a VM in the Azure portal, an NSG is automatically created and associated to the NIC the portal creates. The name of the NSG is a combination of the name of the VM and -nsg. This NSG contains one inbound rule with a priority of 1000, service set to RDP, the protocol set to TCP, port set to 3389, and action set to Allow. If you want to allow any other inbound traffic to the VM, you must add additional rules to the NSG.
What is Load Balancer?
Azure Load Balancer delivers high availability and network performance to your applications. A load balancer can be configured to balance incoming Internet traffic to VMs or balance traffic between VMs in a VNet. A load balancer can also balance traffic between on-premises computers and VMs in a cross-premises network, or forward external traffic to a specific VM.
The load balancer maps incoming and outgoing traffic between the public IP address and port on the load balancer and the private IP address and port of the VM.
What configuration elements need to considered when you create a load balancer?
When you create a load balancer, you must also consider these configuration elements:
- Front-end IP configuration – A load balancer can include one or more front-end IP addresses, otherwise known as virtual IPs (VIPs). These IP addresses serve as ingress for the traffic.
- Back-end address pool – IP addresses that are associated with the NIC to which load is distributed.
- NAT rules - Defines how inbound traffic flows through the front-end IP and distributed to the back-end IP.
- Load balancer rules - Maps a given front-end IP and port combination to a set of back-end IP addresses and port combination. A single load balancer can have multiple load balancing rules. Each rule is a combination of a front-end IP and port and back-end IP and port associated with VMs.
- Probes - Monitors the health of VMs. When a probe fails to respond, the load balancer stops sending new connections to the unhealthy VM. The existing connections are not affected, and new connections are sent to healthy VMs.
Note: You can't currently create an internet-facing load balancer using the Azure port
How to migrate an existing VM to VNet?
If you create a VM and later want to migrate it into a VNet, it is not a simple configuration change. You must redeploy the VM into the VNet. The easiest way to redeploy is to delete the VM, but not any disks attached to it, and then re-create the VM using the original disks in the VNet.
What is Azure Load Balancer?
An Azure load balancer is a Layer-4 (TCP, UDP) load balancer that provides high availability by distributing incoming traffic among healthy VMs. A load balancer health probe monitors a given port on each VM and only distributes traffic to an operational VM.
You define a front-end IP configuration that contains one or more public IP addresses. This front-end IP configuration allows your load balancer and applications to be accessible over the Internet.
Virtual machines connect to a load balancer using their virtual network interface card (NIC). To distribute traffic to the VMs, a back-end address pool contains the IP addresses of the virtual (NICs) connected to the load balancer.
To control the flow of traffic, you define load balancer rules for specific ports and protocols that map to your VMs.
What is NSG?
A network security group (NSG) contains a list of security rules that allow or deny network traffic to resources connected to Azure Virtual Networks (VNet).
NSG's can be associated at what levels?
NSGs can be associated to subnets or individual network interfaces. When an NSG is associated with a network interface, it applies only the associated VM. When an NSG is associated to a subnet, the rules apply to all resources connected to the subnet.
What are the default rules for NSG's?
All NSGs contain a set of default rules. The default rules cannot be deleted, but because they are assigned the lowest priority, they can be overridden by the rules that you create.
The default rules for NSGs are:
- Virtual network - Traffic originating and ending in a virtual network is allowed both in inbound and outbound directions.
- Internet - Outbound traffic is allowed, but inbound traffic is blocked.
- Load balancer - Allow Azure’s load balancer to probe the health of your VMs and role instances. If you are not using a load balanced set, you can override this rule.
===========
6) Configure Storage for VM:
What is Azure Managed Disks?
An Azure managed disk is a virtual hard disk (VHD). You can think of it like a physical disk in an on-premises server but, virtualized. Azure managed disks are stored as page blobs, which are a random IO storage object in Azure. We call a managed disk ‘managed’ because it is an abstraction over page blobs, blob containers, and Azure storage accounts. With managed disks, all you have to do is provision the disk, and Azure takes care of the rest.
What are the various types of disks?
The available types of disks are Ultra Solid State Drives (SSD), Premium SSD, Standard SSD, and Standard Hard Disk Drives (HDD).
What are the benefits of managed disks?
a) Highly durable and available:
Managed disks are designed for 99.999% availability. Managed disks achieve this by providing you with three replicas of your data, allowing for high durability. If one or even two replicas experience issues, the remaining replicas help ensure persistence of your data and high tolerance against failures.
b) Simple and scalable VM deployment:
Using managed disks, you can create up to 50,000 VM disks of a type in a subscription per region, allowing you to create thousands of VMs in a single subscription. This feature also further increases the scalability of virtual machine scale sets by allowing you to create up to 1,000 VMs in a virtual machine scale set using a Marketplace image.
c) Integration with availability sets:
Managed disks are integrated with availability sets to ensure that the disks of VMs in an availability set are sufficiently isolated from each other to avoid a single point of failure. Disks are automatically placed in different storage scale units (stamps). If a stamp fails due to hardware or software failure, only the VM instances with disks on those stamps fail. For example, let's say you have an application running on five VMs, and the VMs are in an Availability Set. The disks for those VMs won't all be stored in the same stamp, so if one stamp goes down, the other instances of the application continue to run.
d) Azure Backup support:
To protect against regional disasters, Azure Backup can be used to create a backup job with time-based backups and backup retention policies. This allows you to perform easy VM restorations at will. Currently Azure Backup supports disk sizes up to four tebibyte (TiB) disks.
e) Granular access control
You can use Azure role-based access control (RBAC) to assign specific permissions for a managed disk to one or more users. Managed disks expose a variety of operations, including read, write (create/update), delete, and retrieving a shared access signature (SAS) URI for the disk. You can grant access to only the operations a person needs to perform their job.
What are the various Disk roles available?
There are 3 disk roles available - 1) Data disks 2) OS disks 3) Temporary disks
What is Data disks?
A data disk is a managed disk that's attached to a virtual machine to store application data, or other data you need to keep. Data disks are registered as SCSI drives and are labeled with a letter that you choose. Each data disk has a maximum capacity of 4,095 gibibytes (GiB). The size of the virtual machine determines how many data disks you can attach to it and the type of storage you can use to host the disks.
What is OS disks?
Every virtual machine has one attached operating system disk. That OS disk has a pre-installed OS, which was selected when the VM was created.
This disk has a maximum capacity of 2,048 GiB.
What is Temporary disk?
Every VM contains a temporary disk, which is not a managed disk. The temporary disk provides short-term storage for applications and processes and is intended to only store data such as page or swap files. Data on the temporary disk may be lost during a maintenance event event or when you redeploy a VM. During a successful standard reboot of the VM, the data on the temporary drive will persist.
What is (Managed disk) snapshots?
A managed disk snapshot is a read-only full copy of a managed disk that is stored as a standard managed disk by default. With snapshots, you can back up your managed disks at any point in time. These snapshots exist independent of the source disk and can be used to create new managed disks.
What is image?
This image contains all managed disks associated with a VM, including both the OS and data disks. This managed custom image enables creating hundreds of VMs using your custom image without the need to copy or manage any storage accounts.
What is difference between image vs snapshots?
Image of a VM includes all the disks associated (both OS and data disks) where as snapshot applies only to one disk.
Snapshot is problematic to use in scenarios that require the coordination of multiple disks, such as striping, where as image is not.
What disk types are available in Azure?
Azure has 4 disk types:
1) Ultra Solid-state-drives (SSD) - In preview, as of writing this blog.
2) Premium SSD
3) Standard SSD
4) Standard hard disk drives (HDD)
What is the comparison for various disk types in Azure?
The following is the comparison for 4 disk types:
How is the billing done for managed disks?
When using managed disks, the following billing considerations apply:
What is Azure Managed Disks?
An Azure managed disk is a virtual hard disk (VHD). You can think of it like a physical disk in an on-premises server but, virtualized. Azure managed disks are stored as page blobs, which are a random IO storage object in Azure. We call a managed disk ‘managed’ because it is an abstraction over page blobs, blob containers, and Azure storage accounts. With managed disks, all you have to do is provision the disk, and Azure takes care of the rest.
What are the various types of disks?
The available types of disks are Ultra Solid State Drives (SSD), Premium SSD, Standard SSD, and Standard Hard Disk Drives (HDD).
a) Highly durable and available:
Managed disks are designed for 99.999% availability. Managed disks achieve this by providing you with three replicas of your data, allowing for high durability. If one or even two replicas experience issues, the remaining replicas help ensure persistence of your data and high tolerance against failures.
b) Simple and scalable VM deployment:
Using managed disks, you can create up to 50,000 VM disks of a type in a subscription per region, allowing you to create thousands of VMs in a single subscription. This feature also further increases the scalability of virtual machine scale sets by allowing you to create up to 1,000 VMs in a virtual machine scale set using a Marketplace image.
c) Integration with availability sets:
Managed disks are integrated with availability sets to ensure that the disks of VMs in an availability set are sufficiently isolated from each other to avoid a single point of failure. Disks are automatically placed in different storage scale units (stamps). If a stamp fails due to hardware or software failure, only the VM instances with disks on those stamps fail. For example, let's say you have an application running on five VMs, and the VMs are in an Availability Set. The disks for those VMs won't all be stored in the same stamp, so if one stamp goes down, the other instances of the application continue to run.
d) Azure Backup support:
To protect against regional disasters, Azure Backup can be used to create a backup job with time-based backups and backup retention policies. This allows you to perform easy VM restorations at will. Currently Azure Backup supports disk sizes up to four tebibyte (TiB) disks.
e) Granular access control
You can use Azure role-based access control (RBAC) to assign specific permissions for a managed disk to one or more users. Managed disks expose a variety of operations, including read, write (create/update), delete, and retrieving a shared access signature (SAS) URI for the disk. You can grant access to only the operations a person needs to perform their job.
What are the various Disk roles available?
There are 3 disk roles available - 1) Data disks 2) OS disks 3) Temporary disks
What is Data disks?
A data disk is a managed disk that's attached to a virtual machine to store application data, or other data you need to keep. Data disks are registered as SCSI drives and are labeled with a letter that you choose. Each data disk has a maximum capacity of 4,095 gibibytes (GiB). The size of the virtual machine determines how many data disks you can attach to it and the type of storage you can use to host the disks.
What is OS disks?
Every virtual machine has one attached operating system disk. That OS disk has a pre-installed OS, which was selected when the VM was created.
This disk has a maximum capacity of 2,048 GiB.
What is Temporary disk?
Every VM contains a temporary disk, which is not a managed disk. The temporary disk provides short-term storage for applications and processes and is intended to only store data such as page or swap files. Data on the temporary disk may be lost during a maintenance event event or when you redeploy a VM. During a successful standard reboot of the VM, the data on the temporary drive will persist.
What is (Managed disk) snapshots?
A managed disk snapshot is a read-only full copy of a managed disk that is stored as a standard managed disk by default. With snapshots, you can back up your managed disks at any point in time. These snapshots exist independent of the source disk and can be used to create new managed disks.
What is image?
This image contains all managed disks associated with a VM, including both the OS and data disks. This managed custom image enables creating hundreds of VMs using your custom image without the need to copy or manage any storage accounts.
What is difference between image vs snapshots?
Image of a VM includes all the disks associated (both OS and data disks) where as snapshot applies only to one disk.
Snapshot is problematic to use in scenarios that require the coordination of multiple disks, such as striping, where as image is not.
What disk types are available in Azure?
Azure has 4 disk types:
1) Ultra Solid-state-drives (SSD) - In preview, as of writing this blog.
2) Premium SSD
3) Standard SSD
4) Standard hard disk drives (HDD)
What is the comparison for various disk types in Azure?
The following is the comparison for 4 disk types:
How is the billing done for managed disks?
When using managed disks, the following billing considerations apply:
- Disk type (Ultra SSD, Premium SSD, Standard SSD, Standard HDD)
- managed disk Size
- Snapshots
- Outbound data transfers (data going out of Azure data centers)
- Number of transactions
What are the Application performance indicators (from Storage Layer perspective) ?
IOPS (Input/output Operations Per Second) , Throughput and Latency are called application performance indicators.
What is IOPS (from Storage Layer perspective)?
IOPS, or Input/output Operations Per Second, is the number of requests that your application is sending to the storage disks in one second. An input/output operation could be read or write, sequential, or random. Ex: Online Transaction Processing (OLTP) applications like an online retail website need to process many concurrent user requests immediately. The user requests are insert and update intensive database transactions, which the application must process quickly. Therefore, OLTP applications require very high IOPS.
What is Throughput (from Storage Layer perspective)?
Throughput, or bandwidth is the amount of data that your application is sending to the storage disks in a specified interval. If your application is performing input/output operations with large IO unit sizes, it requires high throughput. Ex: Data warehouse applications tend to issue scan intensive operations that access large portions of data at a time and commonly perform bulk operations. In other words, such applications require higher throughput.
What is the relation between IOPS and throughput?
IOPS X IO Size = Throughput
As you try to optimize one, the other also gets affected. Therefore, it is important to determine the optimal throughput and IOPS values that your application requires.
What is Latency (from Storage Layer perspective)?
Latency is the time it takes an application to receive a single request, send it to the storage disks and send the response to the client. The Latency of a premium storage disk is the time it takes to retrieve the information for a request and communicate it back to your application. Premium Storage provides consistent low latencies. If you enable ReadOnly host caching on premium storage disks, you can get much lower read latency. Ex: Extremely low Latency is crucial for real-time applications like live video streaming websites.
What the Performance Application checklist for disks?
1. Understand the performance requirements of your application. This includes identifying which of the performance indicators (like IOPS, Throughput and Latency) are critical to your application
2. Optimize the application to achieve optimal performance
3. Measure the maximum performance requirements of your application throughout its lifetime. Record the maximum performance requirements during normal, peak, and off-hours workload periods.
4. Find out the 50 percentile, 90 percentile, and 99 percentile requirements. This helps filter out any outliers in the performance requirements and you can focus your efforts on optimizing for the right values.
What are the Counters to measure application performance requirements?
The best way to measure performance requirements of your application, is to use performance-monitoring tools provided by the operating system of the server. You can use PerfMon for Windows and iostat for Linux. The PerfMon counters are available for processor, memory and, each logical disk and physical disk of your server. When you use premium storage disks with a VM, the physical disk counters are for each premium storage disk, and logical disk counters are for each volume created on the premium storage disks. You must capture the values for the disks that host your application workload.
What are the main factors that influence application performance (from Storage perspective)?
The main factors that influence performance of an application running on Premium Storage are:
How "Nature of IO requests" will affect the application peformance (from Storage perspective)?
An IO request is a unit of input/output operation that your application will be performing. Identifying the nature of IO requests, random or sequential, read or write, small or large, will help you determine the performance requirements of your application. It is important to understand the nature of IO requests, to make the right decisions when designing your application infrastructure.
IO size is one of the more important factors. The IO size is the size of the input/output operation request generated by your application. The IO size has a significant impact on performance especially on the IOPS and Bandwidth that the application is able to achieve.
How "High scale VM sizes" will affect the application peformance (from Storage perspective)?
Premium Storage comes with High Scale VM sizes that can run applications requiring higher compute power and a high local disk I/O performance. These VMs provide faster processors, a higher memory-to-core ratio, and a Solid-State Drive (SSD) for the local disk. Examples of High Scale VMs supporting Premium Storage are the DS, DSv2, and GS series VMs.
High Scale VMs are available in different sizes with a different number of CPU cores, memory, OS, and temporary disk size. Each VM size also has maximum number of data disks that you can attach to the VM. Therefore, the chosen VM size will affect how much processing, memory, and storage capacity is available for your application. It also affects the Compute and Storage cost.
How "disk sizes" will affect the application peformance (from Storage perspective)?
Azure offers various premium storage disk sizes (P4 to P50) ranging from different disk size, IOPS, Throughput. How many disks you choose depends on the disk size chosen. You could use a single P50 disk or multiple P10 disks to meet your application requirement.
How "Disk caching" will affect the application peformance (from Storage perspective)?
High Scale VMs that leverage Azure Premium Storage have a multi-tier caching technology called BlobCache. BlobCache uses a combination of the Virtual Machine RAM and local SSD for caching. This cache is available for the Premium Storage persistent disks and the VM local disks. By default, this cache setting is set to Read/Write for OS disks and ReadOnly for data disks hosted on Premium Storage. With disk caching enabled on the Premium Storage disks, the high scale VMs can achieve extremely high levels of performance that exceed the underlying disk performance.
What is alternative solution to Azure Backup?
Consistent snapshots are alternatives to Azure Backup. Creating consistent snapshots for all the disks used by a VM and then replicating those snapshots to another region (which is complicated though).
If you use read-access geo-redundant storage/geo-redundant storage for disks, snapshots are automatically replicated to a secondary datacenter. If you use locally redundant storage for disks, you need to replicate the data yourself.
What are the steps to follow to create a snapshot while the VM is running?
Although you can take a snapshot at any time, if the VM is running, there is still data being streamed to the disks. The snapshots might contain partial operations that were in flight. Also, if there are several disks involved, the snapshots of different disks might have occurred at different times. These scenarios may cause to the snapshots to be uncoordinated.
To avoid this situation, the backup process must implement the following steps:
IOPS (Input/output Operations Per Second) , Throughput and Latency are called application performance indicators.
What is IOPS (from Storage Layer perspective)?
IOPS, or Input/output Operations Per Second, is the number of requests that your application is sending to the storage disks in one second. An input/output operation could be read or write, sequential, or random. Ex: Online Transaction Processing (OLTP) applications like an online retail website need to process many concurrent user requests immediately. The user requests are insert and update intensive database transactions, which the application must process quickly. Therefore, OLTP applications require very high IOPS.
What is Throughput (from Storage Layer perspective)?
Throughput, or bandwidth is the amount of data that your application is sending to the storage disks in a specified interval. If your application is performing input/output operations with large IO unit sizes, it requires high throughput. Ex: Data warehouse applications tend to issue scan intensive operations that access large portions of data at a time and commonly perform bulk operations. In other words, such applications require higher throughput.
What is the relation between IOPS and throughput?
IOPS X IO Size = Throughput
As you try to optimize one, the other also gets affected. Therefore, it is important to determine the optimal throughput and IOPS values that your application requires.
What is Latency (from Storage Layer perspective)?
Latency is the time it takes an application to receive a single request, send it to the storage disks and send the response to the client. The Latency of a premium storage disk is the time it takes to retrieve the information for a request and communicate it back to your application. Premium Storage provides consistent low latencies. If you enable ReadOnly host caching on premium storage disks, you can get much lower read latency. Ex: Extremely low Latency is crucial for real-time applications like live video streaming websites.
What the Performance Application checklist for disks?
1. Understand the performance requirements of your application. This includes identifying which of the performance indicators (like IOPS, Throughput and Latency) are critical to your application
2. Optimize the application to achieve optimal performance
3. Measure the maximum performance requirements of your application throughout its lifetime. Record the maximum performance requirements during normal, peak, and off-hours workload periods.
4. Find out the 50 percentile, 90 percentile, and 99 percentile requirements. This helps filter out any outliers in the performance requirements and you can focus your efforts on optimizing for the right values.
What are the Counters to measure application performance requirements?
The best way to measure performance requirements of your application, is to use performance-monitoring tools provided by the operating system of the server. You can use PerfMon for Windows and iostat for Linux. The PerfMon counters are available for processor, memory and, each logical disk and physical disk of your server. When you use premium storage disks with a VM, the physical disk counters are for each premium storage disk, and logical disk counters are for each volume created on the premium storage disks. You must capture the values for the disks that host your application workload.
What are the main factors that influence application performance (from Storage perspective)?
The main factors that influence performance of an application running on Premium Storage are:
- Nature of IO requests
- VM size
- Disk size
- Number of disks
- Disk caching
- Multithreading
- Queue depth
How "Nature of IO requests" will affect the application peformance (from Storage perspective)?
An IO request is a unit of input/output operation that your application will be performing. Identifying the nature of IO requests, random or sequential, read or write, small or large, will help you determine the performance requirements of your application. It is important to understand the nature of IO requests, to make the right decisions when designing your application infrastructure.
IO size is one of the more important factors. The IO size is the size of the input/output operation request generated by your application. The IO size has a significant impact on performance especially on the IOPS and Bandwidth that the application is able to achieve.
How "High scale VM sizes" will affect the application peformance (from Storage perspective)?
Premium Storage comes with High Scale VM sizes that can run applications requiring higher compute power and a high local disk I/O performance. These VMs provide faster processors, a higher memory-to-core ratio, and a Solid-State Drive (SSD) for the local disk. Examples of High Scale VMs supporting Premium Storage are the DS, DSv2, and GS series VMs.
High Scale VMs are available in different sizes with a different number of CPU cores, memory, OS, and temporary disk size. Each VM size also has maximum number of data disks that you can attach to the VM. Therefore, the chosen VM size will affect how much processing, memory, and storage capacity is available for your application. It also affects the Compute and Storage cost.
How "disk sizes" will affect the application peformance (from Storage perspective)?
Azure offers various premium storage disk sizes (P4 to P50) ranging from different disk size, IOPS, Throughput. How many disks you choose depends on the disk size chosen. You could use a single P50 disk or multiple P10 disks to meet your application requirement.
How "Disk caching" will affect the application peformance (from Storage perspective)?
High Scale VMs that leverage Azure Premium Storage have a multi-tier caching technology called BlobCache. BlobCache uses a combination of the Virtual Machine RAM and local SSD for caching. This cache is available for the Premium Storage persistent disks and the VM local disks. By default, this cache setting is set to Read/Write for OS disks and ReadOnly for data disks hosted on Premium Storage. With disk caching enabled on the Premium Storage disks, the high scale VMs can achieve extremely high levels of performance that exceed the underlying disk performance.
What is Resiliency?
Resiliency refers to the tolerance for normal failures that occur in hardware components. Resiliency is the ability to recover from failures and continue to function. It's not about avoiding failures, but responding to failures in a way that avoids downtime or data loss. The goal of resiliency is to return the application to a fully functioning state following a failure.
How Azure supports resiliency with VM's?
A virtual machine consists mainly of two parts: a compute server and the persistent disks. Both affect the fault tolerance of a virtual machine.
Compute:
1) Automatic restore of VM
2) Availability set (two or more VM's in separate fault domains with different power, network, and server components)
3) Azure SLA coverage
Disks:
1) Three redundant copies of the data stored locally (automatic span of new copy)
What is Disaster Recovery?
Disaster recovery is the ability to recover from rare, but major, incidents and large-scale outages like catastrophic events (such as if a datacenter is hit by a hurricane, earthquake, fire, or if there is a large-scale hardware unit failure). These incidents include non-transient, wide-scale failures, such as service disruption that affects an entire region. Disaster recovery includes data backup and archiving, and might include manual intervention, such as restoring a database from a backup.
What should the Disaster Recovery considerations?
Primarily, two things:
1) High availability:
The ability of the application to continue running in a healthy state, without significant downtime. By healthy state, this state means that the application is responsive, and users can connect to the application and interact with it.
2) Data Durability:
In some cases, the main consideration is ensuring that the data is preserved if a disaster happens. Therefore, you might need a backup of your data in a different site.
What are the Disaster recovery solutions from Azure?
1) Azure Backup
2) Consistent Snapshots
The above two are in-addition to the Automatic replication options like Locally Redundant Storage (Local), Geo-Redundant Storage (cross-region), Read-access geo-redundant storage (cross-region)
DR at Storage levels:
DR at Application or Infra levels:
What is Azure Backup?
Azure Backup can back up your VMs running Windows or Linux to the Azure recovery services vault. Backing up and restoring business-critical data is complicated by the fact that business-critical data must be backed up while the applications that produce the data are running.
To address this issue, Azure Backup provides application-consistent backups for Microsoft workloads. It uses the volume shadow service to ensure that data is written correctly to storage. For Linux VMs, only file-consistent backups are possible, because Linux does not have functionality equivalent to the volume shadow service.
How does the Azure Backup process work?
1. When Azure Backup initiates a backup job at the scheduled time, it triggers the backup extension installed in the VM to take a point-in-time snapshot.
2. A snapshot is taken in coordination with the volume shadow service to get a consistent snapshot of the disks in the virtual machine without having to shut it down.
3. The backup extension in the VM flushes all writes before taking a consistent snapshot of all of the disks.
4. After taking the snapshot, the data is transferred by Azure Backup to the backup vault. To make the backup process more efficient, the service identifies and transfers only the blocks of data that have changed after the last backup. When the data transfer is complete, the snapshot is removed, and a recovery point is created.
5. To restore, you can view the available backups through Azure Backup and then initiate a restore. You can create and restore Azure backups through the Azure portal, by using PowerShell, or by using the Azure CLI.
Resiliency refers to the tolerance for normal failures that occur in hardware components. Resiliency is the ability to recover from failures and continue to function. It's not about avoiding failures, but responding to failures in a way that avoids downtime or data loss. The goal of resiliency is to return the application to a fully functioning state following a failure.
How Azure supports resiliency with VM's?
A virtual machine consists mainly of two parts: a compute server and the persistent disks. Both affect the fault tolerance of a virtual machine.
Compute:
1) Automatic restore of VM
2) Availability set (two or more VM's in separate fault domains with different power, network, and server components)
3) Azure SLA coverage
Disks:
1) Three redundant copies of the data stored locally (automatic span of new copy)
What is Disaster Recovery?
Disaster recovery is the ability to recover from rare, but major, incidents and large-scale outages like catastrophic events (such as if a datacenter is hit by a hurricane, earthquake, fire, or if there is a large-scale hardware unit failure). These incidents include non-transient, wide-scale failures, such as service disruption that affects an entire region. Disaster recovery includes data backup and archiving, and might include manual intervention, such as restoring a database from a backup.
What should the Disaster Recovery considerations?
Primarily, two things:
1) High availability:
The ability of the application to continue running in a healthy state, without significant downtime. By healthy state, this state means that the application is responsive, and users can connect to the application and interact with it.
2) Data Durability:
In some cases, the main consideration is ensuring that the data is preserved if a disaster happens. Therefore, you might need a backup of your data in a different site.
What are the Disaster recovery solutions from Azure?
1) Azure Backup
2) Consistent Snapshots
The above two are in-addition to the Automatic replication options like Locally Redundant Storage (Local), Geo-Redundant Storage (cross-region), Read-access geo-redundant storage (cross-region)
DR at Storage levels:
What is Azure Backup?
Azure Backup can back up your VMs running Windows or Linux to the Azure recovery services vault. Backing up and restoring business-critical data is complicated by the fact that business-critical data must be backed up while the applications that produce the data are running.
To address this issue, Azure Backup provides application-consistent backups for Microsoft workloads. It uses the volume shadow service to ensure that data is written correctly to storage. For Linux VMs, only file-consistent backups are possible, because Linux does not have functionality equivalent to the volume shadow service.
How does the Azure Backup process work?
1. When Azure Backup initiates a backup job at the scheduled time, it triggers the backup extension installed in the VM to take a point-in-time snapshot.
2. A snapshot is taken in coordination with the volume shadow service to get a consistent snapshot of the disks in the virtual machine without having to shut it down.
3. The backup extension in the VM flushes all writes before taking a consistent snapshot of all of the disks.
4. After taking the snapshot, the data is transferred by Azure Backup to the backup vault. To make the backup process more efficient, the service identifies and transfers only the blocks of data that have changed after the last backup. When the data transfer is complete, the snapshot is removed, and a recovery point is created.
5. To restore, you can view the available backups through Azure Backup and then initiate a restore. You can create and restore Azure backups through the Azure portal, by using PowerShell, or by using the Azure CLI.
What is alternative solution to Azure Backup?
Consistent snapshots are alternatives to Azure Backup. Creating consistent snapshots for all the disks used by a VM and then replicating those snapshots to another region (which is complicated though).
If you use read-access geo-redundant storage/geo-redundant storage for disks, snapshots are automatically replicated to a secondary datacenter. If you use locally redundant storage for disks, you need to replicate the data yourself.
What are the steps to follow to create a snapshot while the VM is running?
Although you can take a snapshot at any time, if the VM is running, there is still data being streamed to the disks. The snapshots might contain partial operations that were in flight. Also, if there are several disks involved, the snapshots of different disks might have occurred at different times. These scenarios may cause to the snapshots to be uncoordinated.
To avoid this situation, the backup process must implement the following steps:
- Freeze all the disks.
- Flush all the pending writes.
- Create a blob snapshot for all the disks.
7) Configure VM Size:
What are the various categories of Azure Virtual machine?
The following are various types of VM's available:
1. General Purpose
2. Compute optimized
3. Memory optimized
4. Storage optimized
5. GPU
6. High performance compute
What is the size and purpose of each type of VM?
What is Azure Compute Unit (ACU)?
The concept of the Azure Compute Unit (ACU) provides a way of comparing compute (CPU) performance across Azure SKUs. This will help you easily identify which SKU is most likely to satisfy your performance needs. ACU is currently standardized on a Small (Standard_A1) VM being 100 and all other SKUs then represent approximately how much faster that SKU can run a standard benchmark.
What is General purpose VM sizes?
General purpose VM sizes provide balanced CPU-to-memory ratio. Ideal for testing and development, small to medium databases, and low to medium traffic web servers. B, Dsv3, Dv3, DSv2, Dv2, Av2, DC series VM's are considered as General purpose VM's.
What is DC-Series?
The DC-series is a new family of virtual machines in Azure that can help protect the confidentiality and integrity of your data and code while it’s processed in the public cloud.
What is AV2-Series?
The Av2-series VMs can be deployed on a variety of hardware types and processors. A-series VMs have CPU performance and memory configurations best suited for entry level workloads like development and test. The size is throttled, based upon the hardware, to offer consistent processor performance for the running instance, regardless of the hardware it is deployed on. To determine the physical hardware on which this size is deployed, query the virtual hardware from within the Virtual Machine.
What is Dv2-Series VM's?
Dv2-series, a follow-on to the original D-series, features a more powerful CPU and optimal CPU-to-memory configuration making them suitable for most production workloads. The Dv2-series CPU is about 35% faster than the D-series CPU. The Dv2-series has the same memory and disk configurations as the D-series.
What is Dv3-Series VM's?
The Dv3-series provides better value proposition for most general purpose workloads. Memory has been expanded (from ~3.5 GiB/vCPU to 4 GiB/vCPU) while disk and network limits have been adjusted on a per core basis to align with the move to hyperthreading. The Dv3 no longer has the high memory VM sizes of the D/Dv2 families, those have been moved to the new Ev3 family.
What is Av2-series VM's?
The Av2-series VMs can be deployed on a variety of hardware types and processors. A-series VMs have CPU performance and memory configurations best suited for entry level workloads like development and test. The size is throttled, based upon the hardware, to offer consistent processor performance for the running instance, regardless of the hardware it is deployed on. To determine the physical hardware on which this size is deployed, query the virtual hardware from within the Virtual Machine.
What is B-Series VM's?
The B-series burstable VMs are ideal for workloads that do not need the full performance of the CPU continuously, like web servers, small databases and development and test environments.
Premium Storage: Supported
Premium Storage Caching: Not Supported
What is Dsv3-series VM's?
The Dsv3-series sizes offer a combination of vCPU, memory, and temporary storage for most production workloads.
ACU: 160-190
Premium Storage: Supported
Premium Storage Caching: Supported
What is Dv3-series VM's?
The Dv3-series sizes offer a combination of vCPU, memory, and temporary storage for most production workloads.
ACU: 160-190
Premium Storage: Not Supported
Premium Storage Caching: Not Supported
Data disk storage is billed separately from virtual machines. To use premium storage disks, use the Dsv3 sizes. The pricing and billing meters for Dsv3 sizes are the same as Dv3-series.
What is DSv2-series VM's?
ACU: 210-250
Premium Storage: Supported
Premium Storage Caching: Supported
What is Dv2-series VM's?
ACU: 210-250
Premium Storage: Not Supported
Premium Storage Caching: Not Supported
What is Av2-series VM's?
ACU: 100
Premium Storage: Not Supported
Premium Storage Caching: Not Supported
What is DC-series VM's?
Premium Storage: Supported
Premium Storage Caching: Supported
What is Compute optimized VM sizes?
Compute optimized VM sizes have a high CPU-to-memory ratio and are good for medium traffic web servers, network appliances, batch processes, and application servers. Fsv2, Fs, F series VM's are considered as Compute optimized VM's.
What is Fsv2-series VM's?
Fsv2-series are really fast for any computational workload. At a lower per-hour list price, the Fsv2-series is the best value in price-performance in the Azure portfolio based on the Azure Compute Unit (ACU) per vCPU. The Fs-series provides all the advantages of the F-series, in addition to Premium storage.
ACU: 195 - 210
Premium Storage: Supported
Premium Storage Caching: Supported
What is F-series VM's?
F-series is based on the same CPU performance as the Dv2-series of VMs. F-series VMs are an excellent choice for workloads that demand faster CPUs but do not need as much memory or temporary storage per vCPU. Workloads such as analytics, gaming servers, web servers, and batch processing will benefit from the value of the F-series.
ACU: 210 - 250
Premium Storage: Supported
Premium Storage Caching: Supported
What is F-series VM's?
ACU: 210 - 250
Premium Storage: Not Supported
Premium Storage Caching: Not Supported
What is Memory optimized VM sizes?
Memory optimized VM sizes offer a high memory-to-CPU ratio that are great for relational database servers, medium to large caches, and in-memory analytics. Esv3, Ev3, M, GS, G, Dsv2, Dv2 series VM's are considered as Memory optimized VM's.
What is M-Series VM's?
The M-Series offers the highest vCPU count (up to 128 vCPUs) and largest memory (up to 3.8 TiB) of any VM in the cloud. It’s ideal for extremely large databases or other applications that benefit from high vCPU counts and large amounts of memory.
ACU: 160-180 1
Premium Storage: Supported
Premium Storage Caching: Supported
What is Dv2-Series VM's?
Dv2-series, G-series, and the DSv2/GS counterparts are ideal for applications that demand faster vCPUs, better temporary storage performance, or have higher memory demands. They offer a powerful combination for many enterprise-grade applications. The Dv2-series has the same memory and disk configurations as the D-series.
What is Ev3-Series VM's?
The Ev3-series features at providing a better value proposition for most general purpose workloads, and bringing the Ev3 into alignment with the general purpose VMs of most other clouds. Memory has been expanded (from 7 GiB/vCPU to 8 GiB/vCPU) while disk and network limits have been adjusted on a per core basis to align with the move to hyperthreading. The Ev3 is the follow up to the high memory VM sizes of the D/Dv2 families.
ACU: 160 - 190 1
Premium Storage: Not Supported
Premium Storage Caching: Not Supported
What is Esv3-series VM's?
ACU: 160-190
Premium Storage: Supported
Premium Storage Caching: Supported
What is GS-series VM's?
ACU: 180 - 240
Premium Storage: Supported
Premium Storage Caching: Supported
What is G-series VM's?
ACU: 180 - 240
Premium Storage: Not Supported
Premium Storage Caching: Not Supported
What is DSv2-series 11-15 VM's?
ACU: 210 - 250
Premium Storage: Supported
Premium Storage Caching: Supported
What is Dv2-series 11-15 VM's?
ACU: 210 - 250
Premium Storage: Not Supported
Premium Storage Caching: Not Supported
What is Constrained VCPU capable VM sizes?
Some database workloads like SQL Server or Oracle require high memory, storage, and I/O bandwidth, but not a high core count. Many database workloads are not CPU-intensive. Azure offers certain VM sizes where you can constrain the VM vCPU count to reduce the cost of software licensing, while maintaining the same memory, storage, and I/O bandwidth.
What is Storage optimized virtual machine sizes?
Storage optimized VM sizes offer high disk throughput and IO, and are ideal for Big Data, SQL, NoSQL databases, data warehousing and large transactional databases. Examples include Cassandra, MongoDB, Cloudera and Redis. Lsv2 and Ls series are the VM's come under this category.
What is Lsv2-series VM's?
ACU: 150-175
Premium Storage: Supported
Premium Storage Caching: Not Supported
What is Lsv2-series VM's?
ACU: 150-175
Premium Storage: Supported
Premium Storage Caching: Not Supported
What is GPU optimized virtual machine sizes?
GPU optimized VM sizes are specialized virtual machines available with single or multiple NVIDIA GPUs. These sizes are designed for compute-intensive, graphics-intensive, and visualization workloads.
What is NC-series VM's?
NC-series sizes are optimized for compute-intensive and network-intensive applications and algorithms. Some examples are CUDA- and OpenCL-based applications and simulations, AI, and Deep Learning. The NCv3-series is focused on high-performance computing workloads featuring NVIDIA’s Tesla V100 GPU. The ND-series is focused on training and inference scenarios for deep learning. It uses the NVIDIA Tesla P40 GPU.
Premium Storage: Not Supported
Premium Storage Caching: Not Supported
WHat is NCv2-series VM's?
NCv2-series VM's have GPUs that can provide more than 2x the computational performance of the NC-series. Customers can take advantage of these updated GPUs for traditional HPC workloads such as reservoir modeling, DNA sequencing, protein analysis, Monte Carlo simulations, and others.
Premium Storage: Supported
Premium Storage Caching: Supported
What is NCv3-series VM's?
NCv3-series VMs are powered by NVIDIA Tesla V100 GPUs. These GPUs can provide 1.5x the computational performance of the NCv2-series. Customers can take advantage of these updated GPUs for traditional HPC workloads such as reservoir modeling, DNA sequencing, protein analysis, Monte Carlo simulations, and others. The NC24rs v3 configuration provides a low latency, high-throughput network interface optimized for tightly coupled parallel computing workloads.
Premium Storage: Supported
Premium Storage Caching: Supported
What is NDv2-series VM's?
NDv2-series virtual machine is a new addition to the GPU family designed for the needs of the HPC, AI, and machine learning workloads.
Premium Storage: Supported
Premium Storage Caching: Supported
Infiniband: Not supported
What is ND-series VM's?
The ND-series virtual machines are a new addition to the GPU family designed for AI, and Deep Learning workloads. They offer excellent performance for training and inference. These instances provide excellent performance for single-precision floating point operations, for AI workloads utilizing Microsoft Cognitive Toolkit, TensorFlow, Caffe, and other frameworks.
The ND-series also offers a much larger GPU memory size (24 GB), enabling to fit much larger neural net models. Like the NC-series, the ND-series offers a configuration with a secondary low-latency, high-throughput network through RDMA, and InfiniBand connectivity so you can run large-scale training jobs spanning many GPUs.
What is NV-series VM's?
The NV-series virtual machines are powered by NVIDIA Tesla M60 GPUs and NVIDIA GRID technology for desktop accelerated applications and virtual desktops where customers are able to visualize their data or simulations.
8) Deploy and configure scale sets:
What are the availability considerations during Scheduled Maintenance?
a) Paired Regions:
Each Azure region is paired with another region within the same geography and together they make a regional pair. In scheduled maintenance phase, Azure will only update the VMs in a single region of a region pair. For example, when updating the VM in North Central US, Azure won't update any VM in South Central US at the same time.
b) Availability sets and scale sets:
When deploying a workload on Azure VMs, you can create the VMs within an availability set to provide high availability to your application. This ensures that during either an outage or rebootful maintenance events, at least one VM is available.
Within an availability set, individual VMs are spread across up to 20 update domains (UDs). During scheduled maintenance, only a single update domain is updated at any given time. The order of update domains being updated doesn't necessarily happen sequentially.
Virtual machine scale sets are an Azure compute resource that enables you to deploy and manage a set of identical VMs as a single resource. The scale set is automatically deployed across update domains, like VMs in an availability set. Just like with availability sets, with scale sets only a single update domain is updated at any given time during scheduled maintenance.
Auto-scaling:
How to automatically scale VM's in Azure?
You can automatically scale your VM's when you use any of the following options:
1) Virtual Machine scale sets
2) Autoscaling feature of Azure Monitor
What is Horizontal or Vertical scaling?
The autoscale feature of Azure Monitor only scales horizontally, which is an increase ("out") or decrease ("in") of the number of VMs. Horizontal scaling is more flexible in a cloud situation as it allows you to run potentially thousands of VMs to handle load. You scale horizontally by either automatically or manually changing the capacity (or instance count) of the scale set.
Vertical scaling keeps the same number of VMs, but makes the VMs more ("up") or less ("down") powerful. Power is measured in attributes such as memory, CPU speed, or disk space. Vertical scaling is dependent on the availability of larger hardware, which quickly hits an upper limit and can vary by region. Vertical scaling also usually requires a VM to stop and restart. You scale vertically by setting a new size in the configuration of the VMs in the scale set.
Using runbooks in Azure Automation, you can easily scale VMs in a scale set up or down.
How to scale VM's using VM scale set?
Scale sets make it easy for you to deploy and manage identical VMs as a set. Automatic scaling provides the right number of VMs to handle the load on your application. It enables you to add VMs to handle increases in load and save money by removing VMs that are sitting idle. You specify a minimum and maximum number of VMs to run based on a set of rules. Having a minimum makes sure your application is always running even under no load. Having a maximum value limits your total possible hourly cost. You can create a scale set, install the extension, and configure autoscale using an Azure Resource Manager template. In the Azure portal, enable autoscale from Azure Monitor, or enable autoscale from the scale set settings.
How to Autoscale by metrics through Azure Monitor?
The autoscale feature of Azure Monitor enables you to scale the number of running VMs up or down based on metrics. By default, VMs provide basic host-level metrics for disk, network, and CPU usage. When you configure the collection of diagnostics data using the diagnostic extension, additional guest OS performance counters become available for disk, CPU, and memory.
If your application needs to scale based on metrics that are not available through the host, then the VMs in the scale set need to have either the Linux diagnostic extension or Windows diagnostics extension installed. If you create a scale set using the Azure portal, you need to also use Azure PowerShell or the Azure CLI to install the extension with the diagnostics configuration that you need.
How to set up rules in combination with metrics for autoscaling?
Rules combine a metric with an action to be performed. When rule conditions are met, one or more autoscale actions are triggered. For example, you might have a rule defined that increases the number of VMs by 1 if the average CPU usage goes above 85 percent.
Is it possible to set alerts for autoscale rules?
Yes, you can set up triggers so that specific web URLs are called or emails are sent based on the autoscale rules that you create. Webhooks allow you to route the Azure alert notifications to other systems for post-processing or custom notifications.
===========
What is Scale set?
A virtual machine scale set allows you to deploy and manage a set of identical, auto-scaling virtual machines. VMs in a scale set are distributed across logic fault and update domains in one or more placement groups. These are groups of similarly configured VMs, similar to availability sets.
How is the scaling achieved in Scale Set?
VMs are created as needed in a scale set. You define autoscale rules to control how and when VMs are added or removed from the scale set. These rules can be triggered based on metrics such as CPU load, memory usage, or network traffic.
What is Scale Sets limit?
Scale sets support up to 1,000 VMs when you use an Azure platform image. For workloads with significant installation or VM customization requirements, you may wish to Create a custom VM image. You can create up to 300 VMs in a scale set when using a custom image.
a) Paired Regions:
Each Azure region is paired with another region within the same geography and together they make a regional pair. In scheduled maintenance phase, Azure will only update the VMs in a single region of a region pair. For example, when updating the VM in North Central US, Azure won't update any VM in South Central US at the same time.
b) Availability sets and scale sets:
When deploying a workload on Azure VMs, you can create the VMs within an availability set to provide high availability to your application. This ensures that during either an outage or rebootful maintenance events, at least one VM is available.
Within an availability set, individual VMs are spread across up to 20 update domains (UDs). During scheduled maintenance, only a single update domain is updated at any given time. The order of update domains being updated doesn't necessarily happen sequentially.
Virtual machine scale sets are an Azure compute resource that enables you to deploy and manage a set of identical VMs as a single resource. The scale set is automatically deployed across update domains, like VMs in an availability set. Just like with availability sets, with scale sets only a single update domain is updated at any given time during scheduled maintenance.
Auto-scaling:
How to automatically scale VM's in Azure?
You can automatically scale your VM's when you use any of the following options:
1) Virtual Machine scale sets
2) Autoscaling feature of Azure Monitor
What is Horizontal or Vertical scaling?
The autoscale feature of Azure Monitor only scales horizontally, which is an increase ("out") or decrease ("in") of the number of VMs. Horizontal scaling is more flexible in a cloud situation as it allows you to run potentially thousands of VMs to handle load. You scale horizontally by either automatically or manually changing the capacity (or instance count) of the scale set.
Vertical scaling keeps the same number of VMs, but makes the VMs more ("up") or less ("down") powerful. Power is measured in attributes such as memory, CPU speed, or disk space. Vertical scaling is dependent on the availability of larger hardware, which quickly hits an upper limit and can vary by region. Vertical scaling also usually requires a VM to stop and restart. You scale vertically by setting a new size in the configuration of the VMs in the scale set.
Using runbooks in Azure Automation, you can easily scale VMs in a scale set up or down.
How to scale VM's using VM scale set?
Scale sets make it easy for you to deploy and manage identical VMs as a set. Automatic scaling provides the right number of VMs to handle the load on your application. It enables you to add VMs to handle increases in load and save money by removing VMs that are sitting idle. You specify a minimum and maximum number of VMs to run based on a set of rules. Having a minimum makes sure your application is always running even under no load. Having a maximum value limits your total possible hourly cost. You can create a scale set, install the extension, and configure autoscale using an Azure Resource Manager template. In the Azure portal, enable autoscale from Azure Monitor, or enable autoscale from the scale set settings.
How to Autoscale by metrics through Azure Monitor?
The autoscale feature of Azure Monitor enables you to scale the number of running VMs up or down based on metrics. By default, VMs provide basic host-level metrics for disk, network, and CPU usage. When you configure the collection of diagnostics data using the diagnostic extension, additional guest OS performance counters become available for disk, CPU, and memory.
If your application needs to scale based on metrics that are not available through the host, then the VMs in the scale set need to have either the Linux diagnostic extension or Windows diagnostics extension installed. If you create a scale set using the Azure portal, you need to also use Azure PowerShell or the Azure CLI to install the extension with the diagnostics configuration that you need.
How to set up rules in combination with metrics for autoscaling?
Rules combine a metric with an action to be performed. When rule conditions are met, one or more autoscale actions are triggered. For example, you might have a rule defined that increases the number of VMs by 1 if the average CPU usage goes above 85 percent.
Is it possible to set alerts for autoscale rules?
Yes, you can set up triggers so that specific web URLs are called or emails are sent based on the autoscale rules that you create. Webhooks allow you to route the Azure alert notifications to other systems for post-processing or custom notifications.
===========
What is Scale set?
A virtual machine scale set allows you to deploy and manage a set of identical, auto-scaling virtual machines. VMs in a scale set are distributed across logic fault and update domains in one or more placement groups. These are groups of similarly configured VMs, similar to availability sets.
How is the scaling achieved in Scale Set?
VMs are created as needed in a scale set. You define autoscale rules to control how and when VMs are added or removed from the scale set. These rules can be triggered based on metrics such as CPU load, memory usage, or network traffic.
What is Scale Sets limit?
Scale sets support up to 1,000 VMs when you use an Azure platform image. For workloads with significant installation or VM customization requirements, you may wish to Create a custom VM image. You can create up to 300 VMs in a scale set when using a custom image.
9) Misc - VM Infra Automation & Security:
Infrastructure Automation:
Infrastructure Automation:
What are the tools and solutions that allow you to automate the complete Azure infrastructure deployment and management lifecycle?
These tools commonly fit in to one of the following approaches:
- Automate the configuration of VMs
- Tools include Ansible, Chef, and Puppet.
- Tools specific to VM customization include cloud-init for Linux VMs, PowerShell Desired State Configuration (DSC), and the Azure Custom Script Extension for all Azure VMs.
- Automate infrastructure management
- Tools include Packer to automate custom VM image builds, and Terraform to automate the infrastructure build process.
- Azure Automation can perform actions across your Azure and on-premises infrastructure.
- Automate application deployment and delivery
- Examples include Azure DevOps Services and Jenkins.
Security
How to secure and use policies on VM's in Azure?
The following are the tools available in Azure to secure the VM's:
1) Antimalware
2) Azure Security Center
3) Encryption
4) Key Vault and SSH Keys
5) Managed identities for Azure resources in Azure AD
6) Azure Policies
7) Role-based access control
What is Anitmalware?
Microsoft Antimalware for Azure is a free real-time protection capability that helps identify and remove viruses, spyware, and other malicious software. Alerts can be configured to notify you when known malicious or unwanted software attempts to install itself or run on your VM.
What is Azure Security Center?
Azure Security Center helps you prevent, detect, and respond to threats to your VMs. Security Center provides integrated security monitoring and policy management across your Azure subscriptions, helps detect threats that might otherwise go unnoticed, and works with a broad ecosystem of security solutions.
Security Center’s just-in-time access can be applied across your VM deployment to lock down inbound traffic to your Azure VMs, reducing exposure to attacks while providing easy access to connect to VMs when needed. When just-in-time is enabled and a user requests access to a VM, Security Center checks what permissions the user has for the VM. If they have the correct permissions, the request is approved and Security Center automatically configures the Network Security Groups (NSGs) to allow inbound traffic to the selected ports for a limited amount of time. After the time has expired, Security Center restores the NSGs to their previous states.
What is Encryption?
Virtual disks on Windows VMs are encrypted at rest using BitLocker. Virtual disks on Linux VMs are encrypted at rest using dm-crypt. There is no charge for encrypting virtual disks in Azure. Cryptographic keys are stored in Azure Key Vault using software-protection, or you can import or generate your keys in Hardware Security Modules (HSMs) certified to FIPS 140-2 level 2 standards.
What is Key Vault?
Secrets and certificates can be modeled as resources and provided by Key Vault. You can use Azure PowerShell to create key vaults for Windows VMs and the Azure CLI for Linux VMs. You can also create keys for encryption. Key vault access policies grant permissions to keys, secrets, and certificates separately. For example, you can give a user access to only keys, but no permissions for secrets. However, permissions to access keys or secrets or certificates are at the vault level. In other words, key vault access policy does not support object level permissions.
What is the purpose of SSH Keys?
When you connect to VMs, you should use public-key cryptography to provide a more secure way to sign in to them. This process involves a public and private key exchange using the secure shell (SSH) command to authenticate yourself rather than a username and password. Passwords are vulnerable to brute-force attacks, especially on Internet-facing VMs such as web servers. With a secure shell (SSH) key pair, you can create a Linux VM that uses SSH keys for authentication, eliminating the need for passwords to sign-in. You can also use SSH keys to connect from a Windows VM to a Linux VM.
What is Managed identities for Azure resources?
A common challenge when building cloud applications is how to manage the credentials in your code for authenticating to cloud services. Keeping the credentials secure is an important task. Ideally, the credentials never appear on developer workstations and aren't checked into source control. Azure Key Vault provides a way to securely store credentials, secrets, and other keys, but your code has to authenticate to Key Vault to retrieve them.
The managed identities for Azure resources feature in Azure Active Directory (Azure AD) solves this problem. The feature provides Azure services with an automatically managed identity in Azure AD. You can use the identity to authenticate to any service that supports Azure AD authentication, including Key Vault, without any credentials in your code. Your code that's running on a VM can request a token from two endpoints that are accessible only from within the VM.
What are Policies?
Azure policies can be used to define the desired behavior for your organization's Windows VMs and Linux VMs. By using policies, an organization can enforce various conventions and rules throughout the enterprise.
What is Role-based access control?
Using role-based access control (RBAC), you can segregate duties within your team and grant only the amount of access to users on your VM that they need to perform their jobs. Instead of giving everybody unrestricted permissions on the VM, you can allow only certain actions. You can configure access control for the VM in the Azure portal, using the Azure CLI, orAzure PowerShell.
VM Governance:
What are various levels of management scope in Azure?
Azure provides 4 levels of management scope:
a) Management
b) Subscription
c) Resource Group
d) Resource
How to apply management settings at various levels of scope?
The level you select determines how widely the setting is applied. Lower levels inherit settings from higher levels. When you apply a setting to the subscription, that setting is applied to all resource groups and resources in your subscription. When you apply a setting on the resource group, that setting is applied the resource group and all its resources. However, another resource group does not have that setting.
Usually, it makes sense to apply critical settings at higher levels and project-specific requirements at lower levels. For example, you might want to make sure all resources for your organization are deployed to certain regions. To accomplish this requirement, apply a policy to the subscription that specifies the allowed locations. As other users in your organization add new resource groups and resources, the allowed locations are automatically enforced.
What are the various tools available to manage VM's?
The following are the various tools to manage VM's:
1) Role-based access control
2) Azure Policy
3) Resource Locks
4) Tags
What is role-based access control?
You want to make sure users in your organization have the right level of access to these resources. You don't want to grant unlimited access to users, but you also need to make sure they can do their work. Role-based access control enables you to manage which users have permission to complete specific actions at a scope.
https://docs.microsoft.com/en-us/azure/role-based-access-control/overview
To create and remove role assignments, users must have Microsoft.Authorization/roleAssignments/* access. This access is granted through the Owner or User Access Administrator roles.
What are the three resource-specific roles used to manage VM?
1) Virtual Machine Contributor
2) Network Contributor
3) Storage Account Contributor
Instead of assigning roles to individual users, it's often easier to use an Azure Active Directory group that has users who need to take similar actions. Then, assign that group to the appropriate role.
What is Azure Policy?
Azure Policy helps you make sure all resources in subscription meet corporate standards. Your subscription already has several policy definitions.
https://docs.microsoft.com/en-us/azure/governance/policy/overview
Few sample policies that can be applied to VM:
--> Limit the locations for all resources.
--> Limit the SKUs for virtual machines.
--> Audit virtual machines that don't use managed disks.
What is Resource Locks?
Resource locks prevent users in your organization from accidentally deleting or modifying critical resources. Unlike role-based access control, resource locks apply a restriction across all users and roles. You can set the lock level to CanNotDelete or ReadOnly.
To create or delete management locks, you must have access to Microsoft.Authorization/locks/* actions. Of the built-in roles, only Owner and User Access Administrator are granted those actions.
What is Tag?
Tags can be applied to Azure Resources to logically organize them by categories. Each tag consists of a name and a value. For example, you can apply the name "Environment" and the value "Production" to all the resources in production.
Is Storage Account required when creating Azure Managed disks?
Storage account is not required.
What is Serial console in Windows VM?
The virtual machine (VM) serial console in the Azure portal provides access to a text-based console for Windows virtual machines. This serial connection connects to the COM1 serial port of the virtual machine, providing access to it, independent of the virtual machine's network or operating system state. Access to the serial console for a virtual machine can be done only by using the Azure portal. It's allowed only for those users who have an access role of Virtual Machine Contributor or higher to the virtual machine.
What are the pre-requisites for accessing a serial console on Windows VM?
The VM in which you're accessing a serial console must use the resource management deployment model. Classic deployments aren't supported.
The VM in which you're accessing a serial console must have boot diagnostics enabled.
An account using a serial console must have the Virtual Machine Contributor role for the VM and the boot diagnostics storage account.
The VM in which you're accessing a serial console must have a password-based account. You can create one with the reset password function of the VM access extension. Select Reset password from the Support + troubleshooting section
What is the NSG Port to be enabled for WinRM connection to Azure VM?
5985 for HTTP and 5986 for HTTPS
Is Windows 10 Pro is available in all Microsoft Azure subscription offers?
No
HDD refers to Standard Disks
What is the purpose of Generalization of VM?
Resets server-specific data:
-Computer Name
-Security identifier (SIDs)
-Local Administrator or root identity
-Device driver cache
-Event logs
What tools are used to Generalize a VM?
Windows VM:
1) sysprep - removes all your personal account and security information, and then prepares the machine to be used as an image.
Linux VM:
2)Open SSH console and run the following command:
sudo waagent -deprovision+user
Note: Before running the above commands, it is always good practice to backup the VM first because generalization is destructive and permanent
What are the difference between Disks, Snapshots, and images from managed disks perspective?
Disks:
Disks are OS and VHD file storage where no storage account (management) is required, you will pay for pre-allocated storage (Ex; P10 = 128 GB SSD VHD)
Snapshots:
Read-only full copy of a managed disks. Snapshot a disk means copying only ONE disk.
Images:
Generalized VM disk image. Stored permanently in Azure image library. Snapshots can be converted into images
What are the high-level steps to create a managed image of a generalized VM in Azure?
1) Generalize the Windows VM using Sysprep tool
2) Optional step: Create a backup of the VM to Recovery services vault (for safer side)
3) Create an VM image (including OS disks and data disks) from a managed disk
- First, take a snapshot of the disk (OS disks and data disks) from Azure portal by clicking the button "Create Snapshot" at the top.
- Now take the snapshot of the VM
4) Deploy a new VM based on the image of VM and snapshot of the disks
VM Governance:
What are various levels of management scope in Azure?
Azure provides 4 levels of management scope:
a) Management
b) Subscription
c) Resource Group
d) Resource
How to apply management settings at various levels of scope?
The level you select determines how widely the setting is applied. Lower levels inherit settings from higher levels. When you apply a setting to the subscription, that setting is applied to all resource groups and resources in your subscription. When you apply a setting on the resource group, that setting is applied the resource group and all its resources. However, another resource group does not have that setting.
Usually, it makes sense to apply critical settings at higher levels and project-specific requirements at lower levels. For example, you might want to make sure all resources for your organization are deployed to certain regions. To accomplish this requirement, apply a policy to the subscription that specifies the allowed locations. As other users in your organization add new resource groups and resources, the allowed locations are automatically enforced.
What are the various tools available to manage VM's?
The following are the various tools to manage VM's:
1) Role-based access control
2) Azure Policy
3) Resource Locks
4) Tags
What is role-based access control?
You want to make sure users in your organization have the right level of access to these resources. You don't want to grant unlimited access to users, but you also need to make sure they can do their work. Role-based access control enables you to manage which users have permission to complete specific actions at a scope.
https://docs.microsoft.com/en-us/azure/role-based-access-control/overview
To create and remove role assignments, users must have Microsoft.Authorization/roleAssignments/* access. This access is granted through the Owner or User Access Administrator roles.
What are the three resource-specific roles used to manage VM?
1) Virtual Machine Contributor
2) Network Contributor
3) Storage Account Contributor
Instead of assigning roles to individual users, it's often easier to use an Azure Active Directory group that has users who need to take similar actions. Then, assign that group to the appropriate role.
What is Azure Policy?
Azure Policy helps you make sure all resources in subscription meet corporate standards. Your subscription already has several policy definitions.
https://docs.microsoft.com/en-us/azure/governance/policy/overview
Few sample policies that can be applied to VM:
--> Limit the locations for all resources.
--> Limit the SKUs for virtual machines.
--> Audit virtual machines that don't use managed disks.
What is Resource Locks?
Resource locks prevent users in your organization from accidentally deleting or modifying critical resources. Unlike role-based access control, resource locks apply a restriction across all users and roles. You can set the lock level to CanNotDelete or ReadOnly.
To create or delete management locks, you must have access to Microsoft.Authorization/locks/* actions. Of the built-in roles, only Owner and User Access Administrator are granted those actions.
What is Tag?
Tags can be applied to Azure Resources to logically organize them by categories. Each tag consists of a name and a value. For example, you can apply the name "Environment" and the value "Production" to all the resources in production.
Is Storage Account required when creating Azure Managed disks?
Storage account is not required.
What is Serial console in Windows VM?
The virtual machine (VM) serial console in the Azure portal provides access to a text-based console for Windows virtual machines. This serial connection connects to the COM1 serial port of the virtual machine, providing access to it, independent of the virtual machine's network or operating system state. Access to the serial console for a virtual machine can be done only by using the Azure portal. It's allowed only for those users who have an access role of Virtual Machine Contributor or higher to the virtual machine.
What are the pre-requisites for accessing a serial console on Windows VM?
The VM in which you're accessing a serial console must use the resource management deployment model. Classic deployments aren't supported.
The VM in which you're accessing a serial console must have boot diagnostics enabled.
An account using a serial console must have the Virtual Machine Contributor role for the VM and the boot diagnostics storage account.
The VM in which you're accessing a serial console must have a password-based account. You can create one with the reset password function of the VM access extension. Select Reset password from the Support + troubleshooting section
What is the NSG Port to be enabled for WinRM connection to Azure VM?
5985 for HTTP and 5986 for HTTPS
Is Windows 10 Pro is available in all Microsoft Azure subscription offers?
No
HDD refers to Standard Disks
What is the purpose of Generalization of VM?
Resets server-specific data:
-Computer Name
-Security identifier (SIDs)
-Local Administrator or root identity
-Device driver cache
-Event logs
What tools are used to Generalize a VM?
Windows VM:
1) sysprep - removes all your personal account and security information, and then prepares the machine to be used as an image.
Linux VM:
2)Open SSH console and run the following command:
sudo waagent -deprovision+user
Note: Before running the above commands, it is always good practice to backup the VM first because generalization is destructive and permanent
What are the difference between Disks, Snapshots, and images from managed disks perspective?
Disks:
Disks are OS and VHD file storage where no storage account (management) is required, you will pay for pre-allocated storage (Ex; P10 = 128 GB SSD VHD)
Snapshots:
Read-only full copy of a managed disks. Snapshot a disk means copying only ONE disk.
Images:
Generalized VM disk image. Stored permanently in Azure image library. Snapshots can be converted into images
What are the high-level steps to create a managed image of a generalized VM in Azure?
1) Generalize the Windows VM using Sysprep tool
2) Optional step: Create a backup of the VM to Recovery services vault (for safer side)
3) Create an VM image (including OS disks and data disks) from a managed disk
- First, take a snapshot of the disk (OS disks and data disks) from Azure portal by clicking the button "Create Snapshot" at the top.
- Now take the snapshot of the VM
4) Deploy a new VM based on the image of VM and snapshot of the disks
Thank you for explaining the AZ: 300 Architect Technologies analysis in a effective way so that it reaches us in a manner which is understandable .Keep Sharing.
ReplyDeleteThank you for explaining the AZ: 300 Architect Technologies in a effective way so that it reaches us in a manner which is understandable .Keep Sharing.
ReplyDelete
ReplyDeleteGood Post! Thank you so much for sharing this post, it was so good to read and useful to improve my knowledge on AZ: 300 Architect Technologies, keep blogging.
Great post! Thanks for sharing this amazing post on AZ: 300 Architect Technologies, keep blogging.
ReplyDelete