Misconception about vMotion Traffic usage:

Set of Common  questions always arise during  vSphere migration design discussions , like what network / interface used for migration  ? How to perform the network segmentation for optimal migration experience without any impacts. What exactly gets transferred in VMotion enabled networks .

There are multiple different opinions about the data flow in migration networking and most of the percentage misunderstand that the vMotion networking is solely responsible for VM migration in all scenarios,  even during storage vMotion and thus resulting in improper planning and can cause potential  performance issues during the actual migration.

Lets dig in deeper to understand based on scenarios:

There are two states in VM migration:

  1. Cold Migration ( virtual machine that is powered off in the entire duration of migration – Cold data)
  2. Hot Migration  (virtual machine is powered on and actively available during the migration – Hot Data)

Table below depicts the vMotion networking usage with scenarios.

Item NoMigration ActionMigration StateNetworking UsedNotes
1vMotion(Compute)HotvMotionTransfer only the ownership of the VM files like .vmx , .swp (we are not moving any data here, just change the VM ownership from host1 to host 2). Snapshots, Clones & vmdk will use management enabled network.
2vMotion(Compute)ColdManagementvMotion network is not used to perform this migration. cold data like powered off virtual machines, clones and snapshots are migrated over the Provisioning network if that is enabled. It is not enabled by default. If it’s not configured, the Management enabled network will be used.
3Storage vMotion(Storage)HotManagementvMotion network is not used to perform this migration.
All data (VMDKs) goes through Provisioning or Management enabled network.
4Storage vMotion(Storage)ColdManagementvMotion network is not used to perform this migration. All data (VMDKs) goes through Provisioning or Management enabled network
5xMotion(Compute + Storage)HotvMotion + ManagementHot data like .vmx. .swp will go through vMotion enabled network (like Item No1) and rest go through Provisioning or Management enabled network
6xMotion(Compute + Storage)ColdManagementvMotion network is not used to perform this migration. All data (VMDKs) goes through Provisioning or Management enabled network

Additional Points to be considered:

  • Management service is enabled by default on the first VMkernel interface, the other VMkernel interfaces and services are typically configured post-installation of ESXi. This can be customized based on our design requirements
  • You can test yourself the traffic flow with the help of wireshark inspection and also, like disabling the vmotion when migrating the cold VM and observe.

Hybrid and Multi/Poly Cloud – Azure Arc Series – Part 1 (Servers)


In this Blog series, we are going to deep dive and see various feature and benefits offered by Azure Arc service.

Customer environments are increasingly opted towards diverse IT infrastructure (Poly/Multi Cloud, On-premises Datacentres, IOT devices and Edge, and other solution models) and due to this shift in paradigm  , often there are challenges in management of resources, agile governance and security across the IT estate.

To overcome this challenges for customers and to maintain  consistent  resource management across various dimensions  , Microsoft introduced Azure Arc as a set of technology to help provide a unified management experience across entire IT estate despite the resource object location . Azure Arc  enables a single pane of glass view of  heterogeneous environment and the ability to govern and manage all these resources in a consistent way. 

Azure Arc extends the Azure management feature sets and capabilities  to any infrastructure  and customers can enable the arc services to manage the below listed resources:

  • Servers – Linux and Windows Servers (Physical and Virtual)
  • Kubernetes
  • Data Services (SQL Managed Instance, SQL server  and PostgreSQL)  – Preview
  • Application Services (App service, Functions and Logic Apps) – Preview

Operations and Workability:

Azure Arc is built on the substructure of ARM (Azure Resource Manager) and this enables the customers to register their resources outside of Azure using combination of agents running to bring under Azure control pane with great ease.

Many of the core features of Azure Resource Manager are enabled by Arc. This includes the Azure Portal, RBAC, Resource Groups, Azure Policy, Search, Tagging and more. Additionally, customers can also use hybrid services such as Azure Monitor, Azure Security Centre, Azure Sentinel etc.

In the upcoming section, we are going to discuss about the operating instructions to enable one of the On-Premises VM (Based on VMware) to bring under Azure Arc management.

Arc Enabled Servers

Currently Azure Arc enabled servers are GA now and available across globally to use and not restricted to any regions. This service offered at no additional cost.

Note:  Using of any Azure hybrid management services like (Azure monitor, Automation, etc) will incur billing. No extra costs is only for Azure Arc core control panel functionality.

Lab Setup and Pre-Requisites Information :

The table below depicts about our Lab setup and Prerequires information to enable Arc service on VM based on windows server 2019

Disclaimer : The deployment method can be referenced for any production deployment scenario but its majorly developed for Demo/ educational  purpose.

On-Premises HypervisorVMware Workstation 
On-Premises VMWindows Server 2019 with Admin Rights   NET Framework 4.6 or later is required and Windows PowerShell 5.1.    Supported OS: Windows and Linux, physical and virtual, domain-joined, and non-domain-joined servers. Currently we officially support Windows Server 2012R2 and higher, Ubuntu 16.04 and 18.04, CentOS Linux 7, SUSE Linux Enterprise Server 15, Red Hat Enterprise Linux 7, and Amazon Linux 2.
SubscriptionAzure Free with Required Rights for on-boarding
Location – AU
Azure Contributor role for the designated Resource Group to on-board only.
Refer for more info on Azure Arc required Permissions for other operations:
Overview of the Connected Machine agent – Azure Arc | Microsoft Docs
ShellAzure ShellTo execute scripts and other management activities during server on-boarding . This might incur a small billing to your subscription, so please be watchful.
Resource GroupRg-test-arc01A dedicated RG for arc enabled servers. Ease of mangmnet.
On-Boarding Method to Azure ArcManual Installation : Scripted
Azure Connection Machine Agent
More Granular Details about the Agent prerequisites refer here : Overview of the Connected Machine agent – Azure Arc | Microsoft Docs  
ConnectivityThe communication to the cloud is outbound and uses HTTPS. The machine just needs access to public Azure endpointsPrivate Link connection is on Preview
Agent Performance impact on VMsIt’s a lightweight tool which send updates to every 5 minutes to AzureMy personal recommendation is to keep 5% overhead of compute/memory utilization for calculation.
CostThere is no additional cost for onboarding and managing servers using Azure Arc. 

Server On-Boarding Procedure:

Step 1:

Connect to Azure Shell by logging into to https://portal.azure.com/

Step 2:

Register the Azure providers by executing below commands in Azure Shell window.

Step 3:

Next Step is to generate the Installation script to automate the agent installation on On-Prem VM as part of on-boarding Process.

  • Goto Azure Portal -> Search –> Servers – Azure Arc –> Add a Single server

This operation will generate a script to run on the target server.

Step 4:

Proceed to further config by clicking “Next: Resource Details” and fill up the required details.

Step 5:

Fill in the details as per the required values to generate a installation script and click next.

Step 6:

Define Tags based on use cases and click on to “Download and run script”. This action will download a PowerShell script to install and configure the Agent based on settings defined in Step 5.

We are installing the agent manually but there are well documented planner from Microsoft github repo to deploy in scale for various solution.

Scale Deployment Example:
azure_arc/docs/azure_arc_jumpstart/azure_arc_servers/scaled_deployment at main · microsoft/azure_arc · GitHub

Step 7:

Execute the downloaded Onboarding script from Step 6 on VMware workstation VM  (windows server 2019) and follow the following steps

  • Copy the script to server (any folder path)
  • Open PowerShell in elevated mode (Admin account)
  • Execute the copied script ./OnboardingScript.ps1 and wait for it to complete.
  • During the script  execution, It will prompt you to login to Azure portal for successful authentication .Please key-in your respective credentials.
  • It will take several minutes to complete the registration process and show as registered.
The Azure Connected Machine agent package contains several logical components, which are bundled together to pass the information through agent.

• Azure Subscription ID
• Location
• Resource Group
• Azure Service Principal
• Hybrid Instance Metadata and Guest Configuration /Extension Manager

The Connected Machine agent cannot be installed on an Azure Windows virtual machine. If you attempt to, the installation detects this and rolls back.

Step 8:

Verify the on-boarding status in Azure portal. Navigate to “Servers-Azure Arc” from search (Similar to Step 3), we should now see our new machine connected and ready for use.

Since I have taken the screenshot immediately after script execution and registration, The status message is still not reflecting in the below screenshot and showing empty. However in general cases, is should show as connected.

Now that we onboarded the machine in Azure, we can leverage azure control pane and hybrid management services to manged this on-premises VM from Azure poral. In Next blog post, we will discuss about management operations of Arc enabled servers. in detail