Set of Common questions always arise during vSphere migration design discussions , like what network / interface used for migration ? How to perform the network segmentation for optimal migration experience without any impacts. What exactly gets transferred in VMotion enabled networks .
There are multiple different opinions about the data flow in migration networking and most of the percentage misunderstand that the vMotion networking is solely responsible for VM migration in all scenarios, even during storage vMotion and thus resulting in improper planning and can cause potential performance issues during the actual migration.
Lets dig in deeper to understand based on scenarios:
There are two states in VM migration:
Cold Migration ( virtual machine that is powered off in the entire duration of migration – Cold data)
Hot Migration (virtual machine is powered on and actively available during the migration – Hot Data)
Table below depicts the vMotion networking usage with scenarios.
Item No
Migration Action
Migration State
Networking Used
Notes
1
vMotion(Compute)
Hot
vMotion
Transfer only the ownership of the VM files like .vmx , .swp (we are not moving any data here, just change the VM ownership from host1 to host 2). Snapshots, Clones & vmdk will use management enabled network.
2
vMotion(Compute)
Cold
Management
vMotion network is not used to perform this migration. cold data like powered off virtual machines, clones and snapshots are migrated over the Provisioning network if that is enabled. It is not enabled by default. If it’s not configured, the Management enabled network will be used.
3
Storage vMotion(Storage)
Hot
Management
vMotion network is not used to perform this migration. All data (VMDKs) goes through Provisioning or Management enabled network.
4
Storage vMotion(Storage)
Cold
Management
vMotion network is not used to perform this migration. All data (VMDKs) goes through Provisioning or Management enabled network
5
xMotion(Compute + Storage)
Hot
vMotion + Management
Hot data like .vmx. .swp will go through vMotion enabled network (like Item No1) and rest go through Provisioning or Management enabled network
6
xMotion(Compute + Storage)
Cold
Management
vMotion network is not used to perform this migration. All data (VMDKs) goes through Provisioning or Management enabled network
Additional Points to be considered:
Management service is enabled by default on the first VMkernel interface, the other VMkernel interfaces and services are typically configured post-installation of ESXi. This can be customized based on our design requirements
You can test yourself the traffic flow with the help of wireshark inspection and also, like disabling the vmotion when migrating the cold VM and observe.
In this Blog series, we are going to deep dive and see various feature and benefits offered by Azure Arc service.
Customer environments are increasingly opted towards diverse IT infrastructure (Poly/Multi Cloud, On-premises Datacentres, IOT devices and Edge, and other solution models) and due to this shift in paradigm , often there are challenges in management of resources, agile governance and security across the IT estate.
To overcome this challenges for customers and to maintain consistent resource management across various dimensions , Microsoft introduced Azure Arc as a set of technology to help provide a unified management experience across entire IT estate despite the resource object location . Azure Arc enables a single pane of glass view of heterogeneous environment and the ability to govern and manage all these resources in a consistent way.
Azure Arc extends the Azure management feature sets and capabilities to any infrastructure and customers can enable the arc services to manage the below listed resources:
Servers – Linux and Windows Servers (Physical and Virtual)
Kubernetes
Data Services (SQL Managed Instance, SQL server and PostgreSQL) – Preview
Application Services (App service, Functions and Logic Apps) – Preview
Operations and Workability:
Azure Arc is built on the substructure of ARM (Azure Resource Manager) and this enables the customers to register their resources outside of Azure using combination of agents running to bring under Azure control pane with great ease.
Many of the core features of Azure Resource Manager are enabled by Arc. This includes the Azure Portal, RBAC, Resource Groups, Azure Policy, Search, Tagging and more. Additionally, customers can also use hybrid services such as Azure Monitor, Azure Security Centre, Azure Sentinel etc.
In the upcoming section, we are going to discuss about the operating instructions to enable one of the On-Premises VM (Based on VMware) to bring under Azure Arc management.
Arc Enabled Servers
Currently Azure Arc enabled servers are GA now and available across globally to use and not restricted to any regions. This service offered at no additional cost.
Note: Using of any Azure hybrid management services like (Azure monitor, Automation, etc) will incur billing. No extra costs is only for Azure Arc core control panel functionality.
Lab Setup and Pre-Requisites Information :
The table below depicts about our Lab setup and Prerequires information to enable Arc service on VM based on windows server 2019
Disclaimer : The deployment method can be referenced for any production deployment scenario but its majorly developed for Demo/ educational purpose.
Category
Value
Notes
On-Premises Hypervisor
VMware Workstation
On-Premises VM
Windows Server 2019 with Admin Rights NET Framework 4.6 or later is required and Windows PowerShell 5.1.
Supported OS: Windows and Linux, physical and virtual, domain-joined, and non-domain-joined servers. Currently we officially support Windows Server 2012R2 and higher, Ubuntu 16.04 and 18.04, CentOS Linux 7, SUSE Linux Enterprise Server 15, Red Hat Enterprise Linux 7, and Amazon Linux 2.
Subscription
Azure Free with Required Rights for on-boarding Location – AU
To execute scripts and other management activities during server on-boarding . This might incur a small billing to your subscription, so please be watchful.
Resource Group
Rg-test-arc01
A dedicated RG for arc enabled servers. Ease of mangmnet.
Register the Azure providers by executing below commands in Azure Shell window.
Step 3:
Next Step is to generate the Installation script to automate the agent installation on On-Prem VM as part of on-boarding Process.
Goto Azure Portal -> Search –> Servers – Azure Arc –> Add a Single server
This operation will generate a script to run on the target server.
Step 4:
Proceed to further config by clicking “Next: Resource Details” and fill up the required details.
Step 5:
Fill in the details as per the required values to generate a installation script and click next.
Step 6:
Define Tags based on use cases and click on to “Download and run script”. This action will download a PowerShell script to install and configure the Agent based on settings defined in Step 5.
Note: We are installing the agent manually but there are well documented planner from Microsoft github repo to deploy in scale for various solution.
Execute the downloaded Onboarding script from Step 6 on VMware workstation VM (windows server 2019) and follow the following steps
Copy the script to server (any folder path)
Open PowerShell in elevated mode (Admin account)
Execute the copied script ./OnboardingScript.ps1 and wait for it to complete.
During the script execution, It will prompt you to login to Azure portal for successful authentication .Please key-in your respective credentials.
It will take several minutes to complete the registration process and show as registered.
Note: The Azure Connected Machine agent package contains several logical components, which are bundled together to pass the information through agent.
• Azure Subscription ID • Location • Resource Group • Azure Service Principal • Hybrid Instance Metadata and Guest Configuration /Extension Manager
The Connected Machine agent cannot be installed on an Azure Windows virtual machine. If you attempt to, the installation detects this and rolls back.
Step 8:
Verify the on-boarding status in Azure portal. Navigate to “Servers-Azure Arc” from search (Similar to Step 3), we should now see our new machine connected and ready for use.
Since I have taken the screenshot immediately after script execution and registration, The status message is still not reflecting in the below screenshot and showing empty. However in general cases, is should show as connected.
Now that we onboarded the machine in Azure, we can leverage azure control pane and hybrid management services to manged this on-premises VM from Azure poral. In Next blog post, we will discuss about management operations of Arc enabled servers. in detail
Traditional management of Jump severs in cloud is painful and it involves lots of tasks to setup and for on-going management.
To avoid potential attacks from external threats, Jump servers need to be completely locked-down and hard, like enabling aggressive block-list of apps and IPs, implement 2FA, zero admin mode, auditing and if possible re-build jump server every alternative days using automation tools to make it more secure in cloud.
I know a lot of you out there rely on “jump servers.” and not much serious about Jump server day to day operation management tasks. Many SMB segment companies were attacked because of exploits through poorly managed jump servers and in order to overcome those problems and make security more controlled, Microsoft introduced Bastian host last year as a replacement for traditional jump server architecture.
Azure bastion is a fully managed PaaS service that you provision inside your virtual network. It provides secure and seamless RDP/SSH connectivity to your virtual machines directly in the azure portal over SSL. This eliminates the need to expose the Virtual Machines RDP and SSH ports to the internet. After the initial release, the solution is being adopted by various azure customers and received great feedback.
Moreover, bastion hosts are very easy to setup with few clicks and the service will be ready within few minutes after deployment. Bastian type services are also available in other clouds with different feature sets.
Azure Bastian key characteristics and what it can offer to azure customers:
No public required for VMs in Azure.
Remote Session over TLS and firewall traversal for RDP/SSH.
Agent-less and no additional software’s required on VMs.
Internally it’s a VM scale set and it can expand based on connections requirement.
Centrally hardened and protects against port scanning, zero-day exploits and malware.
Additional security layer can be leveraged using NSG on top of bastion host
Access to Windows Server VMs by Azure Bastion does not require an RDS CAL when used solely for administrative purposes.
As the name implies “Bastian” , it will act as true source of strength to enter into azure VM world.
High-level architecture:
This figure shows the architecture of an Azure Bastion deployment aligning to above listed characteristics.
Whats planned next – feature updates (not sure when but its under the pipeline):
Bastian host required to deployed under each vNet and currently doesn’t support vNet peering options. However, in upcoming product updates it should soon start supporting the peering options.
Azure AD SSO integration
Currently Bastian can only accessed through azure portal (via HTM5 browser) and its expected to start support of native RDP/SSH clients soon.
Being a big fan of x-men series and security products, I couldn’t resist researching on the new SecOps offering from Microsoft called ” Azure Sentinel’. Those who are not familiar with x-men comics, Sentinel are mutant hunting robots which can monitor, detect and kill mutants 😛
Azure Sentinel service is a great SIEM and SOAR offering from Microsoft with the proper equation of SecOps + AI in a cloud native solution.
The great advantage and differentiator of this offering from other SIEM solution is, its backed by Microsoft which is turning into a biggest security company in the world with huge loads of investment going into security research. It’s clearly going to disrupt the SOC as a cloud security solution.
Since it’s a cloud native solution, no worries about capacity planning and other on-prem bottle necks.
Alright, upon doing some research and Lab on the Azure Sentinel, below are the list of high-level characteristics and feature set about this service offering.
Azure Sentinel SecOps flow:
Enable Azure Sentinel to easily aggregate security data generated by end point devices, network infrastructure, and other security systems, then leverage it to detect and respond to threats in your environment.
Collect / Visibility
Collect security data at cloud scale from mostly any source (On Premises, Cloud (Including Other Clouds like AWS) and SaaS Apps.
Data collection options: On-prem and Cloud:
Agent based collection for Linux and windows devices (OS events, OS firewall, DNS, DHCP etc)
Syslog based collection using syslog connector/CEF – either can be deployed in on-premises or cloud over TLS
Supports REST API based collection for F5, Barracuda and Symantec and other similar products.
Custom connectors using Azure functions for log storage like S3.
Azure log analytic data will also support sentinel analysis.
Visualise:
Workbooks – Interactive dasboarding with the analysis. There are good number of built-in workbook templates available from Azure sentinel like, Azure Activity, Azure AD audit-logs, Azure AD Sign-in logs and variety of other product supports (AWS Network activities, AWS user activities). Based on the need custom workbook can be created
Detect – Analytics / Hunting
There are good number of built-in analytics available in Azure sentinel to choose for detecting threats.
Option available to create custom KQL based queries for analytics.
Trigger automated playbooks to tackle threats.
Leverage Machine learning to increase your catch rate without increasing the noise.
Investigate – Incidents
Track investigation from sentinel security incident, raised based on priority.
Bring you own ITSM framework by integrating with ticketing tool for the incident track and resolution.
Visualize the entire threat attack to determine the scope and impact, by navigating the relationships between alerts.
Respond – Automation
Automate and orchestrate the scopes using integrated Azure logic Apps. Logic Apps will help in building automated and scalable playbooks that integrate across tools. Able to setup the complete workflow from alert trigger to resolution.
Again like other sections, there are good number of sample library available to configure and test the playbooks.
Free Trail – Azure Sentinel can be enabled at no additional cost on an Azure Monitor Log Analytics workspace for the first 31-days. Usage beyond the first 31-days will be charged per pricing listed above. Charges related to Azure Monitor Log Analytics for data ingestion and additional capabilities for automation. And bring your “own machine learning” are still applicable during the free trial.
Link – https://docs.microsoft.com/en-us/powershell/scripting/whats-new/what-s-new-in-powershell-core-60?view=powershell-5.1
Installing PowerShell Core on Windows : https://docs.microsoft.com/en-us/powershell/scripting/setup/Installing-PowerShell-Core-on-Windows?view=powershell-6
Dev's Technical Blog 