Deploying an Entire Environment using Azure and PowerShell, Part 2

In a previous post, I detailed how to automate the creation of a standard multi-server environment using the IaaS capabilities in Azure. During the last days I had the opportunity of enhancing these scripts a bit. This second part of the post describe the enhancements.

Basically, the original scripts followed these set of steps:

  1. Defining environment variables
  2. Create cloud service
  3. Create storage account
  4. Create VM-n

Pretty simple uh? Yeah, but the resulting environment presented some issues:

  • Are VMs created close enough (same subnet)? Most probably not..
  • What if machines are created in the same rack (Fault Domain), and there is a hardware issue? The whole environment is gone!
  • What if we have multiple WFEs? We would need to load balance them..

All these issues are resolved with the introduction of some Azure concepts such as: Affinity Groups, Availability Sets, Load Balancing. So I enhanced the scripts with them on mind, and the resulting steps are:

  1. Defining environment variables
  2. Create a new Affinity Group (New-AzureAffinityGroup)
  3. Create storage account (add it to the new Affinity Group)
  4. Create cloud service (add it to the new Affinity Group)
  5. For each new load balanced VM:
    1. Create the VM,
    2. Add it to the cloud service
    3. Add it to the same availability set
    4. Create and attach disks as necessary
    5. Configure endpoints (firewall)
    6. Configure load balancer and probe port.

The resulting scripts are:

$ErrorActionPreference = "Stop"   # stop on any error

 

function GetLatestImage($family){

$images = Get-AzureVMImage `

| where { $_.ImageFamily -eq $family } `

| Sort-Object -Descending -Property PublishedDate

 

$latestImage = $images[0]

return $latestImage

}

 

$myAzureSubscription = 'Windows Azure MSDN - Visual Studio Ultimate'

# Environment variables are defined here:

# ONLY LOWERCASE LETTERS HERE!!

$EnvironmentName = "azrtest"

$tag = get-date -format 'hhmmss'

# Create Storage Account through the Portal (vmstorageazrtest)

$StorageAccount = "vmstorage$EnvironmentName"

 

Write-Host $StorageAccount

 

$AzurePubSettingsFile = "C:\Windows Azure MSDN - Visual Studio Ultimate-12-19-2013-credentials.publishsettings"

# ExtraSmall, Small, Medium, Large, ExtraLarge, A6, A7

$VMSize = "Small"

# Region - East US, West US, East Asia, Southeast Asia, North Europe, West Europe

$Location = "Southeast Asia"

$AdminUserName = "admin2K"

$AdminPwd = "password2K"

$OSFamily = "Windows Server 2008 R2 SP1"

 

# Server names cannot be more than 15 chars 

$WFE_1_Name = 'WFESrv1'+$tag

$WFE_2_Name = 'WFESrv2'+$tag

$APPSRV_1_Name = 'AppSrv1'+$tag

$DBSRV_Name = 'DBSrv'+$tag

 

$myDataDiskSize        = 20  # in GB     # User-specified

 

# This must be unique

$CloudServiceName = "azrtest"+$tag

# Run GetLatestImage.ps1

$Image = GetLatestImage($OSFamily)

$ImageName = $Image.ImageName

 

# Affinity Groups - groups machines 'closer' 

$myAffinityGroupName   = $EnvironmentName+'-ag'  # User-defined

# Availability Sets - defines resources on different HA Fault Domains

# One Availability Set is created per application tier. Not needed for the DB server

$myAvailabilitySetName_WFE = $EnvironmentName+'wfe-as'  # User-defined

$myAvailabilitySetName_APPSRV = $EnvironmentName+'appsrv-as'  # User-defined

 

$myEndpointName        = $EnvironmentName+'-ep'  # User-defined

$myLoadBalancerName    = $EnvironmentName+'-lb'  # User-defined

 

Select-AzureSubscription –Default $myAzureSubscription

 

# Config subscription

import-azurepublishsettingsfile $AzurePubSettingsFile

 

 

# Step 1 - Create the Affinity Group

New-AzureAffinityGroup -Name $myAffinityGroupName -Location $Location

 

# Step 2 - Create Storage Account                       

# Create Storage Account through the Portal

# IF NO StorageAccount exists, ONE IS CREATED HERE!

#New-AzureStorageAccount -StorageAccountName $StorageAccount -Location $Location -Label "azrtest"

# Remove-AzureStorageAccount -StorageAccountName $StorageAccount

New-AzureStorageAccount -StorageAccountName $StorageAccount -AffinityGroup $myAffinityGroupName 

 

# Step 3 Create Azure Cloud Service

New-AzureService -ServiceName $CloudServiceName -AffinityGroup $myAffinityGroupName    

 

# Step 4               

#Get-AzureStorageAccount | Select Label

Set-AzureSubscription -SubscriptionName $myAzureSubscription -CurrentStorageAccount $StorageAccount

 

# Step 5

# Create WFE Machine(s)

# Create WFE #1

New-AzureVMConfig -ImageName    $ImageName `

                  -InstanceSize $VMSize `

                  -Name         $WFE_1_Name `

                  -availabilitysetname $myAvailabilitySetName_WFE `

                  -DiskLabel "OS" `

| Add-AzureProvisioningConfig -Windows `

                              -DisableAutomaticUpdates `

                              -AdminUserName $AdminUserName `

                              -Password      $AdminPwd `

| Add-AzureDataDisk -CreateNew -DiskSizeInGB $myDataDiskSize `

                    -DiskLabel 'DataDisk0' `

                    -LUN 0 `

| Add-AzureEndpoint -Name          $myEndpointName `

                    -Protocol      tcp `

                    -LocalPort     80 `

                    -PublicPort    80 `

                    -LBSetName     $myLoadBalancerName `

                    -ProbePort     8080 `

                    -ProbeProtocol tcp `

                    -ProbeIntervalInSeconds 15 `

| New-AzureVM -ServiceName $CloudServiceName        

 

# Create WFE #2

New-AzureVMConfig -ImageName    $ImageName `

                  -InstanceSize $VMSize `

                  -Name         $WFE_2_Name `

                  -availabilitysetname $myAvailabilitySetName_WFE `

                  -DiskLabel "OS" `

| Add-AzureProvisioningConfig -Windows `

                              -DisableAutomaticUpdates `

                              -AdminUserName $AdminUserName `

                              -Password      $AdminPwd `

| Add-AzureDataDisk -CreateNew -DiskSizeInGB $myDataDiskSize `

                    -DiskLabel 'DataDisk0' `

                    -LUN 0 `

| Add-AzureEndpoint -Name          $myEndpointName `

                    -Protocol      tcp `

                    -LocalPort     80 `

                    -PublicPort    80 `

                    -LBSetName     $myLoadBalancerName `

                    -ProbePort     8080 `

                    -ProbeProtocol tcp `

                    -ProbeIntervalInSeconds 15 `

| New-AzureVM -ServiceName $CloudServiceName    

            

 

# Create AppServer

New-AzureVMConfig -ImageName    $ImageName `

                  -InstanceSize $VMSize `

                  -Name         $APPSRV_1_Name `

                  -availabilitysetname $myAvailabilitySetName_APPSRV `

                  -DiskLabel "OS" `

| Add-AzureProvisioningConfig -Windows `

                              -DisableAutomaticUpdates `

                              -AdminUserName $AdminUserName `

                              -Password      $AdminPwd `

| Add-AzureDataDisk -CreateNew -DiskSizeInGB $myDataDiskSize `

                    -DiskLabel 'DataDisk0' `

                    -LUN 0 `

| Add-AzureEndpoint -Name          $myEndpointName `

                    -Protocol      tcp `

                    -LocalPort     80 `

                    -PublicPort    80 `

                    -LBSetName     $myLoadBalancerName `

                    -ProbePort     8080 `

                    -ProbeProtocol tcp `

                    -ProbeIntervalInSeconds 15 `

| New-AzureVM -ServiceName $CloudServiceName

 

# Create the DB Server

# We do not need to load balance the DB server...

# It would be better to use a SQL Server Image here..

New-AzureVMConfig -ImageName    $ImageName `

                  -InstanceSize $VMSize `

                  -Name         $DBSRV_Name `

                  -DiskLabel "OS" `

| Add-AzureProvisioningConfig -Windows `

                              -DisableAutomaticUpdates `

                              -AdminUserName $AdminUserName `

                              -Password      $AdminPwd `

| Add-AzureDataDisk -CreateNew -DiskSizeInGB $myDataDiskSize `

                    -DiskLabel 'DataDisk0' `

                    -LUN 0 `

| Add-AzureDataDisk -CreateNew -DiskSizeInGB $myDataDiskSize `

                    -DiskLabel 'DataDisk1' `

                    -LUN 1 `

| New-AzureVM -ServiceName $CloudServiceName

 

 

                    

Enjoy!

Advertisements

Deploying an Entire Environment using Azure and PowerShell

The IaaS capabilities of Azure could be very handy when you need to create temporary development/test environments during the SDLC. Automating the creation and clean-up of these environments could save a lot of time and compute time ($$).

Azure exposes an interface based on PowerShell to automate all the steps required to do this, and I spent some time researching how to properly do it. You will find many references and blog post on how to create VMs using the PowerShell API in Azure; however I did not find many updated, accurate references of how to do it for an entire environment. Probably because the API has evolved so quickly and these articles are no longer relevant.. The results are summarized in the following script which demonstrates the creation of a standard deployment of an enterprise multi-tier environment (web front-end, application server and database server).

$ErrorActionPreference = "Stop"   # stop on any error

 

function GetLatestImage($family){

$images = Get-AzureVMImage `

| where { $_.ImageFamily -eq $family } `

| Sort-Object -Descending -Property PublishedDate

 

$latestImage = $images[0]

return $latestImage

}

 

 

# Environment variables are defined here:

# ONLY LOWERCASE LETTERS HERE!!

$EnvironmentName = "azrtest"

# Create Storage Account through the Portal (vmstorageazrtest)

$StorageAccount = "vmstorage$EnvironmentName"

 

Write-Host $StorageAccount

 

$AzurePubSettingsFile = "C:\MyStuff\MyDrop\Dropbox\Personal\Windows Azure MSDN - Visual Studio Ultimate-12-19-2013-credentials.publishsettings"

$VMSize = "Small"

$Location = "Southeast Asia"

$AdminUserName = "admin2K"

$AdminPwd = "password2K"

$OSFamily = "Windows Server 2008 R2 SP1"

 

$server_A_Name = "WFEServer"

$server_B_Name = "DBServer"

$server_C_Name = "AppServer"

 

# This must be unique

$CloudServiceName = "vmstorageazrtest"

# Run GetLatestImage.ps1

$Image = GetLatestImage($OSFamily)

$ImageName = $Image.ImageName

 

# Create Storage Account through the Portal

# IF NO StorageAccount exists, ONE IS CREATED HERE!

#New-AzureStorageAccount -StorageAccountName $StorageAccount -Location $Location -Label "azrtest"

# Remove-AzureStorageAccount -StorageAccountName $StorageAccount

 

 

# Config subscription

import-azurepublishsettingsfile $AzurePubSettingsFile

#Get-AzureStorageAccount | Select Label

Set-AzureSubscription -SubscriptionName "Windows Azure MSDN - Visual Studio Ultimate" -CurrentStorageAccount $StorageAccount

 

# Create Azure Service

New-AzureService -ServiceName $CloudServiceName -Location $Location

 

# Create Machine (1) - Windows

# You can create a new virtual machine in an existing Windows Azure cloud service, or you can create a new cloud service by using the Location parameter.

New-AzureQuickVM -Windows -ServiceName $CloudServiceName -Name $server_A_Name -ImageName $ImageName -Password $AdminPwd -AdminUsername $AdminUserName -Verbose

New-AzureQuickVM -Windows -ServiceName $CloudServiceName -Name $server_B_Name -ImageName $ImageName -Password $AdminPwd -AdminUsername $AdminUserName -Verbose

New-AzureQuickVM -Windows -ServiceName $CloudServiceName -Name $server_C_Name -ImageName $ImageName -Password $AdminPwd -AdminUsername $AdminUserName -Verbose

 

 

This sample script will create 3 VMs running “Windows Server 2008 R2 SP1”: WFE, App Server and DB Server. All VMs will be “grouped” in the same Azure Cloud Service: $CloudServiceName.

If you plan to use it, make sure you edit the environment variables in the top of the script:

  • The Environment Name: $EnvironmentName
  • The storage account used to store the VHDs: $StorageAccount
  • Location of the Azure Publishing settings file: $AzurePubSettingsFile
  • Size of the VMs: $VMSize
  • Location –  use get-azurelocation for a list of locations: : $Location
  • Admin Username and Password – this is the local account you will use to remote into them: $AdminUserName and $AdminPwd
  • OS: $OSFamily

To shut down and clean-up the VMs created, you could use the following script:

#CleanUp

 

# Environment variables are defined here:

# ONLY LOWERCASE LETTERS HERE!!

$EnvironmentName = "azrtest"

# Create Storage Account through the Portal (vmstorageazrtest)

$StorageAccount = "vmstorage$EnvironmentName"

 

Write-Host $StorageAccount

 

$AzurePubSettingsFile = "C:\MyStuff\MyDrop\Dropbox\Personal\Windows Azure MSDN - Visual Studio Ultimate-12-19-2013-credentials.publishsettings"

$VMSize = "Small"

$Location = "Southeast Asia"

$AdminUserName = "admin2K"

$AdminPwd = "password2K"

$OSFamily = "Windows Server 2008 R2 SP1"

 

$server_A_Name = "WFEServer"

$server_B_Name = "DBServer"

# They must be unique

$CloudServiceName = "vmstorageazrtest"

$server_A_ImageName = $ImageName

$server_B_ImageName = $ImageName

 

# Config subscription

import-azurepublishsettingsfile $AzurePubSettingsFile

#Get-AzureStorageAccount | Select Label

Set-AzureSubscription -SubscriptionName "Windows Azure MSDN - Visual Studio Ultimate" -CurrentStorageAccount $StorageAccount

 

# Stop & Remove  VMs

$vm = Get-AzureVM -ServiceName $CloudServiceName -Name $server_A_Name

if($vm){

    Stop-AzureVM -ServiceName $CloudServiceName -Name $server_A_Name

    Remove-AzureVM -ServiceName $CloudServiceName -Name $server_A_Name

}

 

$vm = Get-AzureVM -ServiceName $CloudServiceName -Name $server_B_Name

if($vm){

    Stop-AzureVM -ServiceName $CloudServiceName -Name $server_B_Name

    Remove-AzureVM -ServiceName $CloudServiceName -Name $server_B_Name

}

 

 

# Remove any existing Azure Cloud Service

$azureService = Get-AzureService -ServiceName $CloudServiceName

if($azureService){

    Write-Host "Cloud service: $CloudServiceName found!, deleting it.."

    Remove-AzureService -ServiceName $CloudServiceName -Force

}

 

#Remove Storage Account

# Remove container first

Remove-AzureStorageContainer -Name vhds -Force

Remove-AzureStorageAccount -StorageAccountName $StorageAccount

Some aspects were not fully covered in this version of the script:

· Networking: VMs will be able to talk between each other, but we currently do not have any control on the addressing assigned to them.

· AD: VMs are created as standalone servers, not part of an AD domain.

I hope you can find this helpful and time saving.