Scale-out to the cloud, scale back to your rack

Edit on GitHub

That is a bad blog post title, really! If Steve and Ryan have this post in the Cloud Cover show news I bet they will make fun of the title. Anyway…

Imagine you have an application running in your own datacenter. Everything works smoothly, except for some capacity spikes now and then. Someone has asked you for doing something about it with low budget. Not enough budget for new hardware, and frankly new hardware would be ridiculous to just ensure capacity for a few hours each month.

A possible solution would be: migrating the application to the cloud during capacity spikes. Not all the time though: the hardware is in house and you may be a server-hugger that wants to see blinking LAN and HDD lights most of the time. I have to admit: blinking lights are cool! But I digress.

Wouldn’t it be cool to have a Powershell script that you can execute whenever a spike occurs? This script would move everything to Windows Azure. Another script should exist as well, migrating everything back once the spike cools down. Yes, you hear me coming: that’s what this blog post is about.

For those who can not wait, here’s the download: ScaleOutToTheCloud.zip (2.81 kb)

Schematical overview

Since every cool idea goes with fancy pictures, here’s a schematical overview of what could happen when you read this post to the end. First of all: you have a bunch of users making use of your application. As a good administrator, you have deployed IIS Application Request Routing as a load balancer / reverse proxy in front of your application server. Everyone is happy!

IIS Application Request Routing

Unfortunately: sometimes there are just too much users. They keep using the application and the application server catches fire.

Server catches fire!

It is time to do something. Really. Users are getting timeouts and all nasty error messages. Why not run a Powershell script that packages the entire local application for WIndows Azure and deploys the application?

Powershell to the rescue

After deployment and once the application is running in Windows Azure, there’s one thing left for that same script to do: modify ARR and re-route all traffic to Windows Azure instead of that dying server.

Request routing Azure

There you go! All users are happy again, since the application is now running in the cloud one 2, 3, or whatever number of virtual machines.

Let’s try and do this using Powershell…

The Powershell script

The Powershell script will rougly perform 5 tasks:

  • Load settings
  • Load dependencies
  • Build a list of files to deploy
  • Package these files and deploy them
  • Update IIS Application Request Routing servers

Want the download? There you go: ScaleOutToTheCloud.zip (2.81 kb)

Load settings

There are quite some parameters in play for this script. I’ve located them in a settings.ps1 file which looks like this:

# Settings (prod) $global:wwwroot = "C:\inetpub\web.local\" $global:deployProduction = 1 $global:deployDevFabric = 0 $global:webFarmIndex = 0 $global:localUrl = "web.local" $global:localPort = 80 $global:azureUrl = "scaleout-prod.cloudapp.net" $global:azurePort = 80 $global:azureDeployedSite = "http://" + $azureUrl + ":" + $azurePort $global:numberOfInstances = 1 $global:subscriptionId = "" $global:certificate = "C:\Users\Maarten\Desktop\cert.cer" $global:serviceName = "scaleout-prod" $global:storageServiceName = "" $global:slot = "Production" $global:label = Date

Let’s explain these…

$global:wwwrootThe file path to the on-premise application.
$global:deployProductionDeploy to Windows Azure?
$global:deployDevFabricDeploy to development fabric?
$global:webFarmIndexThe 0-based index of your webfarm. Look at IIS manager and note the order of your web farm in the list of webfarms.
$global:localUrlThe on-premise URL that is registered in ARR as the application server.
$global:localPortThe on-premise port that is registered in ARR as the application server.
$global:azureUrlThe Windows Azure URL that will be registered in ARR as the application server.
$global:azurePortThe Windows Azure port that will be registered in ARR as the application server.
$global:azureDeployedSiteThe final URL of the deployed Windows Azre application.
$global:numberOfInstancesNumber of instances to run on Windows Azure.
$global:subscriptionIdYour Windows Azure subscription ID.
$global:certificate
Your certificate for managing Windows Azure.
$global:serviceNameYour Windows Azure service name.
$global:storageServiceNameThe Windows Azure storage account that will be used for uploading the packaged application.
$global:slotThe Windows Azure deployment slot (production/staging)
$global:labelThe label for the deployment. I chose the current date and time.

Load dependencies

Next, our script will load dependencies. There is one additional set of CmdLets tha tyou have to install: the Windows Azure management CmdLets available at http://code.msdn.microsoft.com/azurecmdlets.

Here’s the set we load:

# Load required CmdLets and assemblies $env:Path = $env:Path + "; c:\Program Files\Windows Azure SDK\v1.2\bin\" Add-PSSnapin AzureManagementToolsSnapIn [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Web.Administration")

Build a list of files to deploy

In order to package the application, we need a text file containing all the files that should be packaged and deployed to Windows Azure. This is done by recursively traversing the directory where the on-premise application is hosted.

 

$filesToDeploy = Get-ChildItem $wwwroot -recurse | where {$_.extension -match "\..*"} foreach ($fileToDeploy in $filesToDeploy) { $inputPath = $fileToDeploy.FullName $outputPath = $fileToDeploy.FullName.Replace($wwwroot,"") $inputPath + ";" + $outputPath | Out-File FilesToDeploy.txt -Append }

Package these files and deploy them

I have been polite and included this both for development fabric as well as Windows Azure fabric. Here’s the packaging and deployment code for development fabric:

# Package & run the website for Windows Azure (dev fabric) if ($deployDevFabric -eq 1) { trap [Exception] { del -Recurse ScaleOutService continue } cspack ServiceDefinition.csdef /roleFiles:"WebRole;FilesToDeploy.txt" /copyOnly /out:ScaleOutService /generateConfigurationFile:ServiceConfiguration.cscfg # Set instance count (Get-Content ServiceConfiguration.cscfg) | Foreach-Object {$_.Replace("count=""1""","count=""" + $numberOfInstances + """")} | Set-Content ServiceConfiguration.cscfg # Run! csrun ScaleOutService ServiceConfiguration.cscfg /launchBrowser }

And here’s the same for Windows Azure fabric:

# Package the website for Windows Azure (production) if ($deployProduction -eq 1) { cspack ServiceDefinition.csdef /roleFiles:"WebRole;FilesToDeploy.txt" /out:"ScaleOutService.cspkg" /generateConfigurationFile:ServiceConfiguration.cscfg # Set instance count (Get-Content ServiceConfiguration.cscfg) | Foreach-Object {$_.Replace("count=""1""","count=""" + $numberOfInstances + """")} | Set-Content ServiceConfiguration.cscfg # Run! (may take up to 15 minutes!) New-Deployment -SubscriptionId $subscriptionId -certificate $certificate -ServiceName $serviceName -Slot $slot -StorageServiceName $storageServiceName -Package "ScaleOutService.cspkg" -Configuration "ServiceConfiguration.cscfg" -label $label $deployment = Get-Deployment -SubscriptionId $subscriptionId -certificate $certificate -ServiceName $serviceName -Slot $slot do { Start-Sleep -s 10 $deployment = Get-Deployment -SubscriptionId $subscriptionId -certificate $certificate -ServiceName $serviceName -Slot $slot } while ($deployment.Status -ne "Suspended") Set-DeploymentStatus -Status "Running" -SubscriptionId $subscriptionId -certificate $certificate -ServiceName $serviceName -Slot $slot $wc = new-object system.net.webclient $html = "" do { Start-Sleep -s 60 trap [Exception] { continue } $html = $wc.DownloadString($azureDeployedSite) } while (!$html.ToLower().Contains("<html")) }

Update IIS Application Request Routing servers

This one can be done by abusing the .NET class Microsoft.Web.Administration.ServerManager.

# Modify IIS ARR $mgr = new-object Microsoft.Web.Administration.ServerManager $conf = $mgr.GetApplicationHostConfiguration() $section = $conf.GetSection("webFarms") $webFarms = $section.GetCollection() $webFarm = $webFarms[$webFarmIndex] $servers = $webFarm.GetCollection() $server = $servers[0] $server.SetAttributeValue("address", $azureUrl) $server.ChildElements["applicationRequestRouting"].SetAttributeValue("httpPort", $azurePort) $mgr.CommitChanges()

Running the script

Of course I’ve tested this to see if it works. And guess what: it does!

The script output itself is not very interesting. I did not add logging or meaningful messages to see what it is doing. Instead you’ll just see it working.

Powershell script running

Once it has been fired up, the Windows Azure portal will soon be showing that the application is actually deploying. No hands!

Powershell deployment to Azure

After the usual 15-20 minutes that a deployment + application first start takes, IIS ARR is re-configured by Powershell.

image

And my local users can just keep browsing to http://farm.local which now simply routes requests to Windows Azure. Don’t be fooled: I actually just packaged the default IIS website and deployed it to Windows Azure. Very performant!

image

Conclusion

It works! And it’s fancy and cool stuff. I think this may be a good deployment and scale-out model in some situations, however there may still be a bottleneck in the on-premise ARR server: if this one has too much traffic to cope with, a new burning server is in play. Note that this solution will work for any website hosted on IIS: custom made ASP.NET apps, ASP.NET MVC, PHP, …

Here’s the download: ScaleOutToTheCloud.zip (2.81 kb)

This is an imported post. It was imported from my old blog using an automated tool and may contain formatting errors and/or broken images.

Leave a Comment

avatar

3 responses

  1. Avatar for KristoferA
    KristoferA October 22nd, 2010

    Unfortunately, the bottleneck for data-driven apps tend to be DB I/O bandwidth. Unless the cloud can offer rack-to-cloud db replication, real world rack<=>cloud requires a bit more...

    Just my 2 cents....

  2. Avatar for maartenba
    maartenba October 22nd, 2010

    There is SQL Azure data sync stuff available which moves your SQL Server DB to SQL Azure. May be another component of the above scenario (and may take longer than a few minutes to deploy).

  3. Avatar for KristoferA
    KristoferA October 23rd, 2010

    Yeah, imagine if you could do SQL Server log-shipping from a local live SQL Server instance to SQL Azure. That way you're not more than one transaction log backup away from doing a 'failover' to the cloud. The switch would then be: make local db readonly, backup tx log, ship, make azure live... And then the same thing in the opposite direction when going from Azure back to a local server.