# Maarten Balliauw {blog}

## An autoscaling build farm using TeamCity and Windows Azure

Cloud computing is often referred to as a cost saver due to its billing models. If we can move workloads that are seasonal to the cloud, cost reduction is something that will come. No matter if it’s really “seasonal seasonal” (e.g. a temporary high workload around the holidays) or “daily seasonal” where workloads are different depending on the time of day, these workloads have written cloud all over them.

A workload that may be seasonal is the workload done by build servers. Take TeamCity for example. A TeamCity server instruments a pool of build agents that are either idle or compiling source code into binaries. Depending on how your team is structured and when people work, there is a big chance that pool of build agents is doing nothing for several hours every day, except incurring cost. What if we could move the build agents to a platform like Windows Azure and have them autoscale, depending on the actual load on the build farm?

## Creating a build agent virtual machine

The first step in setting this brilliant scheme in motion is to set up a build agent virtual machine. We can select any virtual machine image we want for our build agent, even upload our own vhd’s if needed. I'm selecting a Windows Server 2012 image here but if you need a different OS for your build agent you can select that instead.

During the creation of this build agent, there is nothing special we should do. We can select a small/medium/large/extra large instance, give ourselves an administrator password and so on. The only important step here is that we set up an endpoint for the TeamCity build agent, listening on TCP port 9090.

Once the machine is started, we will have to install all required prerequisites for our build agent. We can connect using remote desktop (or SSH if it’s a Linux machine). On the machine I have here, I installed all .NET framework versions to ensure I'm able to build .NET projects. On a build machine for Java, we would install the correct runtimes and JDK's for our projects. Anything, really, if it is needed for the sources we’ll be building. On Windows, I typically use Web Platform Installer and Chocolatey to get this done as automated as possible.

## Installing TeamCity build agent

In order for our build agent to communicate with the TeamCity server, we have to install the build agent. We can do this by navigating to our TeamCity server from within the virtual machine and use the Install Build Agents link from the Agents page.

On a Windows server, we can use the Windows Installer but we can also use Java Web Start or even simply extract a ZIP file. This last option can be useful on a Linux machine, for example.

Installing the build agent is pretty much a next, next, finish operation. The only important thing is that we run the agent as a Windows service (or have it automatically start at boot time on other operating systems). We also want to specify the URL to our TeamCity server as well as the port on which the build agent will listen for incoming data from TeamCity. Note that this port should be the one opened in the load balancer earlier, in the case of this machine port 9090.

Before starting the build agent, make sure the local firewall allows incoming connections. Through the Windows firewall, allow incoming connections for port 9090 (and while we’re at it, for a range of ports so we can easily clone this machine and not care about the firewall anymore).

If we now start the build agent service, it should connect to our TeamCity server. Under the agents tab, we should be seeing a new unauthorized agent popping up. If that works, we’re good to go with our build agent farm.

Don’t shut down the machine just yet, we still need to prepare it for creating a build agent image.

## Creating a build agent image

While still connected through remote desktop, open a command prompt and run the sysprep /generalize command from the c:\windows\system32\sysprep folder. On Linux, there’s a similar option in the Windows Azure agent. Sysprep ensures the machine can be cloned into a new machine, getting its own settings like a hostname and IP address. A non-sysprepped machine can thus never be cloned.

Once finished, our RDP connection should be gone and our machine can be shutdown in the Windows Azure Management Portal. In fact, it must be shut down. Once that is done, we can use the Capture button and transform our virtual machine into a template we can create new virtual machines from.

The capturing process will take a couple of minutes and results in having no more build server virtual machine to be found in the Windows Azure Management Portal. Is that bad? No, we can now start cloning the machine and create multiple, all having the exact same configuration and components installed.

## Setting up multiple build agent machines

The next thing we want to have is multiple build agent machines. From the Windows Azure Management portal, we can create them. Not based on a platform image but using the image created during the previous step.

The virtual machine configuration can be whatever we want. Do we want extra small instances or extra large? It’s all up to us and our credit card. On the next page, we have to specify some more important details. First, we have to select or create a cloud service. This will be the DNS host name under which all of our build agents are going to live. We also have to specify the affinity group, in essence a setting telling Windows Azure to never unplug power or networking for all machines in this group at the same time.

We will be creating a couple of machines, so it’s important to get the next page right. Since all our machines will share the same hostname and IP address to the outside world, our build agents have to listen on different TCP ports. Make sure that the first agent maps port 9090 to port 9090, the second one 9091 to 9091 and so on. Not doing this will mess with your mind afterwards when troubleshooting.

Finish the process, let Windows Azure start the machine and create a new one. Important: same cloud service, same availability set and correct endpoint mappings!

## Configuring the build agents

Once we have several machines running, we have to connect to them using remote desktop again. This can be done through the portal. Once in, locate the build agent configuration file (c:\BuildAgent\conf\buildAgent.properties in a default installation) and set the port number on which it listens to the one that was mapped as an external endpoint. Again, agent one will listen on port 9090, agent number two on 9091 and so on. We can also set a better name for the build agent, in my case I’ve chosen to go with “agent2”. Very inspirational and all.

Save and restart the build agent (or the machine). The TeamCity server should now start listing all build agents.

Make sure to authorize them all, as we want to be sure they can connect to TeamCity server later on. Once that has been done and all build agents are listed here, we can shut them all down except for one. We want to have something running, right?

## Configuring autoscaling

It might have been a good question: why did we have to move all these machines under the same cloud service? The reason is simple: we wanted to autoscale our farm and this can only be done within one cloud service. From the cloud service, click the Scale tab and start configuring.

For this post, I’ve chosen the following values:

• Autoscale based on CPU
• Have a minimum of one instance, and a maximum of, well… all of them.
• The target CPU range is 0 to 10. For a production environment this will typically be between 60 and 80 or 40 and 80, depending on the chosen machine size for the build agents. Windows Azure will trigger an autoscaling operation if we go outside this range, having a small and low range means it will trigger a scale operation much faster. Bigger numbers means slower to respond.
• Scale up by and scale down by as well as the number of minutes to wait after the previous operations are up to you. If you want a build agent to remain online for 30 minutes after it has been started, even if CPU usage drops, set it to 30 minutes. If 2 machines should be started at once, increase that number as well.

Scaling will happen based on the average CPU percentage of all running machines. If our builds run at 100% CPU all the time on our agent we can set the thresholds a bit higher. If builds are only taking 20% we might want to run multiple agents on one machine or decrease the scaling thresholds a bit. Want to measure CPU utilization for a given build? Better read up on the TeamCity Performance Monitor then.

## Putting it to the test

Putting it to the test shouldn’t be that hard. Start some builds and make sure the agent gets loaded with builds. Once we hit the CPU threshold, Windows Azure will launch a virtual machine that was previously turned off.

Once it has booted, we will also see it surface on the TeamCity server.

Once the load goes down again, Windows Azure will shutdown machines that are below the thresholds and make sure they don’t incur costs any longer. Which is pretty impressive!

If a development team triggers a massive amount of builds during the day, Windows Azure will pretty soon scale out to a higher number of virtual build agents. And at night when there are only some builds being triggered, it will scale back to lesser instances. If, for example, we manage to run machines only for 12 hours instead of 24 hours a day, that means our build farm’s price goes down by half.

TeamCity’s architecture as well as the way Windows Azure works makes this cost reduction possible. It’s also fun to set up, it gives us a wide range of options (how about a Windows Server 2012 farm, a Linux farm and so on).

Enjoy!

## Windows Azure Traffic Manager Explained

With yesterday’s announcement on Windows Azure Traffic Manager surfacing in the management portal (as a preview), I thought it was a good moment to recap this more than 2 year old service. Windows Azure Traffic Manager allows you to control the distribution of network traffic to your Cloud Services and VMs hosted within Windows Azure.

## What is Traffic Manager?

The Windows Azure Traffic Manager provides several methods of distributing internet traffic among two or more cloud services or VMs, all accessible with the same URL, in one or more Windows Azure datacenters. At its core, it is basically a distributed DNS service that knows which Windows Azure services are sitting behind the traffic manager URL and distributes requests based on three possible profiles:

• Failover: all traffic is mapped to one Windows Azure service, unless it fails. It then directs all traffic to the failover Windows Azure service.
• Performance: all traffic is mapped to the Windows Azure service “closest” (in routing terms) to the client requesting it. This will direct users from the US to one of the US datacenters, European users will probably end up in one of the European datacenters and Asian users, well, somewhere in the Asian datacenters.
• Round-robin: Just distribute requests between various Windows Azure services defined in the Traffic Manager policy

Now I’ve started this post with the slightly bitchy tone that “this service has been around for over two years”. And that’s true! It has been in the old management portal for ages and hasn’t since left the preview stage. However don’t think nothing happened with this service: next to using Traffic Manager for cloud services, we now can also use it for distributing traffic across VM’s. Next to distributing traffic over datacenters for cloud services, we can now do this for VMs as well. What about a SharePoint farm deployed in multiple datacenters, using Traffic Manager to distribute traffic geographically?

## Why should I care?

We’ve seen it before: clouds being down. Amazon EC2, Google, Windows Azure, … They all have had their glitches. With any cloud going down, whether completely or partially, it seems a lot of websites “in the cloud” are down at that time. Most comments you read on Twitter at those times are along the lines of “outrageous!” and “don’t go cloud!”. While I understand these comments, I think they are wrong. These “clouds” can fail. They are even designed to fail, and often provide components and services that allow you to cope with these failures. You just have to expect failure at some point in time and build it into your application.

Yes, I just told you to expect failure when going to the cloud. But don’t consider a failing cloud a bad cloud or a cloud that is down. For your application, a “failing” cloud or server or database should be nothing more than a scaling operation. The only thing is: it’s scaling down to zero. If you design your application so that it can scale out, you should also plan for scaling “in”, eventually to zero. Use different availability zones on Amazon, and if you’re a Windows Azure user you are protected by fault domains within the datacenter, and Traffic Manager can save your behind cross-datacenter. Use it!

## My thoughts on Traffic Manager

Let’s come back to that “2 year old service”. Don’t let that or the fact that is “is still a preview” hold you back from using Traffic Manager. Our MyGet web application is making use of it since it was first introduced. While in the beginning we used it for performance reasons (routing US traffic to a US datacenter and EU traffic to a EU datacenter), we’ve changed the strategy and are now using it as a failover to the North Europe datacenter in which nothing is deployed. The screenshot below highlights a degradation (because there indeed is no deployment in Europe North, currently).

But why failover to a datacenter in which no deployments are done? Well, because if West Europe datacenter would fail, we can simply spin up a new deployment in North Europe. Yes, there will be some downtime, but the last thing we want to have in such situation is downtime from DNS propagation taking too long. Now we simply map www.myget.org to our Traffic Manager domain and whenever we need to switch, Traffic Manager takes care of the DNS part.

In general, Traffic Manager has probably been the most stable service in the Windows Azure platform. I haven’t experienced any issues so far with Traffic Manager over more than two years, preview mode or not.

Enjoy!

Update: Alexandre Brisebois, a colleague MVP, has some additional insights to share.

## Running unit tests when deploying ASP.NET to Windows Azure Web Sites

One of the well-loved features of Windows Azure Web Sites is the fact that you can simply push our ASP.NET application’s source code to the platform using Git (or TFS or DropBox) and that sources are compiled and deployed on your Windows Azure Web Site. If you’ve checked the management portal earlier, you may have noticed that a number of deployment steps are executed: the deployment process searches for the project file to compile, compiles it, copies the build artifacts to the web root and has your website running. But did you know you can customize this process?

[update] Mstest seems to work now as well, using the console runner from VS2012.

## Customizing the build process

To get an understanding of how to customize the build process, I want to explain you how this works. In the root of your repository, you can add a .deployment file, containing a simple directive: which command should be run upon deployment.

1 [config]
2 command = build.bat

This command can be a batch file, a PHP file, a bash file and so on. As long as we can tell Windows Azure Web Sites what to execute. Let’s go with a batch file.

1 @echo off
2 echo This is a custom deployment script, yay!

When pushing this to Windows Azure Web Sites, here’s what you’ll see:

In this batch file, we can use some environment variables to further customize the script:

• DEPLOYMENT_SOURCE - The initial "working directory"
• DEPLOYMENT_TARGET - The wwwroot path (deployment destination)
• DEPLOYMENT_TEMP - Path to a temporary directory (removed after the deployment)
• MSBUILD_PATH - Path to msbuild

After compiling, you can simply xcopy our application to the %DEPLOYMENT_TARGET% variable and have your website live.

## Generating deployment scripts

Creating deployment scripts can be a tedious job, good thing that the azure-cli tools are there! Once those are installed, simply invoke the following command and have both the .deployment file as well as a batch or bash file generated:

1 azure site deploymentscript --aspWAP "path\to\project.csproj"

For reference, here’s what is generated:

 1 @echo off
2
3 :: ----------------------
4 :: KUDU Deployment Script
5 :: ----------------------
6
7 :: Prerequisites
8 :: -------------
9
10 :: Verify node.js installed
11 where node 2>nul >nul
12 IF %ERRORLEVEL% NEQ 0 (
13   echo Missing node.js executable, please install node.js, if already installed make sure it can be reached from current environment.
14   goto error
15 )
16
17 :: Setup
18 :: -----
19
20 setlocal enabledelayedexpansion
21
22 SET ARTIFACTS=%~dp0%artifacts
23
24 IF NOT DEFINED DEPLOYMENT_SOURCE (
25   SET DEPLOYMENT_SOURCE=%~dp0%.
26 )
27
28 IF NOT DEFINED DEPLOYMENT_TARGET (
29   SET DEPLOYMENT_TARGET=%ARTIFACTS%\wwwroot
30 )
31
32 IF NOT DEFINED NEXT_MANIFEST_PATH (
33   SET NEXT_MANIFEST_PATH=%ARTIFACTS%\manifest
34
35   IF NOT DEFINED PREVIOUS_MANIFEST_PATH (
36     SET PREVIOUS_MANIFEST_PATH=%ARTIFACTS%\manifest
37   )
38 )
39
40 IF NOT DEFINED KUDU_SYNC_COMMAND (
41   :: Install kudu sync
42   echo Installing Kudu Sync
43   call npm install kudusync -g --silent
44   IF !ERRORLEVEL! NEQ 0 goto error
45
46   :: Locally just running "kuduSync" would also work
47   SET KUDU_SYNC_COMMAND=node "%appdata%\npm\node_modules\kuduSync\bin\kuduSync"
48 )
49 IF NOT DEFINED DEPLOYMENT_TEMP (
50   SET DEPLOYMENT_TEMP=%temp%\___deployTemp%random%
51   SET CLEAN_LOCAL_DEPLOYMENT_TEMP=true
52 )
53
54 IF DEFINED CLEAN_LOCAL_DEPLOYMENT_TEMP (
55   IF EXIST "%DEPLOYMENT_TEMP%" rd /s /q "%DEPLOYMENT_TEMP%"
56   mkdir "%DEPLOYMENT_TEMP%"
57 )
58
59 IF NOT DEFINED MSBUILD_PATH (
60   SET MSBUILD_PATH=%WINDIR%\Microsoft.NET\Framework\v4.0.30319\msbuild.exe
61 )
62
63 ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
64 :: Deployment
65 :: ----------
66
67 echo Handling .NET Web Application deployment.
68
69 :: 1. Build to the temporary path
70 %MSBUILD_PATH% "%DEPLOYMENT_SOURCE%\path.csproj" /nologo /verbosity:m /t:pipelinePreDeployCopyAllFilesToOneFolder /p:_PackageTempDir="%DEPLOYMENT_TEMP%";AutoParameterizationWebConfigConnectionStrings=false;Configuration=Release
71 IF !ERRORLEVEL! NEQ 0 goto error
72
73 :: 2. KuduSync
74 echo Kudu Sync from "%DEPLOYMENT_TEMP%" to "%DEPLOYMENT_TARGET%"
75 call %KUDU_SYNC_COMMAND% -q -f "%DEPLOYMENT_TEMP%" -t "%DEPLOYMENT_TARGET%" -n "%NEXT_MANIFEST_PATH%" -p "%PREVIOUS_MANIFEST_PATH%" -i ".git;.deployment;deploy.cmd" 2>nul
76 IF !ERRORLEVEL! NEQ 0 goto error
77
78 ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
79
80 goto end
81
82 :error
83 echo An error has occured during web site deployment.
84 exit /b 1
85
86 :end
87 echo Finished successfully.
88 

This script does a couple of things:

• Ensure node.js is installed on Windows Azure Web Sites (needed later on for synchronizing files)
• Setting up a bunch of environment variables
• Run msbuild on the project file we specified
• Use kudusync (a node.js based tool, hence node.js) to synchronize modified files to the wwwroot of our site

Try it: after pushing this to Windows Azure Web Sites, you’ll see the custom script being used. Not much added value so far, but that’s what you have to provide.

## Unit testing before deploying

Unit tests would be nice! All you need is a couple of unit tests and a test runner. You can add it to your repository and store it there, or simply download it during the deployment. In my example, I’m using the Gallio test runner because it runs almost all test frameworks, but feel free to use the test runner for NUnit or xUnit instead.

Somewhere before the line that invokes msbuild and ideally in the “setup” region of the deployment script, add the following:

 1 IF NOT DEFINED GALLIO_COMMAND (
2   IF NOT EXIST "%appdata%\Gallio\bin\Gallio.Echo.exe" (
5     curl -O http://stahlforce.com/dev/unzip.exe
6     IF !ERRORLEVEL! NEQ 0 goto error
7
11     IF !ERRORLEVEL! NEQ 0 goto error
12
13     :: Extracting Gallio
14     echo Extracting Gallio
15     unzip -q -n GallioBundle-3.4.14.0.zip -d %appdata%\Gallio
16     IF !ERRORLEVEL! NEQ 0 goto error
17   )
18
19   :: Set Gallio runner path
20   SET GALLIO_COMMAND=%appdata%\Gallio\bin\Gallio.Echo.exe
21 )

See what happens there?  We check if the local system on which your files are stored in WindowsAzure Web Sites already has a copy of the Gallio.Echo.exetest runner. If not, let’s download a tool which allows us to unzip. Next, the entire Gallio test runner is downloaded and extracted. As a final step, the %GALLIO_COMMAND% variable is populated with the full path to the test runner executable.

Right before the line that calls “kudusync”, add the following:

1 echo Running unit tests
2 "%GALLIO_COMMAND%" "%DEPLOYMENT_SOURCE%\SampleApp.Tests\bin\Release\SampleApp.Tests.dll"
3 IF !ERRORLEVEL! NEQ 0 goto error

Yes, the name of your test assembly will be different, you should obviously change that. What happens here? Well, we’re invoking the test runner on our unit tests. If it fails, we abort deployment. Push it to Windows Azure and see for yourself. Here’s what is displayed on success:

All green! And on failure, we get:

In the portal, you can clearly see that deployment was aborted:

That’s it. Enjoy!

## Storing user uploads in Windows Azure blob storage

On one of the mailing lists I follow, an interesting question came up: “We want to write a VSTO plugin for Outlook which copies attachments to blob storage. What’s the best way to do this? What about security?”. Shortly thereafter, an answer came around: “That can be done directly from the client. And storage credentials can be encrypted for use in your VSTO plugin.”

While that’s certainly a solution to the problem, it’s not the best. Let’s try and answer…

## What’s the best way to uploads data to blob storage directly from the client?

The first solution that comes to mind is implementing the following flow: the client authenticates and uploads data to your service which then stores the upload on blob storage.

While that is in fact a valid solution, think about the following: you are creating an expensive layer in your application that just sits there copying data from one network connection to another. If you have to scale this solution, you will have to scale out the service layer in between. If you want redundancy, you need at least two machines for doing this simple copy operation… A better approach would be one where the client authenticates with your service and then uploads the data directly to blob storage.

This approach allows you to have a “cheap” service layer: it can even run on the free version of Windows Azure Web Sites if you have a low traffic volume. You don’t have to scale out the service layer once your number of clients grows (at least, not for the uploading scenario).But how would you handle uploading to blob storage from a security point of view…

## What about security? Shared access signatures!

The first suggested answer on the mailing list was this: “(…) storage credentials can be encrypted for use in your VSTO plugin.” That’s true, but you only have 2 access keys to storage. It’s like giving the master key of your house to someone you don’t know. It’s encrypted, sure, but still, the master key is at the client and that’s a potential risk. The solution? Using a shared access signature!

Shared access signatures (SAS) allow us to separate the code that signs a request from the code that executes it. It basically is a set of query string parameters attached to a blob (or container!) URL that serves as the authentication ticket to blob storage. Of course, these parameters are signed using the real storage access key, so that no-one can change this signature without knowing the master key. And that’s the scenario we want to support…

On the service side, the place where you’ll be authenticating your user, you can create a Web API method (or ASMX or WCF or whatever you feel like) similar to this one:

public class UploadController
: ApiController
{
[Authorize]
public string Put(string fileName)
{
var account = CloudStorageAccount.DevelopmentStorageAccount;
var blobClient = account.CreateCloudBlobClient();
blobContainer.CreateIfNotExists();

var blob = blobContainer.GetBlockBlobReference("customer1-" + fileName);

var uriBuilder = new UriBuilder(blob.Uri);
uriBuilder.Query = blob.GetSharedAccessSignature(new SharedAccessBlobPolicy
{
Permissions = SharedAccessBlobPermissions.Write,
SharedAccessStartTime = DateTime.UtcNow,
}).Substring(1);

return uriBuilder.ToString();
}
}

This method does a couple of things:

• Authenticate the client using your authentication mechanism
• Create a blob reference (not the actual blob, just a URL)
• Signs the blob URL with write access, allowed from now until now + 5 minutes. That should give the client 5 minutes to start the upload.

On the client side, in our VSTO plugin, the only thing to do now is call this method with a filename. The web service will create a shared access signature to a non-existing blob and returns that to the client. The VSTO plugin can then use this signed blob URL to perform the upload:

Uri url = new Uri("http://...../uploads/customer1-test.txt?sv=2012-02-12&st=2012-12-18T08%3A11%3A57Z&se=2012-12-18T08%3A16%3A57Z&sr=b&sp=w&sig=Rb5sHlwRAJp7mELGBiog%2F1t0qYcdA9glaJGryFocj88%3D");
var blob = new CloudBlockBlob(url);
blob.Properties.ContentType = "test/plain";

using (var data = new MemoryStream(
Encoding.UTF8.GetBytes("Hello, world!")))
{
}

Easy, secure and scalable. Enjoy!

## Hands-on Windows Azure Services for Windows

A couple of weeks ago, Microsoft announced their Windows Azure Services for Windows Server. If you’ve ever heard about the Windows Azure Appliance (which is vaporware imho :-)), you’ll be interested to see that the Windows Azure Services for Windows Server are in fact bringing the Windows Azure Services to your datacenter. It’s still a Technical Preview, but I took the plunge and installed this on a bunch of virtual machines I had lying around. In this post, I’ll share you with some impressions, ideas, pains and speculations.

Why would you run Windows Azure Services in your own datacenter? Why not! You will make your developers happy because they have access to all services they are getting to know and getting to love. You’ll be able to provide self-service access to SQL Server, MySQL, shared hosting and virtual machines. You decide on the quota. And if you’re a server hugger like a lot of companies in Belgium: you can keep hugging your servers. I’ll elaborate more on the “why?” further in this blog post.

Note: Currently only SQL Server, MySQL, Web Sites and Virtual Machines are supported in Windows Azure Services for Windows Server. Not storage, not ACS, not Service Bus, not...

You can sign up for my “I read your blog plan” at http://cloud.balliauw.net and create your SQL Server databases on the fly! (I’ll keep this running for a couple of days, if it’s offline you’re too late). It's down.

## My setup

Since I did not have enough capacity to run enough virtual machines (you need at least four!) on my machine, I decided to deploy the Windows Azure Services for Windows Server on a series of virtual machines in Windows Azure’s IaaS offering.

You will need servers for the following roles:

• Controller node (the management portal your users will be using)
• SQL Server (can be hosted on the controller node)
• Storage server (can be on the cntroller node as well)

If you want to host Windows Azure Websites (shared hosting):

• At least one load balancer node (will route HTTP(S) traffic to a frontend node)
• At least one frontend node (will host web sites, more frontends = more websites / redundancy)
• At least one publisher node (will serve FTP and Webdeploy)

If you want to host Virtual Machines:

• A System Center 2012 SP1 CTP2 node (managing VM’s)
• At least one Hyper-V server (running VM’s)

Being a true ITPro (forgot the <irony /> element there…), I decided I did not want to host those virtual machines on the public Internet. Instead, I created a Windows Azure Virtual Network. Knowing CIDR notation (<irony />), I quickly crafted the BalliauwCloud virtual network: 172.16.240.0/24.

So a private network… Then again: I wanted to be able to access some of the resources hosted in my cloud on the Internet, so I decided to open up some ports in Windows Azure’s load balancer and firewall so that my users could use the SQL Sever both internally (172.16.240.9) and externally (sql1.cloud.balliauw.net). Same with high-density shared hosting in the form of Windows Azure Websites by the way.

Being a Visio pro (no <irony /> there!), here’s the schematical overview of what I setup:

Nice, huh? Even nicer is my to-be diagram where I also link crating Hyper-V machines to this portal (not there yet…):

## My setup experience

I found the detailed step-by-step installation guide and completed the installation as described. Not a great success! The Windows Azure Websites feature requires a file share and I forgot to open up a firewall port for that. The result? A failed setup. I restarted setup and ended with 500 Internal Server Terror a couple of times. Help!

Being a Technical Preview product, there is no support for cleaning / restarting a failed setup. Luckily, someone hooked me up with the team at Microsoft who built this and thanks to Andrew (thanks, Andrew!), I was able to continue my setup.

If everything works out for your setup: enjoy! If not, here’s some troubleshooting tips:

Keep an eye on the C:\inetpub\MgmtSvc-ConfigSite\trace.txt  log file. It holds valuable information, as well as the event log (Applications and Services Log > Microsoft > Windows > Antares).

If you’re also experiencing issues and want to retry installation, here are the steps to clean your installation:

1. On the controller node: stop services:
net stop w3svc
net stop WebFarmService
net stop ResourceMetering
net stop QuotaEnforcement
2. In IIS Manager (inetmgr), clean up the Hosting Administration REST API service. Under site MgmtSvc-WebSites:
- Remove IIS application HostingAdministration (just the app, NOT the site itself)
3. Drop databases, and logins by running the SQL script: C:\inetpub\MgmtSvc-ConfigSite\Drop-MgmtSvcDatabases.sql
4. (Optional, but helped in my case) Repair permissions
5. Clean up registry keys by deleting the three folders under the following registry key (NOT the key itself, just the child folders):
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\IIS Extensions\Web Hosting Framework

Delete these folders: HostingAdmin, Metering, Security
6. Restart IIS
net start w3svc
7. Re-run the installation with https://localhost:30101/

## Configuration

After installation comes configuration. Configuration depends on the services you want to offer. I’m greedy so I wanted to provide them all. First, I registered my SQL Server and told the Windows Azure Services for Windows Server management portal that I have about 80 GB to spare for hosting my user’s databases. I did the same with MySQL (setup is similar):

You can add more SQL Servers and even define groups. For example, if you have a SQL Server which can be used for development purposes, add that one. If you have a high-end, failover setup for production, you can add that as a separate group so that only designated users can create databases on that SQL Server cluster of yours.

For Windows Azure Web Sites, I deployed one node of every role that was required:

What I liked in this setup is that if I want to add one of these roles, the only thing required is a fresh Windows Server 2008 R2 or 2012. No need to configure the machine: the Windows Azure Services for Windows Server management portal does that for me. All I have to do as an administrator in order to grow my pool of shared resources is spin up a machine and enter the IP address. Windows Azure Services for Windows Server management portal  takes care of the installation, linking, etc.

The final step in offering services to my users is creating at least one plan they can subscribe to. Plans define the services provided as well as the quota on these services. Here’s an example quota configuration for SQL Server in my “Cloud Basics” plan:

Plans can be private (you assign them to a user) or public (users can self-subscribe, optionally only when they have a specific access code).

## End-user experience

As an end user, I can have a plan. Either I enroll myself or an administrator enrolls me. You can sign up for my “I read your blog plan” at http://cloud.balliauw.net and create your SQL Server databases on the fly! (I’ll keep this running for a couple of days, if it’s offline you’re too late).

Side note: as an administrator, you can modify this page. It’s a bunch of ASP.NET MVC .cshtml files located under C:\inetpub\MgmtSvc-TenantSite\Views.

After signing in, you’ll be given access to a portal which resembles Windows Azure’s portal. You’ll have an at-a-glance look at all services you are using and can optionally just delete your account. Here’s the initial portal:

You’ll be able to manage services yourself, for example create a new SQL Server database:

After creating a database, you can see the connection information from within the portal:

Just imagine you could create databases on-the-fly, whenever you need them, in your internal infrastructure. Without an administrator having to interfere. Without creating a support ticket or a formal request…

## Speculations

I’m not sure if I’m supposed to disclose this information, but… The following paragraphs are based on what I can see in the installation of my “private cloud” using Windows Azure Services for Windows Server.

• I have a suspicion that the public cloud services can enter in Windows Azure Services for Windows Server. The SQL Server database for this management portal contains various additional tables, such as a table in which SQL Azure servers can be added to a pool linked to a plan. My guess is that you’ll be able to spread users and plans between public cloud (maybe your cheap test databases can go there) and private cloud (production applications run on a SQL Server cluster in your basement).
• The management portals are clearly build with extensibility in mind. Yes, I’ve cracked open some assemblies using ILSpy, yes I’ve opened some of the XML configuration files in there. I expect the recently announced Service Bus for Windows Server to pop up in this product as well. And who knows, maybe a nice SDK to create your own services embedded in this portal so that users can create mailboxes as they please. Or link to a VMWare cloud, I know they have management API’s.

## Conclusion

I’ve opened this post with a “Why?”, let’s end it with that question. Why would you want to use this? The product was announced on Microsoft’s hosting subsite, but the product name (Windows Azure Services for Windows Server) and my experience with it so far makes me tend to think that this product is a fit for any enterprise!

You will make your developers happy because they have access to all services they are getting to know and getting to love. You’ll be able to provide self-service access to SQL Server, MySQL, shared hosting and virtual machines. You decide on the quota. You manage this. The only thing you don’t have to manage is the actual provisioning of services: users can use the self-service possibilities in Windows Azure Services for Windows Server.

Want your departments to be able to quickly setup a Wordpress or Drupal site? No problem: using Web Sites, they are up and running. And depending on the front-end role you assign them, you can even put them on internet, intranet or both. (note: this is possible throug some Powershell scripting, by default it's just one pool of servers there)

The fact that there is support for server groups (say, development servers and high-end SQL Server clusters or 8-core IIS machines running your web applications) makes it easy for administrators to grant access to specific resources while some other resources are reserved for production applications. And I suspect this will extend to the public cloud making it possible to go hybrid if you wish. Some services out there, some in your basement.

I’m keeping an eye on this one.

Note: You can sign up for my “I read your blog plan” at http://cloud.balliauw.net and create your SQL Server databases on the fly! (I’ll keep this running for a couple of days, if it’s offline you’re too late). It's down.

## Tweaking Windows Azure Web Sites

A while ago, I was at a customer who wanted to run his own WebDAV server (using www.sabredav.org) on Windows Azure Web Sites. After some testing, it seemed that this PHP-based WebDAV server was missing some configuration at the webserver level. Some HTTP keywords required for the WebDAV protocol were not mapped to the PHP runtime making it virtually impossible to run a custom WebDAV implementation on PHP. Unless there’s some configuration possible…

I’ve issued a simple phpinfo(); on Windows Azure Websites, simply outputting the PHP configuration and all available environment variables in Windows Azure Websites. This revealed the following interesting environment variable:

Aha! That’s an interesting one! It’s basically the configuration of the IIS web server you are running. It contains which configuration sections can be overridden using your own Web.config file and which ones can not. I’ve read the file (it seems you have access to this path) and have placed the output of it here: applicationhost.config (70.04 kb). There’s also a file called rootweb.config: rootweb.config (36.66 kb)

## Overridable configuration parameters

For mere humans not interested in reading through the entire applicationhost.config and rootweb.config here’s what you can override in your own Web.config. Small disclaimer: these are implementation details and may be subject to change. I’m not Microsoft so I can not predict if this will all continue to work. Use your common sense.

 Configuration parameter Can be overriden in Web.config? system.webServer.caching Yes system.webServer.defaultDocument Yes system.webServer.directoryBrowse Yes system.webServer.httpErrors Yes system.webServer.httpProtocol Yes system.webServer.httpRedirect Yes system.webServer.security.authorization Yes system.webServer.security.requestFiltering Yes system.webServer.staticContent Yes system.webServer.tracing.traceFailedRequests Yes system.webServer.urlCompression Yes system.webServer.validation Yes system.webServer.rewrite.rules Yes system.webServer.rewrite.outboundRules Yes system.webServer.rewrite.providers Yes system.webServer.rewrite.rewriteMaps Yes system.webServer.externalCache.diskCache Yes system.webServer.handlers Yes, but some are locked system.webServer.modules Yes, but some are locked

All others are probably not possible.

## Project Kudu

There are some interesting things in the applicationhost.config (70.04 kb). Of course, you decide what’s interesting so read for yourself. Here’s what I found interesting: project Kudu is in there! Project Kudu? Yes, the open-source engine behind Windows Azure Web Sites (which implies that you can in fact host your own Windows Azure Web Sites-like service).

If you look at the architectural details, here’s an interesting statement:

The Kudu site runs in the same sandbox as the real site. This has some important implications.

First, the Kudu site cannot do anything that the site itself wouldn't be able to do itself. (…) But being in the same sandbox as the site, the only thing it can harm is the site itself.

Furthermore, the Kudu site shares the same quotas as the site. That is, the CPU/RAM/Disk used by the Kudu service is counted toward the site's quota. (…)

So to summarize, the Kudu services completely relies on the security model of the Azure Web Site runtime, which keeps it both simple and secure.

Proof can be found in applicationhost.config. If you look at the <sites /> definition, you’ll see two sites are defined. Your site, and a companion site named ~1yoursitename. The first one, of course, runs your site. The latter runs project Kudu which allows you to git push and use webdeploy.

In rootweb.config (36.66 kb), you’ll find the loadbalanced nature of Windows Azure Web Sites. A machine key is defined there which will be the same for all your web sites instances, allowing you to share session state, forms authentication cookies etc.

## My PHP HTTP verbs override

To fix the PHP HTTP verb mapping, here’s the Web.config I’ve used at the customer, simply removing and re-adding the PHP handler:

 1 <?xml version="1.0" encoding="UTF-8"?>
2 <configuration>
3     <system.webServer>
4         <handlers>
5             <remove name="PHP53_via_FastCGI" />
7                  verb="GET, PUT, POST, HEAD, OPTIONS, TRACE, PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, LOCK, UNLOCK" modules="FastCgiModule" scriptProcessor="D:\Program Files (x86)\PHP\v5.3\php-cgi.exe"
8                  resourceType="Either" requireAccess="Script" />
9         </handlers>
10     </system.webServer>
11 </configuration>

## Setting up a webfarm using Windows Azure Virtual Machines

With the release of Microsoft’s Windows Azure Virtual Machines, a bunch of new scenarios became available on their cloud platform. If you plan to host multiple web applications, you can either go with Windows Azure Web Sites or go with a webfarm you create using the new IaaS capabilities. The first is okay for any type of application, the latter may be suitable when running a large-scale web application that can not be deployed easily in the PaaS offering. In this blog post, I’ll show you how to build a webfarm with (free!) load balancing.

Note: I’ll be using the built-in Windows Azure load balancer. If required, you can also deploy your own load balancer VM or reverse proxy. But since the Windows Azure load balancer comes with no extra cost, I think it’s the better choice for a lot of scenarios.

## Creating a first virtual machine

After logging in to the new Windows Azure management portal, create a new virtual machine. You can choose to create a Linux or a Windows machine from a template or upload your own VM. I’ll go with a Windows machine but everything explained in this post is valid for a Linux webfarm, too.

Navigate through the wizard, selecting the VM size and administrator username of choice. In step 3 where you have to specify the DNS name and some other settings, be sure to choose an affinity group (giving better networking performance due to the fact that machines are on the same network in the Windows Azure datacenter). The DNS name can be anything you want to name your webfarm.

Before finishing the wizard, there is an important thing to do: in step 4, make sure to create an availability group in which all machines of the webfarm will reside. An availability group ensures that whenever maintenance occurs in the datacenter, this only occurs on one or some of your webfarm machines and not on all at once.

## Adding an HTTP endpoint to the first machine

After the first virtual machine has been created, navigate to its configuration dashboard in the Windows Azure management portal. In order to have port 80 connected to this machine, a new endpoint should be added to the machine. Add the endpoints of choice, I chose to have port 80 open.

It is important to understand that the endpoints added here are only opened at the load balancer level. That’s right: even a single machine will be behind a load balancer. This is incredibly powerful, as you’ll see when we add a new machine to our IaaS webfarm. It also poses an extra configuration step for single machines though: you’ll have to open port 80 on the machine’s firewall, too. You can safely use remote desktop (Windows) or SSH (Linux) to do so:

## Cloning the first machine

To make things easy, I’ve first configured IIS on the first machine. I simply enabled the webserver and made sure Windows Firewall allows connections to IIS. From this point on, I simply want to clone this machine and add it to my webfarm.

The first thing to do when cloning (or “capturing”) a VM is “sysprepping” it. On Linux, there’s a similar option in the Windows Azure agent. Sysprep ensures the machine can be cloned into a new machine, getting it’s own settings like a hostname and IP address. A non-sysprepped machine can thus never be cloned.

After sysprepping the machine, shut it down. If you’ve selected the option during sysprep, the machine will automatically shutdown. Otherwise you can do so through remote desktop or SSH, or simply through the Windows Azure portal.

Next, click the “Capture” button to create a disk image from this machine. Give it a name and  check the “Yes, I’ve sysprepped the machine” checkbox in order to be able to continue.

After clicking the “ok” button, Windows Azure will create an image of our first webserver.

After the image has been created, you’ll notice that your first webserver has disappeared! This is normal: the machine has been disemboweled in order to create a template from it. You can now simply re-create this machine using the same settings as before, except you can now base it on this newly created VM image instead of basing it off a VM template Microsoft provides.

In the endpoints configuration, make sure to add the HTTP endpoint again listening on port 80.

## Creating a second virtual machine

To create the second machine in your webfarm, create a fresh virtual machine. As the base disk, choose the image we’ve created earlier:

In step 3 of the machine creation, make sure to connect this machine to our existing web server. In step 4, locate the VM in the same availability set.

You now have two machines running, yet they aren’t load balanced at this moment. You’ll notice that both machines are already behind the same hostname (http://webfarm.cloudapp.net) and that they share the same public virtual IP address. This is due to the fact that we “linked” the machines earlier. If you don’t, you will never be able to use the out-of-the-box load balancer that comes with Windows Azure. This also means that the public remote desktop endpoint for both machines will be different: there’s only one IP address exposed to the outside world so you’ll have to think about endpoints.

Don’t add the HTTP endpoint to this machine just yet.

## Configuring the Windows Azure load balancer

The last part of setting up our webfarm will be load balancing.  This is in fact really, really easy. Simply go to second machine’s dashboard in the Windows Azure portal and navigate to the Endpoints tab. We’ve already added public HTTP endpoints on our first machine, which means for our second machine we can just subscribe to load balancing:

Easy, huh? You now have free round-robin load balancing with checks every few seconds to ensure that all machines are up and running. And since we linked these machines through an availability set, they are on different fault domains in the datacenter reducing the chance of errors due to malfunctioning hardware or maintenance. You can safely shut down a machine too. In short: anything you’d expect from a load balancer (except sticky sessions).

## Final words

There is of course more to it. In ASP.NET, you’ll have to configure machine keys and such in the same way you would do it on-premise. But at the infrastructure level, we’re covered. Enjoy! And be sure to brag about this adventure to any IT pro you know :-)

## Use NuGet Package Restore to avoid pushing assemblies to Windows Azure Websites

Windows Azure Websites allows you to publish a web site in ASP.NET, PHP, Node, … to Windows Azure by simply pushing your source code to a TFS or Git repository. But how does Windows Azure Websites manage dependencies? Do you have to check-in your assemblies and NuGet packages into source control? How about no…

NuGet 1.6 shipped with a great feature called NuGet Package Restore. This feature lets you use NuGet packages without adding them to your source code repository. When your solution is built by Visual Studio (or MSBuild, which is used in Windows Azure Websites), a build target calls nuget.exe to make sure any missing packages are automatically fetched and installed before the code is compiled. This helps you keep your source repo small by keeping large packages out of version control.

## Enabling NuGet Package Restore

Enabling NuGet package restore can be done from within Visual Studio. Simply right-click your solution and click the “Enable NuGet Package Restore” menu item.

Visual Studio will now do the following with the projects in your solution:

• Create a .nuget folder at the root of your solution, containing a NuGet.exe and a NuGet build target
• Import this NuGet target into all your projects so that MSBuild can find, download and install NuGet packages on-the-fly when creating a build

Be sure to push the files in the .nuget folder to your source control system. The packages folder is not needed, except for the repositories.config file that sits in there.

## But what about my non-public assembly references? What if I don't trust auto-updating from NuGet.org?

Good question. What about them? A simple answer would be to create NuGet packages for them. And if you already have NuGet packages for them, things get even easier. Make sure that you are hosting these packages in an online feed which is not the public NuGet repository at www.nuget.org, unless you want your custom assemblies out there in public. A good choice would be to checkout www.myget.org and host your packages there.

But then a new question surfaces: how do I link a custom feed to my projects? The answer is pretty simple: in the .nuget folder, edit the NuGet.targets file. In the PackageSources element, you can supply a semicolon (;) separated list of feeds to check for packages:

 1 <?xml version="1.0" encoding="utf-8"?>
2 <Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
3     <PropertyGroup>
4         <!-- ... -->
5
6         <!-- Package sources used to restore packages. By default will used the registered sources under %APPDATA%\NuGet\NuGet.Config -->
7         <PackageSources>"http://www.myget.org/F/chucknorris;http://www.nuget.org/api/v2"</PackageSources>
8
9         <!-- ... -->
10     </PropertyGroup>
11
12     <!-- ... -->
13 </Project>

By doing this and pushing the targets file to your Windows Azure Websites Git or TFS repo, the build system backing Windows Azure Websites will go ahead and download your packages from an external location, not cluttering your sources. Which makes for one, happy cloud.

## GitHub for Windows Azure Websites

With the new release of Windows Azure and Windows Azure Websites, a lot of new scenarios with Windows Azure just became possible. One I like a lot, especially since Appharbor and Heroku have similar offers too, is the possibility to push source code (ASP.NET or PHP) to Windows Azure instead of binaries using Windows Azure Websites.

Not everyone out there is a command-line here though: if you want to use Git as a mechanism of pushing sources to Windows Azure Websites chances are you may go crazy if you are unfamiliar with command-line git commands. Luckily, a couple of weeks ago, GitHub released GitHub for Windows. It features an easy-to-use GUI on top of GitHub repositories. And with a small trick also on top of Windows Azure Websites.

## Setting up a Windows Azure Website

Since you’re probably still unfamiliar with Windows Azure Websites, let me guide you through the setup. It’s a simple process. First of all, navigate to the new Windows Azure portal. It looks different than the one you’re used to but it’s way easier to use. In the toolbar at the bottom, click New, select Web site, Quick Create and enter a hostname of choice. I chose “websiteswithgit”:

After a couple of seconds, you’ll be presented with the dashboard of your newly created Windows Azure Website. This dashboard features a lot of interesting metrics about your website such as data traffic, CPU usage, errors, … It also displays the available means for publishing a site to Windows Azure Websites: TFS deploy, Git deploy, Webdeploy and FTP publishing. That’s it: your website has been set up and if you navigate to the newly created URL, you’ll be greeted with the default Windows Azure Websites landing page.

## Setting up Git publishing

Since we’ll be using Git, click the Set up Git Publishing option.

If you haven’t noticed already: Windows Azure Websites makes Windows Azure a lot easier. After a couple of seconds, Git publishing is configured and all it takes to deploy your website is commit your source code, whether ASP.NET, ASP.NET Webpages or PHP to the newly created Git repository. Windows Azure Websites will take care of the build process (cool!) and will deploy this to Windows Azure in just a couple of seconds. Whoever told you deploying to Windows Azure takes ages lied to you!

## Connecting GitHub for Windows to Windows Azure Websites

After setting up Git publishing, you probably have noticed that there’s a Git repository URL being displayed. Copy this one to your clipboard as we’ll be needing it in a minute. Open GitHub for Windows, right-click the UI and choose to “open a shell here”. Make sure you’re in the folder of choice. Next, issue a “git clone <url>” command, where <url> of course is the Git repository URL you’ve just copied.

The (currently empty) Windows Azure Website Git repository will be cloned onto your system. Now close this command-line (I promised we would use GitHub for Windows instead).

Open the folder in which you cloned the Git repo and drag it onto GitHub for Windows. It will look kind of empty, still:

Next, add any file you want. A PHP file, a plain HTML file or a complete ASP.NET or ASP.NET MVC Web Application. GitHub for Windows will detect these changes and you can commit them to your local repository:

All that’s left to do after a commit is clicking the Publish button. GitHub for Windows will now copy all changesets to the Windows Azure Websites GitHub repository which will in turn trigger an eventual build process for your web site. The result? A happy Windows Azure Websites dashboard and a site up and running. Rinse, repeat, commit. Happy deployments to Windows Azure Websites using GitHub for Windows!