Maarten Balliauw {blog}

ASP.NET MVC, Microsoft Azure, PHP, web development ...

NAVIGATION - SEARCH

Geographically distributing Windows Azure applications using Traffic Manager

With the downtime of Amazon EC2 this week, it seems a lot of websites “in the cloud” are down at the moment. Most comments I read on Twitter (and that I also made, jokingly :-)) are in the lines of “outrageous!” and “don’t go cloud!”. While I understand these comments, I think they are wrong. These “clouds” can fail. They are even designed to fail, and often provide components and services that allow you to cope with these failures. You just have to expect failure at some point in time and build it into your application.

Let me rephrase my introduction. I just told you to expect failure, but I actually believe that clouds don’t “fail”. Yes, you may think I’m lunatic there, providing you with two different and opposing views in 2 paragraphs. Allow me to explain: "a “failing” cloud is actually a “scaling” cloud, only thing is: it’s scaling down to zero. If you design your application so that it can scale out, you should also plan for scaling “in”, eventually to zero. Use different availability zones on Amazon, and if you’re a Windows Azure user: try the new Traffic Manager CTP!

The Windows Azure Traffic Manager provides several methods of distributing internet traffic among two or more hosted services, all accessible with the same URL, in one or more Windows Azure datacenters. It’s basically a distributed DNS service that knows which Windows Azure Services are sitting behind the traffic manager URL and distributes requests based on three possible profiles:

  • Failover: all traffic is mapped to one Windows Azure service, unless it fails. It then directs all traffic to the failover Windows Azure service.
  • Performance: all traffic is mapped to the Windows Azure service “closest” (in routing terms) to the client requesting it. This will direct users from the US to one of the US datacenters, European users will probably end up in one of the European datacenters and Asian users, well, somewhere in the Asian datacenters.
  • Round-robin: Just distribute requests between various Windows Azure services defined in the Traffic Manager policy

As a sample, I have uploaded my Windows Azure package to two datacenters: EU North and US North Central. Both have their own URL:

I have created a “performance” policy at the URL http://certgen.ctp.trafficmgr.com/, which redirects users to the nearest datacenter (and fails-over if one goes down):

Windows Azure Traffic Manager geo replicate

If one of the datacenters goes down, the other service will take over. And as a bonus, I get reduced latency because users use their nearest datacenter.

So what’s this have to do with my initial thoughts? Well: design to scale, using an appropriate technique to your specific situation. Use all the tools the platform has to offer, and prepare for scaling out and for scaling '”in”, even to zero instances. And as with backups: test your disaster strategy now and then.

PS: Artwork based on Josh Twist’s sketches

Windows Azure SDK for PHP v3.0.0 BETA released

imageMicrosoft and RealDolmen are very proud to announce the availability of the Windows Azure SDK for PHP v3.0.0 BETA on CodePlex. This releases is something we’ve been working on in the past few weeks, implementing a lot of new features that enable you to fully leverage the Windows Azure platform from PHP.

This release is BETA software, which means it is feature complete. However, since we have one breaking change, we’re releasing a BETA first to ensure every edge case is covered. Of you are using the current version of the Windows Azure SDK for PHP, feel free to upgrade and let us know your comments.

A comment we received a lot for previous versions was the fact that for table storage, datetime values were returned as strings and parsing of them was something you as a developer should do. In this release, we’ve broken that: table storage entities now return native PHP DateTime objects instead of strings for Edm.DateTime properties.

The feature we’re most proud of is the support for the management API: you can now instruct WIndows Azure from PHP, where you would normally do this through the web portal. This means that you can fully automate your Windows Azure deployment, scaling, … from a PHP script. I even have sample of this, check my blog post “Windows Azure and scaling: how? (PHP)”.

Another nice feature is the new logging infrastructure: if you are used to working with loggers and appenders (like for example in Zend Framework), this should be familiar. It is used to provide logging capabilities in a mayor production site, www.hotelpeeps.com (yes, that is PHP on Windows Azure you’re seeing there!). Thanks, Lucian, for contributing this!

Last but not least: the session handler has been updated. It relied on table storage for storing session data, however large session objects were not supported since table storage has a maximum amount of data per record. If you are creating large session objects (which I do not recommend, as a best practice), feel free to pass a blob storage client to the session handler instead to have sessions stored in blob storage.

To close this post, here’s the official changelog:

  • Breaking change: Table storage entities now return DateTime objects instead of strings for Edm.DateTime properties
  • New feature: Service Management API in the form of Microsoft_WindowsAzure_Management_Client
  • New feature: logging infrastructure on top of table storage
  • Session provider now works on table storage for small sessions, larger sessions can be persisted to blob storage
  • Queue storage client: new hasMessages() method
  • Introduction of an autoloader class, increasing speed for class resolving
  • Several minor bugfixes and performance tweaks

Get it while it’s hot: http://phpazure.codeplex.com/releases/view/64047

Do you prefer PEAR? Well... pear channel-discover pear.pearplex.net & pear install pearplex/PHPAzure should do the trick. Make sure you allow BETA stability packages in order to get the fresh bits.

PS: We’re running a PHP on Windows Azure contest in Belgium and surrounding countries. The contest is closed for registration, but there’s good value in the blog posts coming out of it. Check www.phpazurecontest.com for more details.

Slides for my talk at MIX11: Fun with ASP.NET MVC 3, MEF and NuGet

As promised, here are the slides and demo code for my talk "Fun with ASP.NET MVC 3, MEF and NuGet" I presented at MIX in Las Vegas.

Abstract: "So you have a team of developers… And a nice architecture to build on… How about making that architecture easy for everyone and getting developers up to speed quickly? Learn all about integrating the managed extensibility framework (MEF) and ASP.NET MVC with some NuGet sauce for creating loosely coupled, easy to use architectures that anyone can grasp."

The recorded session: (on Channel 9)

 

The slide deck:

The demo code: 2011-04-14 Fun with ASP.NET MVC 3 MEF.zip (6.76 mb)

Enjoy! And thanks for joining!

kick it on DotNetKicks.com

Official Belgium TechDays 2011 Windows Phone 7 app released

I’m proud to announce that we (RealDolmen) have released the official Belgium TechDays 2011 Windows Phone 7 app! The official Belgium TechDays 2011 gives you the ability to browse current & upcoming sessions, as well as provide LIVE feedback to the event organizers. Is the current session awesome? Let us know! Is the food too spicy? Let us know!

Why am I blogging this? Well: one of the first sessions at the event will be Silverlight, Windows Phone 7, Windows Azure, jQuery, OData and RIA Services. Shaken, not stirred, deliverd by Kevin Dockx and myself. It will feature this WIndows Phone 7 application as well as the backoffice for it (Silverlight), the mobile web front-end (jQuery mobile), the web front-end (MVC), the integration points with the event organizers and the deployment on Windows Azure. Not to mention the twitterwall that integrates with this. ANd the top sessions ranking that will be displayed based on input from all the channels I mentioned before. In short: I’m blogging this to plug our session :-)

Interested in what we’ve built? Or just a consumer of WP7 apps? Download the app at http://techdays.realdolmen.com or directly by clicking the picture below:

Download the official Techdays 2011 application for WIndows Phone 7

See you at TechDays!

Lightweight PHP application deployment to Windows Azure

Those of you who are deploying PHP applications to Windows Azure, are probably using the Windows Azure tooling for Eclipse or the fantastic command-line tools available. I will give you a third option that allows for a 100% customized setup and is much more lightweight than the above options. Of course, if you want to have the out-of-the box functionality of those tools, stick with them.

Note: while this post is targeted at PHP developers, it also shows you how to build your own .cspkg from scratch for any other language out there. That includes you, .NET and Ruby!

Oh, my syntax highlighter is broken so you won't see any fancy colours down this post :-)

Phase 1: Creating a baseline package template

Every Windows Azure package is basically an OpenXML package containing your application. For those who don’t like fancy lingo: it’s a special ZIP file. Fact is that it contains an exact copy of a folder structure you can create yourself. All it takes is creating the following folder & file structure:

  • ServiceDefinition.csdef
  • ServiceConfiguration.cscfg
  • PhpOnAzure.Web
    • bin
    • resources
    • Web.config

I’ll go through each of those. First off, the ServiceDefinition.csdef file is the metadata describing your Windows Azure deployment. It (can) contain the following XML:

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="PhpOnAzure" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
  <WebRole name="PhpOnAzure.Web" enableNativeCodeExecution="true">
    <Sites>
      <Site name="Web" physicalDirectory="./PhpOnAzure.Web">
        <Bindings>
          <Binding name="Endpoint1" endpointName="HttpEndpoint" />
        </Bindings>
      </Site>
    </Sites>
    <Startup>
      <Task commandLine="add-environment-variables.cmd" executionContext="elevated" taskType="simple" />
      <Task commandLine="install-php.cmd" executionContext="elevated" taskType="simple" />
    </Startup>
    <Endpoints>
      <InputEndpoint name="HttpEndpoint" protocol="http" port="80" />
    </Endpoints>
    <Imports>
      <Import moduleName="Diagnostics"/>
    </Imports>
    <ConfigurationSettings>
    </ConfigurationSettings>
  </WebRole>
</ServiceDefinition>

Basically, it tells Windows Azure to create a WebRole named “PhpOnAzure.Web” (notice the not-so-coincidental match with one directory of the folder structure described earlier). It will contain one site that listens on a HttpEndpoint (port 80). Next, I added 2 startup tasks, add-environment-variables.cmd and install-php.cmd. More on these later on.

Next, ServiceConfiguration.cscfg is the actual configuration file for your Windows Azure deployment. It looks like this:

<?xml version="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="PhpOnAzure" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="2" osVersion="*">
  <Role name="PhpOnAzure.Web">
    <Instances count="1" />
    <ConfigurationSettings>
      <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="<your diagnostics connection string here>"/>
    </ConfigurationSettings>
  </Role>
</ServiceConfiguration>

Just like in a tooling-based WIndows Azure deployment, it allows you to set configuratio ndetails like the connection string where the diagnostics monitor should write all logging to.

The PhpOnAzure.Web folder is the actual root where my web application will live. It’s the wwwroot of your app, the htdocs folder of your app. Don’t put any contents n here yet, as we’ll automate that later in this post. Anyways, it (optionally) contains a Web.config file where I specify that index.php should be the default document:

<?xml version="1.0"?>
<configuration>
  <system.webServer>
    <defaultDocument>
      <files>
        <clear />
        <add value="index.php" />
      </files>
    </defaultDocument>
  </system.webServer>
</configuration>

Everything still OK? Good! (I won’t take no for an answer :-)). Add a bin folder in there as well as a resources folder. The bin folder will hold our startup tasks (see below), the resources folder will contain a copy of the Web Platform Installer command-line tools.

That’s it! A Windows Azure deployment package is actually pretty simple and easy to create yourself.

Phase 2: Auto-installing the PHP runtime

I must admit: this one’s stolen from the excellent Canadian Windows Azure MVP Cory Fowler aka SyntaxC4. He blogged about using startup tasks and the WebPI Command-line tool to auto-install PHP when your Windows Azure VM boots. Read his post for in-depth details, I’ll just cover the startup task doing this. Which I shamelessly copied from his blog as well. Credits due.

Under PhpOnAzure.Web\bin, add a script named install-php.cmd and copy in the following code:

@echo off
ECHO "Starting PHP installation..." >> ..\startup-tasks-log.txt

md "%~dp0appdata"
cd "%~dp0appdata"
cd ..

reg add "hku\.default\software\microsoft\windows\currentversion\explorer\user shell folders" /v "Local AppData" /t REG_EXPAND_SZ /d "%~dp0appdata" /f
"..\resources\WebPICmdLine\webpicmdline" /Products:PHP53 /AcceptEula  >> ..\startup-tasks-log.txt 2>>..\startup-tasks-error-log.txt
reg add "hku\.default\software\microsoft\windows\currentversion\explorer\user shell folders" /v "Local AppData" /t REG_EXPAND_SZ /d %%USERPROFILE%%\AppData\Local /f

ECHO "Completed PHP installation." >> ..\startup-tasks-log.txt

What it does is:

  • Create a local application data folder
  • Add that folder name to the registry
  • Call “webpicmdline” and install PHP 5.3.x. And of course, /AcceptEula will ensure you don’t have to go to a Windows Azure datacenter, break into a container and click “I accept” on the screen of your VM.
  • Awesomeness happens: PHP 5.3.x is installed!
  • And everything gets logged into the startup-tasks-error-log.txt file in the root of your website. It allows you to inspect the output of all these commands once your VM has booted.

Phase 3: Fixing a problem

So far only sunshine. But… Since the technique used here is creating a full-IIS web role (a good thing), there’s a small problem there… Usually, your web role will spin up IIS hosted core and run in the same process that launched your VM in the first place. In a regular web role, the hosting process contains some interesting environment variables about your deployment: the deployment ID and the role name and even the instance name!

With full IIS, your web role is running inside IIS. The real IIS, that’s right.  And that’s a different process from the one that launched your VM, which means that these useful environment variables are unavailable to your application. No problem for a lot of applications, but if you’re using the PHP-based diagnostics manager from the Windows Azure SDK for PHP (or other code that relies on these environment variables, well, you’re sc…. eh, in deep trouble.

Luckily, startup tasks have access to the Windows Azure assemblies that can also give you this information. So why not create a task that copies this info into a machine environment variable?

We’ll need two scripts: one .cmd file launching PowerShel, and of course PowerShell. Let’s start with a file named add-environment-variables.cmd under PhpOnAzure.Web\bin:

@echo off
ECHO "Adding extra environment variables..." >> ..\startup-tasks-log.txt

powershell.exe Set-ExecutionPolicy Unrestricted
powershell.exe .\add-environment-variables.ps1 >> ..\startup-tasks-log.txt 2>>..\startup-tasks-error-log.txt

ECHO "Added extra environment variables." >> ..\startup-tasks-log.txt

Nothing fancy, just as promised we’re launching PowerShell. But to ensure that we have al possible options in PowerShell, the execution policy is first set to Unrestricted. Next, add-environment-variables.ps1 is launched:

[Reflection.Assembly]::LoadWithPartialName("Microsoft.WindowsAzure.ServiceRuntime")

$rdRoleId = [Environment]::GetEnvironmentVariable("RdRoleId", "Machine")

[Environment]::SetEnvironmentVariable("RdRoleId", [Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment]::CurrentRoleInstance.Id, "Machine")
[Environment]::SetEnvironmentVariable("RoleName", [Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment]::CurrentRoleInstance.Role.Name, "Machine")
[Environment]::SetEnvironmentVariable("RoleInstanceID", [Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment]::CurrentRoleInstance.Id, "Machine")
[Environment]::SetEnvironmentVariable("RoleDeploymentID", [Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment]::DeploymentId, "Machine")


if ($rdRoleId -ne [Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment]::CurrentRoleInstance.Id) {
    Restart-Computer
}

[Environment]::SetEnvironmentVariable('Path', $env:RoleRoot + '\base\x86;' + [Environment]::GetEnvironmentVariable('Path', 'Machine'), 'Machine')

Wow! A lot of code? Yes. First of all, we’re loading the Microsoft.WindowsAzure.ServiceRuntime assembly. Next, we query the current environment variables for a variable named RdRoleId and copy it in a variable named $rdRoleId. Next, we set all environment variables (RdRoleId, RoleName, RoleInstanceID, RoleDeploymentID) to their actual values. Just like that. Isn’t PowerShell a cool thing?

After all this, the $rdRoleId variable is compared with the current RdRoleId environment variable. Are they the same? Good! Are they different? Reboot the instance. Rebooting the instance is the only easiest way for IIS and PHP to pick these new values up.

Phase 4: Automating packaging

One thing left to do: we do have a folder structure now, but I don’t see any .cspkg file around for deployment…  Let’s fix that. by creating a good old batch file that does the packaging for us. Note that this is *not* a necessary part, but it will ease your life. Here’s the script:

@echo off

IF "%1"=="" GOTO ParamMissing

echo Copying default package components to temporary location...
mkdir deploy-temp
xcopy /s /e /h deploy deploy-temp

echo Copying %1 to temporary location...
xcopy /s /e /h %1 deploy-temp\PhpOnAzure.Web

echo Packaging application...
"c:\Program Files\Windows Azure SDK\v1.4\bin\cspack.exe" deploy-temp\ServiceDefinition.csdef /role:PhpOnAzure.Web;deploy-temp\PhpOnAzure.Web /out:PhpOnAzure.cspkg
copy deploy-temp\ServiceConfiguration.cscfg

echo Cleaning up...
rmdir /S /Q deploy-temp

GOTO End

:ParamMissing
echo Parameter missing: please specify the path to the application to deploy.

:End

You can invoke it from a command line:

package c:\patch-to-my\app

This will copy your application to a temporary location, merge in the template we created in the previous steps and create a .cspkg file by calling the cspack.exe from the Windows Azure SDK, and a ServiceConfiguration.cscfg file containing your configuration.

Phase 5: Package hello world!

Let’s create an application that needs massive scale. Here’s the source code for the index.php file which will handle all requests. Put it in your c:\temp or wherever you want.

<?php
echo “Hello, World!”;

Next, call the package.bat created previously:

package c:\patch-to-my\app

There you go: PhpOnAzure.cspkg and ServiceConfiguraton.cscfg at your service. Upload, deploy and enjoy. Once the VM is booted in Windows Azure, all environment variables will be set and PHP will be automatically installed. Feel free to play around with the template I created (lightweight-php-deployment-azure.zip (854.44 kb)), as you can also install, for example, the Wincache extension or SQL Server Driver for PHP from the WebPI command-line tools. Or include your own PHP distro. Or achieve world domination by setting the instance count to a very high number (of course, this requires you to call Microsoft if you want to go beyond 20 instances, just to see if you’re worthy for world domination).

Conclusion

Next to the officially supported packaging tools, there’s also the good old craftsmen’s hand-made deployment. And if you automate some parts, it’s extremely easy to package your application in a very lightweight fashion. Enjoy!

Here’s the download: lightweight-php-deployment-azure.zip (854.44 kb)