Maarten Balliauw {blog}

Web development, NuGet, Microsoft Azure, PHP, ...

NAVIGATION - SEARCH

ASP.NET MVC 3 preview 1 is out! Quick review...

I just noticed a very interesting download: ASP.NET MVC 3 preview 1. Yes, you are reading this correctly, the first bits for v3.0 are there! Let’s have a quick look around and see what’s new...

kick it on DotNetKicks.com

Razor Syntax View Engine

ScottGu blogged about Razor before. ASP.NET MVC has always supported the concept of “view engines”, pluggable modules that allow you to have your views rendered by different engines like for example the WebForms engine, Spark, NHAML, …

Razor is a new view engine, focused on less code clutter and shorter code-expressions for generating HTML dynamically. As an example, have a look at the following view:

[code:c#]

<ul>
  <% foreach (var c in Model.Customers) { %>
    <li><%:c.DisplayName%></li>
  <% } %>
</ul>

[/code]

In Razor syntax, this becomes:

[code:c#]

<ul>
  @foreach (var c in Model.Customers) {
    <li>@c.DisplayName</li>
  }
</ul>

[/code]

Perhaps not the best example to show the strengths of this new engine, but do bear in mind that Razor simply puts code literally in your HTML, making it develop faster (did I mention perfect IntelliSense support in the Razor view editor?).

Also, there’s a nice addition to the “Add View” dialog in Visual Studio: you can now choose for which view engine you want to generate a view.

Razor view engine - Add view dialog

ViewData dictionary “dynamic” support

.NET 4 introduced the “dynamic” keyword, which is abstracting away a lot of reflection code you’d normally have to write yourself. The fun thing is, that the MVC guys abused this thing in a very nice way.

Controller action method, ASP.NET MVC 2:

[code:c#]

public ActionResult Index()
{
    ViewModel["Message"] = "Welcome to ASP.NET MVC!";

    return View();
}

[/code]

Controller action method, ASP.NET MVC 3:

[code:c#]

public ActionResult Index()
{
    ViewModel.Message = "Welcome to ASP.NET MVC!";

    return View();
}

[/code]

“Isn’t that the same?” – Yes, in essence it is exactly the same concept at work. However, by using the dynamic keyword, there’s less “string pollution” in my code. Do note that in most situations, you would create a custom “View Model” and pass that to the view instead of using this ugly dictionary or dynamic object. Nevertheless: I do prefer reading code that uses less dictionaries.

So far for the controller side, there’s also the view side. Have a look at this:

[code:c#]

@inherits System.Web.Mvc.WebViewPage

@{
    View.Title = "Home Page";
    LayoutPage = "~/Views/Shared/_Layout.cshtml";
}

<h2>@View.Message</h2>
<p>
    To learn more about ASP.NET MVC visit <a href="http://asp.net/mvc" title="ASP.NET MVC Website">http://asp.net/mvc</a>.
</p>

[/code]

A lot of new stuff there, right? First of all the Razor syntax, but secondly… There’s just something like @View.Message in this view, and this is rendering something from the ViewData dictionary/dynamic object. Again: very readable and understandable.

It’s a small change on the surface, but I do like it. In my opinion, it’s more readable than using the ViewData dictionary when you are not using a custom view model.

Global action filters

Imagine you have a team of developers, all writing controllers. Imagine that they have to add the [HandleError] action filter to every controller, and they sometimes tend to forget… That’s where global action filters come to the rescue! Add this line to Global.asax:

[code:c#]

GlobalFilters.Filters.Add(new HandleErrorAttribute());

[/code]

This will automatically register that action filter attribute for every controller and action method.

Fun fact: I blogged about exactly this feature about a year ago: Application-wide action filters in ASP.NET MVC. Want it in ASP.NET MVC 2? Go and get it :-)

Cool, eh? Well… I do have mixed feelings about this… I can imagine there are situations where you want to do this more selectively. Here’s my call-out to the ASP.NET MVC team:

  • At least allow to specify action filters on a per-area level, so I can have my “Administration” area have other filters than my default area.
  • In an ideal world, I’d prefer something where I can specify global action filters even more granularly. This can be done using some customizing, but it would be useful to have it out-of-the-box. Here's an example of the ideal world:

[code:c#]

GlobalFilters.Filters.AddTo<HomeController>(new HandleErrorAttribute()) 
                     .AddTo<AccountController>(c => c.ChangePassword(), new AuthorizeAttribute());

[/code]

Dependency injection support

I’m going to be short on this one: there’s 4 new hooks for injecting dependencies:

  • When creating controller factories
  • When creating controllers
  • When creating views (might be interesting!)
  • When using action filters

More on that in my next post on ASP.NET MVC 3, as I think it deserves a full post rather than jut some smaller paragraphs.

Update: Here's that next post on ASP.NET MVC 3 and dependency injection / MEF

New action result types

Not very ground-skaing news, but there's a new set of ActionResult variants available which will make your life easier:

  • HttpNotFoundResult - Speaks for itself, right :-)
  • HttpStatusCodeResult - How about new HttpStatusCodeResult(418, "I'm a teapot");
  • RedirectPermanent, RedirectToRoutePermanent, RedirectToActionPermanent - Writes a permanent redirect header

Conclusion

I only touched the tip of the iceberg. There’s more to ASP.NET MVC 3 preview 1, described in the release notes.

In short, I’m very positive about the amount of progress being made in this framework! Very pleased with the DI portion of it, on which I’ll do a blog post later.

Update: Here's that next post on ASP.NET MVC 3 and dependency injection / MEF

Renewed MVP ASP.NET for 2010!

Just got the best e-mail a Microsoft community member can receive in his mailbox:MVPLogo

Dear Maarten Balliauw,

Congratulations! We are pleased to present you with the 2010 Microsoft® MVP Award! This award is given to exceptional technical community leaders who actively share their high quality, real world expertise with others. We appreciate your outstanding contributions in ASP/ASP.NET technical communities during the past year.

(...)

Toby Richards
General Manager
Community Support Services

I wish to thank everyone who has been supporting me, encouraging me, challenging me and thus bringing me to a second year of MVP duty. I will try to achieve the same for next year: do a lot of sessions, work on open-source, do blog posts, …

With that: if you are not yet at lest challenging me, please do start doing this. It only helps me to learn from your problems which will in turn help the whole community to learn from it.

I can almost copy my blog post from last year on the next part: I will leave the community for the next weeks to enjoy a nice vacation in Austria. No, not traveling for work this time. My next two weeks community will be a community of mountains, sun, große Bier and rest (not REST!). See you!

ASP.NET MVC - MvcSiteMapProvider 2.0 is out!

I’m very proud to announce the release of the ASP.NET MVC MvcSiteMapProvider 2.0! I’m also proud that the name of this product now exceeds the average length of Microsoft product names. In this blog post, I will give you a feel of what you can (and can not) do with this ASP.NET-specific SiteMapProvider.

As a warning: if you’ve used version 1 of this library, you will notice that I have not thought of backwards compatibility. A lot of principles have also changed. For good reasons though: this release is a rewrite of the original version with improved features, extensibility and stability.

The example code is all based on the excellent ASP.NET MVC Music Store sample application by Jon Galloway.

Getting the bits

As always, the bits are available on CodePlex: MvcSiteMapProvider 2.0.0
If you prefer to have the full source code, download the example application or check the source code tab on CodePlex.

Introduction

MvcSiteMapProvider is, as the name implies, an ASP.NET MVC SiteMapProvider implementation for the ASP.NET MVC framework. Targeted at ASP.NET MVC 2, it provides sitemap XML functionality and interoperability with the classic ASP.NET sitemap controls, like the SiteMapPath control for rendering breadcrumbs and the Menu control.

Based on areas, controller and action method names rather than hardcoded URL references, sitemap nodes are completely dynamic based on the routing engine used in an application. The dynamic character of ASP.NET MVC is followed in the MvcSiteMapProvider: there are numerous extensibility points that allow you to extend the basic functionality offered.

Registering the provider

After downloading the MvcSiteMapProvider, you will have to add a reference to the assembly in your project. Also, you will have to register the provider in your Web.config file. Add the following code somewhere in the <system.web> section:

[code:c#]

<siteMap defaultProvider="MvcSiteMapProvider" enabled="true">
  <providers>
    <clear />
    <add name="MvcSiteMapProvider"
         type="MvcSiteMapProvider.DefaultSiteMapProvider, MvcSiteMapProvider"
         siteMapFile="~/Mvc.Sitemap"
         securityTrimmingEnabled="true"
         enableLocalization="true"
         scanAssembliesForSiteMapNodes="true"
         skipAssemblyScanOn=""
         attributesToIgnore="bling"
         nodeKeyGenerator="MvcSiteMapProvider.DefaultNodeKeyGenerator, MvcSiteMapProvider"
         controllerTypeResolver="MvcSiteMapProvider.DefaultControllerTypeResolver, MvcSiteMapProvider"
         actionMethodParameterResolver="MvcSiteMapProvider.DefaultActionMethodParameterResolver, MvcSiteMapProvider"
         aclModule="MvcSiteMapProvider.DefaultAclModule, MvcSiteMapProvider"
         />
  </providers>
</siteMap>

[/code]

The following configuration directives can be specified:

Directive Required? Default Description
siteMapFile No ~/Web.sitemap The sitemap XML file to use.
securityTrimmingEnabled No false Use security trimming? When enabled, nodes that the user can not access will not be displayed in any sitemap control.
enableLocalization No false Enables localization of sitemap nodes.
scanAssembliesForSiteMapNodes No false Scan assemblies for sitemap nodes defined in code?
skipAssemblyScanOn No (empty) Comma-separated list of assemblies that should be skipped when scanAssembliesForSiteMapNodes is enabled.
attributesToIgnore No (empty) Comma-separated list of attributes defined on a sitemap node that should be ignored by the MvcSiteMapProvider.
nodeKeyGenerator No MvcSiteMapProvider.DefaultNodeKeyGenerator, MvcSiteMapProvider Class that will be used to generate sitemap node keys.
controllerTypeResolver No MvcSiteMapProvider.DefaultControllerTypeResolver, MvcSiteMapProvider Class that will be used to resolve the controller for a specific sitemap node.
actionMethodParameterResolver No MvcSiteMapProvider.DefaultActionMethodParameterResolver, MvcSiteMapProvider Class that will be used to determine the list of parameters on a sitemap node.
aclModule No MvcSiteMapProvider.DefaultAclModule, MvcSiteMapProvider Class that will be used to verify security and access rules for sitemap nodes.

 

Creating a first sitemap

The following is a simple sitemap XML file that can be used with the MvcSiteMapProvider:

[code:c#]

<?xml version="1.0" encoding="utf-8" ?>
<mvcSiteMap xmlns="http://mvcsitemap.codeplex.com/schemas/MvcSiteMap-File-2.0" enableLocalization="true">
  <mvcSiteMapNode title="Home" controller="Home" action="Index" changeFrequency="Always" updatePriority="Normal">
    <mvcSiteMapNode title="Browse Store" controller="Store" action="Index" />
    <mvcSiteMapNode title="Checkout" controller="Checkout" />
  </mvcSiteMapNode>
</mvcSiteMap>

[/code]

The following attributes can be given on an XML node element:

Attribute Required? Default Description
title Yes (empty) The title of the node.
description No (empty) Description of the node.
area No (empty) The MVC area for the sitemap node. If not specified, it will be inherited from a node higher in the hierarchy.
controller Yes (empty) The MVC controller for the sitemap node. If not specified, it will be inherited from a node higher in the hierarchy.
action Yes (empty) The MVC action method for the sitemap node. If not specified, it will be inherited from a node higher in the hierarchy.
key No (autogenerated) The unique identifier for the node.
url No (autogenerated based on routes) The URL represented by the node.
roles No (empty) Comma-separated list of roles allowed to access the node and its child nodes.
resourceKey No (empty) Optional resource key.
clickable No True Is the node clickable or just a grouping node?
targetFrame No (empty) Optional target frame for the node link.
imageUrl No (empty) Optional image to be shown by supported HtmlHelpers.
lastModifiedDate No (empty) Last modified date for the node.
changeFrequency No Undefined Change frequency for the node.
updatePriority No Undefined Update priority for the node.
dynamicNodeProvider No (empty) A class name implementing MvcSiteMapProvider.Extensibility.IDynamicNodeProvider and providing dynamic nodes for the site map.

 

Defining sitemap nodes in code

In some cases, defining a sitemap node in code is more convenient than defining it in a sitemap xml file. To do this, decorate an action method with the MvcSiteMapNodeAttribute attribute. For example:

[code:c#]

// GET: /Checkout/Complete
[MvcSiteMapNodeAttribute(Title = "Checkout complete", ParentKey = "Checkout")]
public ActionResult Complete(int id)
{
    // ...
}

[/code]

Note that the ParentKey property should be specified to ensure the MvcSiteMapProvider  can determine the hierarchy for all nodes.

Dynamic sitemaps

In many web applications, sitemap nodes are directly related to content in a persistent store like a database.For example, in an e-commerce application, a list of product details pages in the sitemap maps directly to the list of products in the database. Using dynamic sitemaps, a small class can be provided to the MvcSiteMapProvider offering a list of dynamic nodes that should be incldued in the sitemap. This ensures the product pages do not have to be specified by hand in the sitemap XML.

First of all, a sitemap node should be defined in XML. This node will serve as a template and tell the MvcSiteMapProvider infrastructure to use a custom dynamic node procider:

[code:c#]

<mvcSiteMapNode title="Details" action="Details" dynamicNodeProvider="MvcMusicStore.Code.StoreDetailsDynamicNodeProvider, MvcMusicStore" />

[/code]

Next, a class implementing MvcSiteMapProvider.Extensibility.IDynamicNodeProvider or extending MvcSiteMapProvider.Extensibility.DynamicNodeProviderBase should be created in your application code. Here’s an example:

[code:c#]

public class StoreDetailsDynamicNodeProvider
    : DynamicNodeProviderBase
{
    MusicStoreEntities storeDB = new MusicStoreEntities();

    public override IEnumerable<DynamicNode> GetDynamicNodeCollection()
    {
        // Build value
        var returnValue = new List<DynamicNode>();

        // Create a node for each album
        foreach (var album in storeDB.Albums.Include("Genre"))
        {
            DynamicNode node = new DynamicNode();
            node.Title = album.Title;
            node.ParentKey = "Genre_" + album.Genre.Name;
            node.RouteValues.Add("id", album.AlbumId);

            returnValue.Add(node);
        }

        // Return
        return returnValue;
    }
}

[/code]

Cache dependency

When providing dynamic sitemap nodes to the MvcSiteMapProvider, chances are that the hierarchy of nodes will become stale, for example when adding products in an e-commerce website. This can be solved by specifying a CacheDescriptor on your MvcSiteMapProvider.Extensibility.IDynamicNodeProvider implementation:

[code:c#]

public class StoreDetailsDynamicNodeProvider
    : DynamicNodeProviderBase
{
    MusicStoreEntities storeDB = new MusicStoreEntities();

    public override IEnumerable<DynamicNode> GetDynamicNodeCollection()
    {
        // ...
    }

    public override CacheDescription GetCacheDescription()
    {
        return new CacheDescription("StoreDetailsDynamicNodeProvider")
        {
            SlidingExpiration = TimeSpan.FromMinutes(1)
        };
    }
}

[/code]

HtmlHelper functions

The MvcSiteMapProvider provides different HtmlHelper extension methods which you can use to generate SiteMap-specific HTML code on your ASP.NET MVC views. Here's a list of available HtmlHelper extension methods.

  • Html.MvcSiteMap().Menu() - Can be used to generate a menu
  • Html.MvcSiteMap().SubMenu() - Can be used to generate a submenu
  • Html.MvcSiteMap().SiteMap() - Can be used to generate a list of all pages in your sitemap
  • Html.MvcSiteMap().SiteMapPath() - Can be used to generate a so-called "breadcrumb trail"
  • Html.MvcSiteMap().SiteMapTitle() - Can be used to render the current SiteMap node's title

Note that these should be registered in Web.config, i.e. under <pages> add the following:

[code:c#]

<pages>
    <controls>
        <! -- ... -->
    </controls>
    <namespaces>
        <! -- ... -->
        <add namespace="MvcSiteMapProvider.Web.Html" />
    </namespaces>
</pages>

[/code]

Action Filter Attributes

SiteMapTitle

In some situations, you may want to dynamically change the SiteMap.CurrentNode.Title in an action method. This can be done manually by setting SiteMap.CurrentNode.Title, or by adding the SiteMapTitle action filter attribute.

Imagine you are building a blog and want to use the Blog's Headline property as the site map node title. You can use the following snippet:

[code:c#]

[SiteMapTitle("Headline")]
public ViewResult Show(int blogId) {
   var blog = _repository.Find(blogIdId);
   return blog;
}

[/code]

You can also use a non-strong typed ViewData value as the site map node title:

[code:c#]

[SiteMapTitle("SomeKey")]
public ViewResult Show(int blogId) {
   ViewData["SomeKey"] = "This will be the title";

   var blog = _repository.Find(blogIdId);
   return blog;
}

[/code]

Exporting the sitemap for search engine indexing

When building a website, chances are that you want to provide an XML sitemap used for search engine indexing. The XmlSiteMapResult class creates an XML sitemap that can be submitted to Google, Yahoo and other search engines to help them crawl your website better. The usage is very straightforward:

[code:c#]

public class HomeController
{
    public ActionResult SiteMapXml()
    {
        return new XmlSiteMapResult();
    }
}

[/code]

Optionally, a starting node can also be specified in the constructor of theXmlSiteMapResult .

Conclusion

Get it while it’s hot! MvcSiteMapProvider 2.0.0 is available on CodePlex.

kick it on DotNetKicks.com

Running on Windows Azure - ChronoRace - Autoscaling

image At RealDolmen, we had the luck of doing the first (known) project on Windows Azure in Belgium. Together with Microsoft, we had the opportunity to make the ChronoRace website robust enough to withstand large sports events like the 20km through Brussels.

ChronoRace is a Belgian company based in Malmédy, specialised in electronic timing for large sports events (around 340 per year) troughout Europe in different disciplines like jogging, cycling, sailing, … Each participant is registered through the website, can consult their results and can view a high-definition video of their arrival at the finish line. Of course, these results and videos are also used to brag to co-workers and family members, resulting in about 10 result and video views per participant. Imagine 20.000 or more participants on large events… No wonder their current 1 webserver – 1 database server setup could not handle all load.

Extra investments in hardware, WAN connection upgrades and video streaming were considered, however found too expensive to have these running for 365 days a year while on average this extra capacity would only be needed for 14 days a year. Ideal for cloud computing! Especially with an expected 30.000 participants for the 20km through Brussels... (which would mean 3 TB of data transfers for the videos on 1 day!!!)

Microsoft selected RealDolmen as a partner for this project, mainly because of the knowledge we built over the past year, since the first Azure CTP. Together with ChronoRace, we gradually moved the existing SQL Server databse to SQL Azure. After that, we started moving the video streaming to blob storage, implemented extra caching and automatic scaling.

You probably have seen the following slides in various presentations on cloud computing:

Capacity cloud computing

All marketing gibberish, right? Nope! We actually managed to get very close to this model using our custom autoscaling mechanism. Here are some figures we collected during the peak of the 20km through Brussels:

Windows Azure Auto Scaling

More information on the technical challenges we encountered can be found in my slide deck I presented at KAHOSL last week:

If you want more information on scalability and automatic scaling, feel free to come to the Belgian Community Day 2010 where I will be presenting a session on this topic.

Oh and for the record: I’m not planning on writing marketing posts. I just was so impressed by our actual autoscaling graph that I had to blog this :-)

Taking Care of a Cloud Environment (slides)

It looks like I’m only doing sessions lately :-) Here’s another slide deck for a presentation I did on the Architect Forum last week in Belgium.

Abstract: “No, this session is not about greener IT. Learn about using the RoleEnvironment and diagnostics provided by Windows Azure. Communication between roles, logging and automatic upscaling of your application are just some of the possibilities of what you can do if you know about how the Windows Azure environment works.”

Thanks for attending!

Slides of our VISUG session

As promised, here are the slides of the VISUG session me and Kris van der Mast did yesterday.

Being a pimp without Silverlight!

Abstract: “Don't tell us you're jealous of those Silverlight fanboys! We'll show you that applications with bling can be developed using ASP.NET MVC and jQuery. We're talking MVC, template helpers, AJAX, JSON, transitions, live bindings, ...”

TechDays 2010 Portugal slides and demo code

First of all: thank you for attending the sessions Kevin Dockx and I gave at TechDays 2010 Portugal! A wonder we made it there with all the ash clouds and volcanic interference based in Iceland.

Just Another Wordpress Weblog, But More Cloudy

Abstract: “While working together with Microsoft on the Windows Azure SDK for PHP, we found that we needed an popular example application hosted on Microsoft’s Windows Azure. Wordpress was an obvious choice, but not an obvious task. Learn more about Windows Azure, the PHP SDK that we developed, SQL Azure and about the problems we faced porting an existing PHP application to Windows Azure.”

I can not disclose demo code at this time, sorry. Here’s a list of good resources to get you started though:

PHP and Silverlight

Abstract: “So you have an existing PHP application and would like to spice it up with a rich and attractive front-end. Next to Adobe Flex, you can also choose Silverlight as a solution. This session shows you around in Silverlight and shows that PHP and Silverlight can go together easily.”

Demo code: PHP and Silverlight - DevDays.zip (1.00 mb) (based on Silverlight 2, bug Kevin for a recent version :-))

Using Windows Azure Drive in PHP (or Ruby)

At the JumpIn Camp in Zürich this week, we are trying to get some of the more popular PHP applications running on Windows Azure. As you may know, Windows Azure has different storage options like blobs, tables, queues and drives. There’s the Windows Azure SDK for PHP for most of this, except for drives. Which is normal: drives are at the operating system level and have nothing to do with the REST calls that are used for the other storage types. By the way: I did a post on using Windows Azure Drive (or “XDrive”) a while ago if you want more info.

Unfortunately, .NET code is currently the only way to create and mount these virtual hard drives from Windows Azure. But luckily, IIS7 has this integrated pipeline model which Windows Azure is also using. Among other things, this means that services provided by managed modules (written in .NET) can now be applied to all requests to the server, not just ones handled by ASP.NET! In even other words: you can have some .NET code running in the same request pipeline as the FastCGI process running PHP (or Ruby). Which made me think: it should be possible to create and mount a Windows Azure Drive in a .NET HTTP module and pass the drive letter of this thing to PHP through a server variable. And here’s how...

Note: I’ll start with the implementation part first, the usage part comes after that. If you don’t care about the implementation, scroll down...

Download source code and binaries at http://phpazurecontrib.codeplex.com.

kick it on DotNetKicks.com

Building the Windows Azure Drive HTTP module

Building HTTP modules in .NET is easy! Simply reference the System.Web assembly and create a class implementing IHttpModule:

[code:c#]

public class AzureDriveModule : IHttpModule
{
    void IHttpModule.Dispose()
    {
        throw new NotImplementedException();
    }

    void IHttpModule.Init(HttpApplication context)
    {
        throw new NotImplementedException();
    }
}

[/code]

There’s our skeleton! Now for the implementation… (Note: insane amount of code coming!)

[code:c#]

public class AzureDriveModule : IHttpModule
{
    #region IHttpModule Members

    public void Init(HttpApplication context)
    {
        // Initialize config environment
        CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) =>
        {
            configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));
        });

        // Initialize local cache
        CloudDrive.InitializeCache(
            RoleEnvironment.GetLocalResource("cloudDriveCache").RootPath,
            RoleEnvironment.GetLocalResource("cloudDriveCache").MaximumSizeInMegabytes);

        // Determine drives to map
        for (int i = 0; i < 10; i++)
        {
            string driveConnectionString = null;
            try
            {
                driveConnectionString = RoleEnvironment.GetConfigurationSettingValue("CloudDrive" + i);
            }
            catch (RoleEnvironmentException) { }

            if (string.IsNullOrEmpty(driveConnectionString))
            {
                continue;
            }

            string[] driveConnection = driveConnectionString.Split(new char[] { ';' });

            // Create storage account
            CloudStorageAccount storageAccount = CloudStorageAccount.FromConfigurationSetting(driveConnection[0]);
            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

            // Mount requested drive
            blobClient.GetContainerReference(driveConnection[1]).CreateIfNotExist();

            var drive = storageAccount.CreateCloudDrive(
                blobClient
                    .GetContainerReference(driveConnection[1])
                    .GetPageBlobReference(driveConnection[2])
                    .Uri.ToString()
            );

            try
            {
                drive.Create(int.Parse(driveConnection[3]));
            }
            catch (CloudDriveException ex)
            {
                // handle exception here
                // exception is also thrown if all is well but the drive already exists
            }

            string driveLetter = drive.Mount(
                RoleEnvironment.GetLocalResource("cloudDriveCache").MaximumSizeInMegabytes, DriveMountOptions.None);

            // Add the drive letter to the environment
            Environment.SetEnvironmentVariable("CloudDrive" + i, driveLetter);
        }
    }

    public void Dispose()
    {
    }

    #endregion
}

[/code]

Configuring and using Windows Azure Drive

There are 4 steps involved in using the Windows Azure Drive HTTP module:

  1. Copy the .NET assemblies into your project
  2. Edit ServiceConfiguration.cscfg (and ServiceDefinition.csdef)
  3. Edit Web.config
  4. Use the thing!

The Windows Azure tooling for Eclipse will be used in the following example.

Copy the .NET assemblies into your project

Create a /bin folder in your web role project and copy in all .DLL files provided. Here’s a screenshot of how this looks:

.NET assemblies for XDrive in PHP on Azure

Edit ServiceConfiguration.cscfg (and ServiceDefinition.csdef)

In order to be able to mount, some modifications to ServiceConfiguration.cscfg (and ServiceDefinition.csdef) are required. The ServiceDefinition.csdef file should contain the following additional entries:

[code:c#]

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="TestCustomModules" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
  <WebRole name="WebRole" enableNativeCodeExecution="true">
    <ConfigurationSettings>
      <Setting name="CloudDriveConnectionString" />  
      <Setting name="CloudDrive0" />   
    </ConfigurationSettings>
    <InputEndpoints>
      <!-- Must use port 80 for http and port 443 for https when running in the cloud -->
      <InputEndpoint name="HttpIn" protocol="http" port="80" />
    </InputEndpoints>
    <LocalStorage name="cloudDriveCache" sizeInMB="128"/>
  </WebRole>
</ServiceDefinition>

[/code]

Things to note are the cloudDriveCache local storage entry, which is needed for caching access to the virtual drive. The configuration settings are defined for use in ServiceConfiguration.csdef:

[code:c#]

<?xml version="1.0"?>
<ServiceConfiguration serviceName="TestCustomModules" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration">
  <Role name="WebRole">
    <Instances count="1"/>
    <ConfigurationSettings>
      <Setting name="CloudDriveConnectionString" value="UseDevelopmentStorage=true" />
      <Setting name="CloudDrive0" value="CloudDriveConnectionString;drives;sampledrive.vhd;64" />
    </ConfigurationSettings>
  </Role>
</ServiceConfiguration>

[/code]

The configuration specifies that a cloud drive “CloudDrive0” (up to “CloudDrive9”) should be mounted using the storage account in “CloudDriveConnectionString”, a storage container named “drives” and a virtual hard disk file named “sampledrive.vhd”. Oh, and the drive should be 64 MB in size.

Edit Web.config

Before the HTTP module is used by IIS7 or Windows Azure, the following should be added to Web.config:

[code:c#]

<modules>
  <add name="AzureDriveModule" type="PhpAzureExtensions.AzureDriveModule, PhpAzureExtensions"/>
</modules>

[/code]

Here’s my complete Web.config:

[code:c#]

<?xml version="1.0"?>
<configuration>
  <system.webServer>
    <!-- DO NOT REMOVE: PHP FastCGI Module Handler -->
    <handlers>
      <clear />
      <add name="PHP via FastCGI"
           path="*.php"
           verb="*"
           modules="FastCgiModule"
           scriptProcessor="%RoleRoot%\approot\php\php-cgi.exe"
           resourceType="Unspecified" />
      <add name="StaticFile" path="*" verb="*" modules="StaticFileModule,DefaultDocumentModule,DirectoryListingModule" resourceType="Either" requireAccess="Read" />
    </handlers>
    <!-- Example WebRole IIS 7 Configation -->
    <defaultDocument>
      <files>
        <clear />
        <add value="index.php" />
      </files>
    </defaultDocument>

    <modules>
      <add name="AzureDriveModule" type="PhpAzureExtensions.AzureDriveModule, PhpAzureExtensions"/>
    </modules>
  </system.webServer>
</configuration>

[/code]

Use the thing!

Next thing to do is use your virtual Windows Azure Drive. The HTTP module adds an entry in the $_SERVER variable, named after the CloudDrive0-9 settings defined earlier. The following code example stores a file on a virtual Windows Azure Drive and reads it back afterwards:

[code:c#]

<?php
file_put_contents($_SERVER['CloudDrive0'] . '\sample.txt', 'Hello World!');

echo file_get_contents($_SERVER['CloudDrive0'] . '\sample.txt');

[/code]

Source code in PHPAzureContrib (CodePlex)

Since there already is a project for PHP and Azure contributions, I decided to add this module to that project. The binaries and source code can be found on http://phpazurecontrib.codeplex.com.

Other possible usages

The approach I demonstrated above may be used for other scenarios as well:

  • Modifying php.ini before PHP runs. The module you can access would run before FastCGI runs, an ideal moment to modigy php.ini settings and such.
  • Using .NET authentication modules in PHP, check this site for an example.
  • Download updates to PHP automatically if a new version is available and deploy it into your application at runtime. Probably needs some performance tuning but this trick may work. The same goes for static content and script updates by the way. Imagine pulling a website dynamically from blob storage and deploy it onto your web role without any hassle…

In short: endless possibilities!

kick it on DotNetKicks.com

Put your existing application in the cloud!

As promised during my talk, here's the slide deck for "Put your existing application in the cloud!".

Abstract: "Leverage the highly scalable Windows Azure platform and deploy your existing ASP.NET application to a new home in the clouds. This demo filled session will guide you in how to make successful use of Windows Azure’s hosting and storage platform as well as SQL Azure, the relational database in the cloud, by moving an existing ASP.NET application to a higher level."

And here's the live recording:

Thanks for joining TechDays 2010 and my session!

Using FTP to access Windows Azure Blob Storage

A while ago, I did a blog post on creating an external facing Azure Worker Role endpoint, listening for incoming TCP connections. After doing that post, I had the idea of building a Windows Azure FTP server that served as a bridge to blob storage. Lack of time, other things to do, you name it: I did not work on that idea. Until now, that is.

Being a lazy developer, I did not start from scratch: writing an FTP server may be something that has been done before, and yes: “Binging” for “ Csharp FTP server” led me to this article on CodeGuru.com. Luckily, the author of the article had the idea of abstraction in mind: he did not build his software on top of a real file system, no, he did an abstraction. This would mean I would only have to host this thing in a worker role somehow and add some classes working with blobs and not with files. Cool!

kick it on DotNetKicks.com Shout it

Demo of the FTP to Blob Storage bridge

Well, you can try this one yourself actually… But let’s start with a disclaimer: I’m not logging your account details when you log in. Next, I’m not allowing you to transfer more than 10MB of data per day. If you require this, feel free to contact me and I’ll give you more traffic quotas.

Open up your favourite FTP client (like FileZilla), and open up an FTP connection to ftp.cloudapp.net. Don’t forget to use your Windows Azure storage account name as the username and the storage account key as the password. Connect, and you’ll be greeted in a nice way:

Windows Azure Blob FileZilla

The folders you are seeing are your blob storage containers. Feel free to browse your storage account and:

  • Create, remove and rename blob containers.
  • Create, remove and rename folders inside a container. Note that a .placeholder file will be created when doing this.
  • Upload and download blobs.

Feels like regular FTP, right? There’s more though… Using the Windows Azure storage API, you can also choose if a blob container is private or public. Why not do this using the FTP client? Right-click a blob container,  pick “File permissions…” and here you are: the public read permission is the one that you can use to control access to a blob container.

Change container permission through FTP

Show me the code!

Well… No! I think it’s not stable enough for releasing it to public yet. But what I will do is share some off my struggles I faced while developing on this.

Struggle #1: Quotas

As you may have noticed: I’m not allowing data transfers of more than 10 MB per day per storage account. This is not much, but I did not want  to go pay for other people’s traffic that comes trough a demo app. However: every command you send, every action you take, is generating traffic. I had to choose how this would be logged and persisted.

The strategy used is that all transferred bytes are counted and stored in a cache in the worker role. I created a dedicated thread that monitors this cache, and regularly persists the traffic log in blob storage. There is no fixed interval in which this happens, it just happens. I’m not sure yet that this is the best way to do it, but I feel it is a good mix between intensity of logging and intensity of an expensive write to blob storage.

Struggle #2: Sleep

This is not a technical struggle. Since I had fun, I dedicated a lot of time to this thing, mainly in fine-tuning, testing, testing with multiple concurrent clients, … I learnt that System.Net has some cool classes and also learnt that TcpClient that are closed should also be disposed. Otherwise, the socket will not be released and no new connections will be accepted after a while. Anyway: it caused a lack of sleep. The solution to this was drinking more coffee, just at the moment where I actually was drinking less coffee for over a month or two. I will have to go to coffee-rehab again…

Struggle #3: FTP PASV mode

I will not assume you know this because I also didn’t know the exact difference… When a client connects to an FTP server, it will have 2 connections with that server. One on the standard FTP TCP port 21 used for sending commands back and forth, and one on another TCP port used for transferring data. This second connection can be an active one or a passive one.

The main difference between active and passive FTP lies in the direction of the connection: with active FTP, the FTP server opens a connection to a TCP port on the client, while with passive FTP, the client will open a connection to another TCP port on the server. Here’s more details on that:

The Basics

FTP is a TCP based service exclusively. There is no UDP component to FTP. FTP is an unusual service in that it utilizes two ports, a 'data' port and a 'command' port (also known as the control port). Traditionally these are port 21 for the command port and port 20 for the data port. The confusion begins however, when we find that depending on the mode, the data port is not always on port 20.

Active FTP

In active mode FTP the client connects from a random unprivileged port (N > 1023) to the FTP server's command port, port 21. Then, the client starts listening to port N+1 and sends the FTP command PORT N+1 to the FTP server. The server will then connect back to the client's specified data port from its local data port, which is port 20.

From the server-side firewall's standpoint, to support active mode FTP the following communication channels need to be opened:

  • FTP server's port 21 from anywhere (Client initiates connection)
  • FTP server's port 21 to ports > 1023 (Server responds to client's control port)
  • FTP server's port 20 to ports > 1023 (Server initiates data connection to client's data port)
  • FTP server's port 20 from ports > 1023 (Client sends ACKs to server's data port)

When drawn out, the connection appears as follows:

In step 1, the client's command port contacts the server's command port and sends the command PORT 1027. The server then sends an ACK back to the client's command port in step 2. In step 3 the server initiates a connection on its local data port to the data port the client specified earlier. Finally, the client sends an ACK back as shown in step 4.

The main problem with active mode FTP actually falls on the client side. The FTP client doesn't make the actual connection to the data port of the server--it simply tells the server what port it is listening on and the server connects back to the specified port on the client. From the client side firewall this appears to be an outside system initiating a connection to an internal client--something that is usually blocked.

Passive FTP

In order to resolve the issue of the server initiating the connection to the client a different method for FTP connections was developed. This was known as passive mode, or PASV, after the command used by the client to tell the server it is in passive mode.

In passive mode FTP the client initiates both connections to the server, solving the problem of firewalls filtering the incoming data port connection to the client from the server. When opening an FTP connection, the client opens two random unprivileged ports locally (N > 1023 and N+1). The first port contacts the server on port 21, but instead of then issuing a PORT command and allowing the server to connect back to its data port, the client will issue the PASV command. The result of this is that the server then opens a random unprivileged port (P > 1023) and sends the PORT P command back to the client. The client then initiates the connection from port N+1 to port P on the server to transfer data.

From the server-side firewall's standpoint, to support passive mode FTP the following communication channels need to be opened:

  • FTP server's port 21 from anywhere (Client initiates connection)
  • FTP server's port 21 to ports > 1023 (Server responds to client's control port)
  • FTP server's ports > 1023 from anywhere (Client initiates data connection to random port specified by server)
  • FTP server's ports > 1023 to remote ports > 1023 (Server sends ACKs (and data) to client's data port)

When drawn, a passive mode FTP connection looks like this:

In step 1, the client contacts the server on the command port and issues the PASV command. The server then replies in step 2 with PORT 2024, telling the client which port it is listening to for the data connection. In step 3 the client then initiates the data connection from its data port to the specified server data port. Finally, the server sends back an ACK in step 4 to the client's data port.

While passive mode FTP solves many of the problems from the client side, it opens up a whole range of problems on the server side. The biggest issue is the need to allow any remote connection to high numbered ports on the server. Fortunately, many FTP daemons, including the popular WU-FTPD allow the administrator to specify a range of ports which the FTP server will use. See Appendix 1 for more information.

The second issue involves supporting and troubleshooting clients which do (or do not) support passive mode. As an example, the command line FTP utility provided with Solaris does not support passive mode, necessitating a third-party FTP client, such as ncftp.

With the massive popularity of the World Wide Web, many people prefer to use their web browser as an FTP client. Most browsers only support passive mode when accessing ftp:// URLs. This can either be good or bad depending on what the servers and firewalls are configured to support.

(from http://slacksite.com/other/ftp.html)

Clear enough? Good! In order to support passive FTP, the Windows Azure worker role should be listening on more ports than only port 21. After doing some research, I found that most FTP servers allow specifying the passive FTP port range. Opening a range of over 1000 TCP ports is also something most FTP servers seem to do. Good, I tried this one on Windows Azure, deployed it and… found out that you can only define a maximum of 5 public endpoints per deployment.

This led me to re-implementing PASV mode, opening a new port on demand from a pool of 4 public endpoints defined. Again, I deployed this one but this failed as well: there was too much of a delay in opening a new TcpListener on the fly.

Option three seemed to work: I have a TcpListener open on TCP port 20 all the time and try to dispatch incoming connections immediately. There’s also a downside to this: if users send a lot of PASV requests, there will be a lot of unused connections that may cause the application to crash. So I did a trick here as well: close listening connections after a short delay.

Conclusion

Feel free to use the service and if you require more than 10 MB traffic a day, feel free to contact me. I can specify traffic quotas per storage account and may increase traffic quotas for yours.

kick it on DotNetKicks.com Shout it