Maarten Balliauw {blog}

ASP.NET, ASP.NET MVC, Windows Azure, PHP, ...

NAVIGATION - SEARCH

Advanced scenarios with Windows Azure Queues

For DeveloperFusion, I wrote an article on Windows Azure queues. Interested in working with queues and want to use some advanced techniques? Head over to the article:

Last week, in Brian Prince’s article, Using the Queuing Service in Windows Azure, you saw how to create, add messages into, retrieve and consume those messages from Windows Azure Queues. While being a simple, easy-to-use mechanism, a lot of scenarios are possible using this near-FIFO queuing mechanism. In this article we are going to focus on three scenarios which show how queues can be an important and extremely scalable component in any application architecture:

  • Back off polling, a method to lessen the number of transactions in your queue and therefore reduce the bandwidth used.
  • Patterns for working with large queue messages, a method to overcome the maximal size for a queue message and support a greater amount of data.
  • Using a queue as a state machine.

The techniques used in every scenario can be re-used in many applications and often be combined into an approach that is both scalable and reliable.

To get started, you will need to install the Windows Azure Tools for Microsoft Visual Studio. The current version is 1.4, and that is the version we will be using. You can download it from http://www.microsoft.com/windowsazure/sdk/.

N.B. The code that accompanies this article comes as a single Visual Studio solution with three console application projects in it, one for each scenario covered. To keep code samples short and to the point, the article covers only the code that polls the queue and not the additional code that populates it. Readers are encouraged to discover this themselves.

Want more content? Check Advanced scenarios with Windows Azure Queues. Enjoy!

MyGet now supports pushing from the command line

One of the work items we had opened for MyGet was the ability to push packages to a private feed from the command line. Only a few hours after our initial launch, David Fowler provided us with example code on how to implement NuGet command line pushes on the server side. An evening of coding later, I quickly hacked this into MyGet, which means that we now support pushing packages from the command line!

For those that did not catch up with my blog post overload of the past week: MyGet offers you the possibility to create your own, private, filtered NuGet feed for use in the Visual Studio Package Manager.  It can contain packages from the official NuGet feed as well as your private packages, hosted on MyGet. Want a sample? Add this feed to your Visual Studio package manager: http://www.myget.org/F/chucknorris

Pushing a package from the command line to MyGet

The first thing you’ll be needing is an API key for your private feed. This can be obtained through the “Edit Feed” link, where you’ll see an API key listed as well as a button to regenerate the API key, just in case someone steals it from you while giving a demo of MyGet :-)

image

Once you have the API key, it can be stored into the NuGet executable’s settings by running the following command, including your private API key and your private feed URL:

1 NuGet setApiKey c18673a2-7b57-4207-8b29-7bb57c04f070 -Source http://www.myget.org/F/testfeed

After that, you can easily push a package to your private feed. The package will automatically show up on the website and your private feed. Do note that this can take a few minutes to propagate.

1 NuGet push RouteMagic.0.2.2.2.nupkg -Source http://www.myget.org/F/testfeed

More on the command line can be found on the NuGet documentation wiki.

Other change: authentication to the website

Someone on Twitter (@corydeppen) complained he had to login using Windows Live ID. Because we’re using the Windows Azure AppFabric Access Control Service (which I’ll abbreviate to ACS next time), this was in fact a no-brainer. We now support Windows Live ID, Google, Yahoo! and Facebook as authentication mechanisms for MyGet. Enjoy!

Creating your own private NuGet feed: MyGet

myget - NuGet as a serverEver since NuGet came out, I’ve been thinking about leveraging it in a corporate environment. I've seen two NuGet server implementations appear on the Internet: the official NuGet gallery server and Phil Haack’s NuGet.Server package. As these both are good, there’s one thing wrong with them: you can't be lazy! You have to do some stuff you don’t always want to do, namely: configure and deploy.

After discussing some ideas with my colleague Xavier Decoster, we decided it’s time to turn our heads into the cloud: we’re providing you NuGet-as-a-Service (NaaS)! Say hello to MyGet.

MyGet offers you the possibility to create your own, private, filtered NuGet feed for use in the Visual Studio Package Manager.
It can contain packages from the official NuGet feed as well as your private packages, hosted on MyGet. Want a sample? Add this feed to your Visual Studio package manager: http://www.myget.org/F/chucknorris

But wait, there’s more: we’re open sourcing this thing! Feel free to fork over at CodePlex and extend our "product". We've already covered some feature requests we would love to see, and Xavier has posted some more on his blog. In short: feel free to add your own most-wanted features, provide us with bugfixes (pretty sure there will be a lot since we hacked this together in a very short time). We're hosting on WIndows Azure, which means you should get the Windows Azure SDK installed prior to contributing. Unless you feel that you can write code without locally debugging :-)

Chuck Norris Feed

Feel free to go ahead and create your private feed. Some ideas (more at Xavier's site):

  • A feed containing only the packages you or your company often use
  • A feed containing only your (open-source?) project and its dependencies
  • A feed containing just a few packages that you want to use for a certain project: tell your developers to just install them all

Bugs and feature requests? Feel free to post them as a comment below. Once we release the sources, I’ll kick your mailbox with a request to implement the stuff you proposed. Seems fair to me :-)

Enjoy http://myget.org!

Scaffolding and packaging a Windows Azure project in PHP

Scaffolding CloudWith the fresh release of the Windows Azure SDK for PHP v3.0, it’s time to have a look at the future. One of the features we’re playing with is creating a full-fledged replacement for the current Windows Azure Command-Line tools available. These tools sometimes are a life saver and sometimes a big PITA due to baked-in defaults and lack of customization options. And to overcome that last one, here’s what we’re thinking of: scaffolders.

Basically what we’ll be doing is splitting the packaging process into two steps:

  • Scaffolding
  • Packaging

To get a feeling about all this, I strongly suggest you to download the current preview version of this new concept and play along.

By the way: feedback is very welcome! Just comment on this post and I’ll get in touch.

Scaffolding a Windows Azure project

Scaffolding a Windows Azure project consists of creating a “template” for your Windows Azure project. The idea is that we can provide one or more default scaffolders that can generate a template for you, but there’s no limitation on creating your own scaffolders (or using third party scaffolders).

The default scaffolder currently included is based on a blog post I did earlier about having a lightweight deployment. Creating a template for a Windows Azure project is pretty simple:

1 Package Scaffold -p:"C:\temp\Sample" --DiagnosticsConnectionString:"UseDevelopmentStorage=true"

This command will generate a folder structure in C:\Temp\Sample and uses the default scaffolder (which requires the parameter “DiagnosticsConnectionString to be specified). Nothing however prevents you from creating your own (later in this post).

image

Once you have the folder structure in place, the only thing left is to copy your application contents into the “PhpOnAzure.Web” folder. In case of this default scaffolder, that is all that is required to create your Windows Azure project structure. Nothing complicated until now, and I promise you things will never get complicated. However if you are a brave soul, you can at this point customize the folder structure, add our custom configuration settings, …

Packaging a Windows Azure project

After the scaffolding comes the packaging. Again, a very simple command:

1 Package Create -p:"C:\temp\Sample" -dev:false

The above will create a Sample.cspkg file which you can immediately deploy to Windows Azure. Either through the portal or using the Windows Azure command line tools that are included in the current version of the Windows Azure SDK for PHP.

Building your own scaffolder

Scaffolders are in fact Phar archives, a PHP packaging standard which is in essence a file containing executable PHP code as well as resources like configuration files, images, …

A scaffolder is typically a structure containing a resources folder containing configuration files or a complete PHP deployment or something like that, and a file named index.php, containing the scaffolding logic. Let’s have a look at index.php.

1 <?php 2 class Scaffolder 3 extends Microsoft_WindowsAzure_CommandLine_PackageScaffolder_PackageScaffolderAbstract 4 { 5 /** 6 * Invokes the scaffolder. 7 * 8 * @param Phar $phar Phar archive containing the current scaffolder. 9 * @param string $root Path Root path. 10 * @param array $options Options array (key/value). 11 */ 12 public function invoke(Phar $phar, $rootPath, $options = array()) 13 { 14 // ... 15 } 16 }

Looks simple, right? It is. The invoke method is the only thing that you should implement: this can be a method extracting some content to the $rootPath as well as updating some files in there as well as… anything! If you can imagine ourself doing it in PHP, it’s possible in a scaffolder.

Packaging a scaffolder is the last step in creating one: copying all files into a .phar file. And wouldn’t it be fun if that task was easy as well? Check this command:

1 Package CreateScaffolder -p:"/path/to/scaffolder" -out:"/path/to/MyScaffolder.phar"

There you go.

Ideas for scaffolders

I’m not going to provide all the following scaffolders out of the box, but here’s some scaffolders that I’m thinking would be interesting:

  • A scaffolder including a fully tweaked configured PHP runtime (with SQL Server Driver for PHP, Wincache, …)
  • A scaffolder which enables remote desktop
  • A scaffolder which contains an autoscaling mechanism
  • A scaffolder that can not exist on its own but can provide additional functionality to an existing Windows Azure project

Enjoy! And as I said: feedback is very welcome!

Just released: MvcSiteMapProvider 3.1.0 RC

ASP.NET MVC Sitemap providerIt looks like I’m really cr… ehm… releasing way too much over the past few days, but yes, here’s another one: I just posted MvcSiteMapProvider 3.1.0 RC both on CodePlex and NuGet.

The easiest way to get the current bits is this one:

Install-Package MvcSiteMapProvider

As usual, here are the release notes:

  • Created one NuGet package containing both .NET 3.5 and .NET 4.0 assemblies
  • Significantly improved memory usage and performance
  • Medium Trust optimizations
  • DefaultControllerTypeResolver speed improvement
  • Resolve authorize attributes through FilterProviders.Current (in MVC3)
  • Allow to specify target on SiteMapTitleAttribute
  • Fix the NuGet package DisplayTemplates folder location
  • Fixed: Nuget web.config section duplication
  • Fixed: HelperMenu.Menu() always uses default provider
  • Fixed: 2.x Uses Default Parameters
  • Fixed: Bad Null Checking in MvcSiteMapProvider.DefaultSiteMapProvider
  • Fixed: Exception: An item with the same key has already been added.
  • Fixed: Add id="menu" to default MenuHelperModel DisplayTemplate (not in NuGet yet)
  • Fixed: Wrong Breadcrumb Displayed Under Heavy Load
  • Fixed: Backport Route support to 2.3.1

Windows Azure SDK for PHP v3.0 released

Microsoft and RealDolmen are very proud to announce the availability of the Windows Azure SDK for PHP v3.0 on CodePlex! (here's the official Microsoft post) This open source SDK gives PHP developers a speed dial library to fully take advantage of Windows Azure’s cool features. Version 3.0 of this SDK marks an important milestone because we’re not only starting to witness real world deployment, but also we’re seeing more people joining the project and contributing.

New features include a pluggable logging infrastructure (based on Table Storage) as well as a full implementation of the Windows Azure management API. This means that you can now build your own Windows Azure Management Portal using PHP. How cool is that? What’s even cooler about this… Well… how about combining some features and build an autoscaling web application in PHP? Checkout http://dealoftheday.cloudapp.net/ for a sample of that. Make sure to read through as there are some links to how you can autoscale YOUR apps as well!

A comment we received a lot for previous versions was the fact that for table storage, datetime values were returned as strings and parsing of them was something you as a developer should do. In this release, we’ve broken that: table storage entities now return native PHP DateTime objects instead of strings for Edm.DateTime properties.

Here’s the official changelog:

  • Breaking change: Table storage entities now return DateTime objects instead of strings for Edm.DateTime properties
  • New feature: Service Management API in the form of Microsoft_WindowsAzure_Management_Client
  • New feature: logging infrastructure on top of table storage
  • Session provider now works on table storage for small sessions, larger sessions can be persisted to blob storage
  • Queue storage client: new hasMessages() method
  • Introduction of an autoloader class, increasing speed for class resolving
  • Several minor bugfixes and performance tweaks

Find the current download at http://phpazure.codeplex.com/releases/view/66558. Do you prefer PEAR? Well... pear channel-discover pear.pearplex.net & pear install pearplex/PHPAzure should do the trick.

Microsoft .NET Framework 4 Platform Update 1 KB2478063 Service Pack 5 Feature Set 3.1 R2 November Edition RTW

As you can see, a new .NET Framework version just came out. Read about it at http://blogs.msdn.com/b/endpoint/archive/2011/04/18/microsoft-net-framework-4-platform-update-1.aspx. Now why does my title not match with the title from the blog post I referenced? Well… How is this going to help people?

For those who don’t see the problem, let me explain… If we get new people on board that are not yet proficient enough in .NET, they all struggle with some concepts. Concepts like: service packs for a development framework. Or better: client profile stuff! Stuff that breaks their code because stuff is missing in there! I feel like this is going the Java road where every version has a billion updates associated with it. That’s not where we want to go, right? The Java side?

image

As I’m saying: why not make things clear and call these “updates” something like .NET 4.1 or so? Simple major/minor versions. We’re developers, not marketeers. We’re developers, not ITPro who enjoy these strange names to bill yet another upgrade to their customers

How am I going to persuade my manager to move to the next version? Telling him that we now should use “Microsoft .NET Framework 4 Platform Update 1 KB2478063” instead of telling “hey, there’s a new .NET 4! It’s .NET 4.1 and it’s shiny and new!”.

It seems I’m not alone with this thought. Hadi Hariri also blogged about it. And I expect more to follow... If you feel the same: now is the time to stop this madness! I suspect there’s an R2 November Edition coming otherwise…

[Edit @ 14:00] Here's how to use it in NuGet. Seems this thing is actually ".NET 4.0.1" under the hood.
[Edit @ 14:01] And here's another one. And another one.
[Edit] And Scott Hanselman chimes in: www.hanselman.com/.../...oftProductVersioning.aspx

A Glimpse at Windows Identity Foundation claims

For a current project, I’m using Glimpse to inspect what’s going on behind the ASP.NET covers. I really hope that you have heard about the greatest ASP.NET module of 2011: Glimpse. If not, shame on you! Install-Package Glimpse immediately! And if you don’t know what I mean by that, NuGet it now! (the greatest .NET addition since sliced bread).

This project is also using Windows Identity Foundation. It’s really a PITA to get a look at the claims being passed around. Usually, I do this by putting a breakpoint somewhere and inspecting the current IPrincipal’s internals. But with Glimpse, using a small plugin to just show me the claims and their values is a no-brainer. Check the right bottom of this '(partial) screenshot:

Glimpse Windows Identity Foundation

Want to have this too? Simply copy the following class in your project and you’re done:

1 [GlimpsePlugin()] 2 public class GlimpseClaimsInspectorPlugin : IGlimpsePlugin 3 { 4 public object GetData(HttpApplication application) 5 { 6 // Return the data you want to display on your tab 7 var data = new List<object[]> { new[] { "Identity", "Claim", "Value", "OriginalIssuer", "Issuer" } }; 8 9 // Add all claims found 10 var claimsPrincipal = application.User as ClaimsPrincipal; 11 if (claimsPrincipal != null) 12 { 13 foreach (var identity in claimsPrincipal.Identities) 14 { 15 foreach (var claim in identity.Claims) 16 { 17 data.Add(new object[] { identity.Name, claim.ClaimType, claim.Value, claim.OriginalIssuer, claim.Issuer }); 18 } 19 } 20 } 21 22 return data; 23 } 24 25 public void SetupInit(HttpApplication application) 26 { 27 } 28 29 public string Name 30 { 31 get { return "WIF Claims"; } 32 } 33 }

Enjoy! And if you feel like NuGet-packaging this (or including it with Glimpse), feel free.

Using dynamic WCF service routes

DynamicFor a demo I am working on, I’m creating an OData feed. This OData feed is in essence a WCF service which is activated using System.ServiceModel.Activation.ServiceRoute. The idea of using that technique is simple: map an incoming URL route, e.g. “http://example.com/MyService” to a WCF service. But there’s a catch in ServiceRoute: unlike ASP.NET routing, it does not support the usage of route data. This means that if I want to create a service which can exist multiple times but in different contexts, like, for example, a “private” instance of that service for a customer, the ServiceRoute will not be enough. No support for having http://example.com/MyService/Contoso/ and http://example.com/MyService/AdventureWorks to map to the same “MyService”. Unless you create multiple ServiceRoutes which require recompilation. Or… unless you sprinkle some route magic on top!

Implementing an MVC-style route for WCF

Let’s call this thing DynamicServiceRoute. The goal of it will be to achieve a working ServiceRoute which supports route data and which allows you to create service routes of the format “MyService/{customername}”, like you would do in ASP.NET MVC.

First of all, let’s inherit from RouteBase and IRouteHandler. No, not from ServiceRoute! The latter is so closed that it’s basically a no-go if you want to extend it. Instead, we’ll wrap it! Here’s the base code for our DynamicServiceRoute:

1 public class DynamicServiceRoute 2 : RouteBase, IRouteHandler 3 { 4 private string virtualPath = null; 5 private ServiceRoute innerServiceRoute = null; 6 private Route innerRoute = null; 7 8 public static RouteData GetCurrentRouteData() 9 { 10 } 11 12 public DynamicServiceRoute(string pathPrefix, object defaults, ServiceHostFactoryBase serviceHostFactory, Type serviceType) 13 { 14 } 15 16 public override RouteData GetRouteData(HttpContextBase httpContext) 17 { 18 } 19 20 public override VirtualPathData GetVirtualPath(RequestContext requestContext, RouteValueDictionary values) 21 { 22 } 23 24 public System.Web.IHttpHandler GetHttpHandler(RequestContext requestContext) 25 { 26 } 27 }

As you can see, we’re creating a new RouteBase implementation and wrap 2 routes: an inner ServiceRoute and and inner Route. The first one will hold all our WCF details and will, in one of the next code snippets, be used to dispatch and activate the WCF service (or an OData feed or …). The latter will be used for URL matching: no way I’m going to rewrite the URL matching logic if it’s already there for you in Route.

Let’s create a constructor:

1 public DynamicServiceRoute(string pathPrefix, object defaults, ServiceHostFactoryBase serviceHostFactory, Type serviceType) 2 { 3 if (pathPrefix.IndexOf("{*") >= 0) 4 { 5 throw new ArgumentException("Path prefix can not include catch-all route parameters.", "pathPrefix"); 6 } 7 if (!pathPrefix.EndsWith("/")) 8 { 9 pathPrefix += "/"; 10 } 11 pathPrefix += "{*servicePath}"; 12 13 virtualPath = serviceType.FullName + "-" + Guid.NewGuid().ToString() + "/"; 14 innerServiceRoute = new ServiceRoute(virtualPath, serviceHostFactory, serviceType); 15 innerRoute = new Route(pathPrefix, new RouteValueDictionary(defaults), this); 16 }

As you can see, it accepts a path prefix (e.g. “MyService/{customername}”), a defaults object (so you can say new { customername = “Default” }), a ServiceHostFactoryBase (which may sound familiar if you’ve been using ServiceRoute) and a service type, which is the type of the class that will be your WCF service.

Within the constructor, we check for catch-all parameters. Since I’ll be abusing those later on, it’s important the user of this class can not make use of them. Next, a catch-all parameter {*servicePath} is appended to the pathPrefix parameter. I’m doing this because I want all calls to a path below “MyService/somecustomer/…” to match for this route. Yes, I can try to do this myself, but again this logic is already available in Route so I’ll just reuse it.

One other thing that happens is a virtual path is generated. This will be a fake path that I’ll use as the URL to match in the inner ServiceRoute. This means if I navigate to “MyService/SomeCustomer” or if I navigate to “MyServiceNamespace.MyServiceType-guid”, the same route will trigger. The first one is the pretty one that we’re trying to create, the latter is the internal “make-things-work” URL. Using this virtual path and the path prefix, simply create a ServiceRoute and Route.

Actually, a lot of work has been done in 3 lines of code in the constructor. What’s left is just an implementation of RouteBase which calls the corresponding inner logic. Here’s the meat:

1 public override RouteData GetRouteData(HttpContextBase httpContext) 2 { 3 return innerRoute.GetRouteData(httpContext); 4 } 5 6 public override VirtualPathData GetVirtualPath(RequestContext requestContext, RouteValueDictionary values) 7 { 8 return null; 9 } 10 11 public System.Web.IHttpHandler GetHttpHandler(RequestContext requestContext) 12 { 13 requestContext.HttpContext.RewritePath("~/" + virtualPath + requestContext.RouteData.Values["servicePath"], true); 14 return innerServiceRoute.RouteHandler.GetHttpHandler(requestContext); 15 }

I told you it was easy, right? GetRouteData is used by the routing engine to check if a route matches. We just pass that call to the inner route which is able to handle this. GetVirtualPath will not be important here, so simply return null there. If you really really feel this is needed, it would require some logic that creates a URL from a set of route data. But since you’ll probably never have to do that, null is good here. The most important thing here is GetHttpHandler. It is called by the routing engine to get a HTTP handler for a specific request context if the route matches. In this method, I simply rewrite the requested URL to the internal, ugly “MyServiceNamespace.MyServiceType-guid” URL and ask the inner ServiceRoute to have fun with it and serve the request. There, the magic just happened.

Want to use it? Simply register a new route:

1 var dataServiceHostFactory = new DataServiceHostFactory(); 2 RouteTable.Routes.Add(new DynamicServiceRoute("MyService/{customername}", null, dataServiceHostFactory, typeof(MyService)));

Conclusion

Why would you need this? Well, imagine you are building a customer-specific service where you want to track service calls for a specific sutomer. For example, if you’re creating private NuGet repositories. And yes, this was a hint on a future blog post :-)

Feel this is useful to you as well? Grab the code here: DynamicServiceRoute.cs (1.94 kb)