Azure Image Proxy

The previous couple of articles configured an image resizing Azure Web Role, plopped those resized images on an Azure Service Bus, picked them up with a Worker Role and saved them into Blob Storage.

This one will click in the last missing piece; the proxy at the front to initially attempt to get the pregenerated image from blob storage and failover to requesting a dynamically resized image.

New Web Role

Add a new web role to your cloud project – I’ve called mine “ImagesProxy” – and make it an empty MVC4 Web API project. This is the easiest of the projects, so you can just crack right on and create a new controller – I called mine “Image” (not the best name, but it’ll do).

Retrieve

This whole project will consist of one controller with one action – Retrieve – which does three things;

  1. attempt to retrieve the resized image directly from blob storage
  2. if that fails, go and have it dynamically resized
  3. if that fails, send a 404 image and the correct http header

Your main method/action should look something like this:

[HttpGet]
public HttpResponseMessage Retrieve(int height, int width, string source)
{
    try
    {
        var resizedFilename = BuildResizedFilenameFromParams(height, width, source);
        var imageBytes = GetFromCdn("resized", resizedFilename);
        return BuildImageResponse(imageBytes, "CDN", false);
    }
    catch (StorageException)
    {
        try
        {
            var imageBytes = RequestResizedImage(height, width, source);
            return BuildImageResponse(imageBytes, "Resizer", false);
        }
        catch (WebException)
        {
            var imageBytes = GetFromCdn("origin", "404.jpg");
            return BuildImageResponse(imageBytes, "CDN-Error", true);
        }
    }
}

Feel free to alt-enter and clean up the red squiggles by creating stubs and referencing the necessary assemblies.

You should be able to see the three sections mentioned above within the nested try-catch blocks.

  1. attempt to retrieve the resized image directly from blob storage

    var resizedFilename = BuildResizedFilenameFromParams(height, width, source);
    var imageBytes = GetFromCdn("resized", resizedFilename);
    return BuildImageResponse(imageBytes, "CDN", false);
    
  2. if that fails, go and have it dynamically resized

    var imageBytes = RequestResizedImage(height, width, source);
    return BuildImageResponse(imageBytes, "Resizer", false)
    
  3. if that fails, send a 404 image and the correct http header

    var imageBytes = GetFromCdn("origin", "404.jpg");
    return BuildImageResponse(imageBytes, "CDN-Error", true);
    

So let’s build up those stubs.

BuildResizedFilenameFromParams

Just a little duplication of code to get the common name of the resized image (yes, yes, this logic should have been abstracted out into a common library for all projects to reference, I know, I know..)

private static string BuildResizedFilenameFromParams(int height, int width, string source)
{
    return string.Format("{0}_{1}-{2}", height, width, source.Replace("/", string.Empty));
}

GetFromCDN

We’ve seen this one before too; just connecting into blob storage (within these projects blob storage is synonymous with CDN) to pull out the pregenerated/pre-reseized image:

private static byte[] GetFromCdn(string path, string filename)
{
    var connectionString = CloudConfigurationManager.GetSetting("Microsoft.Storage.ConnectionString");
    var account = CloudStorageAccount.Parse(connectionString);
    var cloudBlobClient = account.CreateCloudBlobClient();
    var cloudBlobContainer = cloudBlobClient.GetContainerReference(path);
    var blob = cloudBlobContainer.GetBlockBlobReference(filename);

    var m = new MemoryStream();
    blob.DownloadToStream(m);

    return m.ToArray();
}

BuildImageResponse

Yes, yes, I know – more duplication.. almost. The method to create an HTTP response message from before, but this time with extras params to set a header saying where the image came from, and allow to set the HTTP status code correctly. We’re just taking the image bytes and putting them in the message content, whilst setting the headers and status code appropriately.

private static HttpResponseMessage BuildImageResponse(byte[] imageBytes, string whereFrom, bool error)
{
    var httpResponseMessage = new HttpResponseMessage { Content = new ByteArrayContent(imageBytes) };
    httpResponseMessage.Content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
    httpResponseMessage.Content.Headers.Add("WhereFrom", whereFrom);
    httpResponseMessage.StatusCode = error ? HttpStatusCode.NotFound : HttpStatusCode.OK;

    return httpResponseMessage;
}

RequestResizedImage

Build up a request to our pre-existing image resizing service via a cloud config setting and the necessary dimensions and filename, and return the response:

private static byte[] RequestResizedImage(int height, int width, string source)
{
    byte[] imageBytes;
    using (var wc = new WebClient())
    {
        imageBytes = wc.DownloadData(
            string.Format("{0}?height={1}&width={2}&source={3}",
                          CloudConfigurationManager.GetSetting("Resizer_Endpoint"), 
                          height, width, source));
    }
    return imageBytes;
}

And that’s all there is to it! A couple of other changes to make within your project in order to allow pretty URLs:

  1. Create the necessary route:

    config.Routes.MapHttpRoute(
        name: "Retrieve",
        routeTemplate: "{height}/{width}/{source}",
        defaults: new { controller = "Image", action = "Retrieve" }
    );
    
  2. Be a moron:

      <system.webServer>
        <modules runAllManagedModulesForAllRequests="true" />
      </system.webServer>
    

That last one is dangerous; I’m using it here as a quick hack to ensure that URLs ending with known file extensions (e.g., /600/200/image1.jpg) are still processed by the MVC app instead of assuming they’re static files on the filesystem. However, this setting is not advised since it means that every request will be picked up by your .Net app; don’t use it in regular web apps which also host images, js, css, etc!

If you don’t use this setting then you’ll go crazy trying to debug your routes, wondering why nothing is being hit even after you install Glimpse..

In action

First request

Hit your proxy with a request for an image that exists within your blob storage “origin” folder; this will raise a storage exception when attempting to retrieve from blob storage and drop into the resizer code chunk e.g.:
image proxy, calling the resizer
Notice the new HTTP header that tells us the request was fulfilled via the Resizer service, and we got an HTTP 200 status code. The resizer web role will have also added a message to the service bus awaiting pick up.

Second request

By the time you refresh that page (if you’re not too trigger happy) the uploader worker role should have picked up the message from the service bus and saved the image data into blob storage, such that subsequent requests should end up with a response similar to:
image proxy, getting it from cdn
Notice the HTTP header tells us the request was fulfilled straight from blob storage (CDN), and the request was successful (HTTP 200 response code).

Failed request

If we request an image that doesn’t exist within the “origin” folder, then execution drops into the final code chunk where we return a default image and set an error status code:
image proxy, failed request

So..

This is the last bit of the original plan:

Azure Image Resizing - Conceptual Architecture

Please grab the source from github, add in your own settings to the cloud config files, and have a go. It’s pretty cool being able to just upload one image and have other dimension images autogenerated upon demand!

Automated Image Resizing and Hosting in Azure #2

Saving the resized images

Last article concluded with us creating a web role that will retrieve an image from blob storage, resize it, raise an event, and stream the result back.

This article is about the worker role to handle those raised events.

Simply enough, all we’ll be doing is creating a worker role, hooking into the same azure service bus queue, picking up each message, pulling out the relevant data within, and uploading that to blob storage.

Overall Process

A reminder of the overall process:
Azure Image Resizing Conceptual Architecture

The Worker Role

The section of that which the worker role is responsible for is as below:

Azure-Image-Resizing-Uploader-Achitecture

Add a new worker role to the Cloud project within the solution from last time (or a new one if you like). This one consists of four little methods; Run, OnStart, and OnEnd, where Run will call an UploadBlob method.

Run

This method will pick up any messages appearing on the queue, deserialize the contents of the message to a known structure, and pass them to an uploading method.

Kick off by pasting over the Run method with this one, including the definitions at the top – set the QueueName to the same queue you configured for the resize notification from the last post:

    const string QueueName = "azureimages";
    QueueClient _client;
    readonly ManualResetEvent _completedEvent = new ManualResetEvent(false);

    public override void Run()
    {
        _client.OnMessage(receivedMessage =>
        {
            try
            {
                // Process the message
                var receivedImage = receivedMessage.GetBody<ImageData>();
                UploadBlob("resized", receivedImage);
            }
            catch (Exception e)
            {
                Trace.WriteLine("Exception:" + e.Message);
            }
        }, new OnMessageOptions
        {
            AutoComplete = true,
            MaxConcurrentCalls = 1
        });

        _completedEvent.WaitOne();
    }

Yes, I’m not doing anything with exceptions; that’s an exercise for the reader.. ahem… (Me? Lazy? Never..happypathhappypathhappypath)

Naturally you’ll get a few squiggles and highlights to fix; Install-Package Microsoft.ServiceBus.NamespaceManager will help with some, as will creating the stub UploadBlob.

Now, to tidy up the reference to ImageData you could do a few things:

  1. Copy the ImageData.cs over from the previous project into this one
  2. Create a reference to the previous project and add in a using to this file
  3. Extract ImageData from the previous project into a common referenced project for them both to share.

I can live with my own conscience, so am just whacking in a reference to the previous project. Don’t hate me.

OnStart and OnStop

   public override bool OnStart()
    {
        // Set the maximum number of concurrent connections 
        ServicePointManager.DefaultConnectionLimit = 2;

        // Create the queue if it does not exist already
        var connectionString = CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");
        var namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
        if (!namespaceManager.QueueExists(QueueName))
        {
            namespaceManager.CreateQueue(QueueName);
        }

        // Initialize the connection to Service Bus Queue
        _client = QueueClient.CreateFromConnectionString(connectionString, QueueName);
        return base.OnStart();
    }

    public override void OnStop()
    {
        // Close the connection to Service Bus Queue
        _client.Close();
        _completedEvent.Set();
        base.OnStop();
    }

OnStart gets a connection to the service bus, creates the named queue if necessary, and creates a queue client referencing that queue within that service bus.

OnStop kills everything off.

So, off you pop and add the requisite service connection string details; right click the role within the cloud project, properties:

Cloud-Service-Role-Properties

Click settings, add setting “Microsoft.ServiceBus.ConnectionString” with the value you used previously.

Role-Settings

Lastly:

UploadBlob

    public void UploadBlob(string path, ImageData image)
    {
        var connectionString = CloudConfigurationManager.GetSetting("Microsoft.Storage.ConnectionString");
        var account = CloudStorageAccount.Parse(connectionString);
        var cloudBlobClient = account.CreateCloudBlobClient();
        var cloudBlobContainer = cloudBlobClient.GetContainerReference(path);

        cloudBlobContainer.CreateIfNotExists();

        var blockref = image.FormattedName ?? Guid.NewGuid().ToString();
        var blob = cloudBlobContainer.GetBlockBlobReference(blockref);

        if (!blob.Exists())
            blob.UploadFromStream(new MemoryStream(image.Data));
    }

Pretty self explanatory, isn’t it? Get a reference to an area of blob storage within a container associated with an account, and stream some data to it if it doesn’t already exist (you might actually want to overwrite it so could remove that check). Bosch. Done. Handsome.

Notice we’re using the FormattedName property on ImageData to get a blob name which includes the requested dimensions; this will be used in the next article where we create the image proxy.

This means that for a request like:

http://127.0.0.1/api/Image/Resize?height=600&width=400&source=image1.jpg

The formatted name will be set to:

600_400-image1.jpg

You shouldn’t get any compile errors here but you’ll need to add in the setting for your storage account (“Microsoft.Storage.ConnectionString”).

Kick it off

To run that you’ll need VS to be running as admin (right click VS, run as admin):

run-as-admin

After you’ve got it running, fire off a request within the resizing web api (if it’s not the same solution/cloud service) for something like:

http://127.0.0.1/api/Image/Resize?height=600&width=400&source=image1.jpg

Resulting in:
Resized-Image

Then open up your Azure storage explorer to see something similar to the below within the “resized” blob container:

Resized-Blob

What happened?

  1. The ImageController on your Resizer Web API web role did the hard work and popped a message on an Azure Service Bus queue containing the image data
  2. The new Uploader worker role is subscribed to the same Azure Service Bus queue
    1. it picks up the message
    2. pulls out the image data
    3. generates an image name based on the image dimensions and origin
    4. streams the image data into a blob block with the generated name

Cool, huh?

The code for this series is up on GitHub

Next up

One more web role to act as a proxy for checking blob storage first before firing off the resize request. Another easy one. Azure is easy. Everyone should be doing this. You should wait and see what else I’ll write about Azure.. it’s awesome.. and easy..!