Strings, Bows, (A)Quiver

Framework Training

This past week I’ve been lucky enough to try my hand at something new to me. Throughout my career my work has consisted almost entirely of writing code, designing solutions, and managing teams (or teams of teams).

Last week I took a small group of techies through a 4 day long Introduction To C# course, covering some basics – such as types and members – through some pretty advanced stuff (in my opinon) – such as multicast delegates and anonymous & lamba methods (consider that the class had not coded in C# before the Monday, and by Tuesday afternoon we were covering pointers to multiple, potentially anonymous, functions).

I also had an extra one on one session on the Friday to help one of the guys from the 4 day course get a bit of an ASP.Net knowledge upgrade in order to get through a SiteCore course and exam.

Do what now?

I’d not done anything similar to this previously, so was a little nervous – not much though, and not even slightly after the first day was over. Public speaking is something that you can easily overcome; I used to be terrified but now you can’t shut me up, even in front of a hundred techies…

Challenge Accepted

The weekend prior to the course starting I found myself painstakingly researching things that have, for almost a decade, been things I “just knew”. I picked up .Net by joining a company that was using it (VB.Net at the time) and staying there for over 5 years. I didn’t take any “Intro” courses as I didn’t think I needed to; I understood the existing code just fine and could develop stuff that seemed to suit the current paradigm, so I must be fine.. right?

The weekend of research tested my exam cram ability (being able to absorb a huge amount of info and process it in a short amount of time!) as I finally learned things that I’ve been just doing for over 8 years. Turns out a lot of stuff I could have done a lot better if I had the grounding that the course attendees, and now I, have.

It. Never. Ends.

Each evening I’d get home, mentally exhausted from trying to pull together the extremely comprehensive information on the slides with both my experiences and my research, trying to end up with cohesive information which the class would understand and be able to use. That was one of the hard parts.

Every evening I’d have to work through what I had planned to cover the next day and if there was anything I was even slightly unsure of I’d hit the googles and stackoverflows until I had enough information to fully comprehend that point in such a way I could explain it to others – potentially from several perspectives, and with pertinent examples, including coming up with a few quick “watch me code” lines.

Never. Ends.

Once I’d got all of the technical info settled in my noggin, then came the real challenge; trying to make this expansive course relevant to each attendee. A couple of them were learning C# in order to learn ASP.Net so that they can move into .Net web development, whilst one was mainly learning to support and develop winforms apps. Also each one was absorbing and processing the information at a different speed, and one even had to leave for one day as he needed to support a production issue, then returned a day later! How do you deal with that gap in someone’s knowledge and make it all relevant without duplicating sections for the others?

Not. Ever.

I’m booked in to lead an Advanced C# course next month and an ASP.Net one the month after, plus I’m looking at the MVC course at some point. All whilst working for a startup at the same time (more on that soon)! 2014 is going to be EPIC. It already is, actually..

Summary

I’m sure others could, and (since I’ve heard about people who do this) would, blag it if there was something they didn’t know, since – hey, these attendees aren’t going to be able to correct me are they?! This is an Intro course!

That’s obviously lame, but for a reason in addition to the one you would imagine; you’re cheating yourself if you do that. I have learned SO MUCH more information to surround my existing experience that I can frame all coding decisions that much better. Forget committing Design Patterns to memory if you don’t know what an event actually is. Sure, it’s basic, but it’s also fundamental.

Teaching is hard.

I like it.

You might too.

Generate a Blob Storage Web Site using RazorEngine

Last episode I introduced the concept of utilising RazorEngine and RazorMachine to generate html files from cshtml Razor view files and json data files, without needing a hosted ASP.Net MVC website.

We ended up with a teeny ikkle console app which could reference a few directories and spew out some resulting html.

This post will build on that concept and use Azure Blob Storage with a worker role.

File System

We already have a basic RazorEngine implementation via the RenderHtmlPage class, and that class uses some basic dependency injection/poor man’s IoC to abstract the functionality to pass in the Razor view, the json data, and the output location.

IContentRepository

The preview implementation of the IContentRepository interface merely read from the filesystem:

[csharp]namespace CreateFlatFileWebsiteFromRazor
{
internal class FileSystemContentRepository : IContentRepository
{
private readonly string _rootDirectory;
private const string Extension = ".cshtml";

public FileSystemContentRepository(string rootDirectory)
{
_rootDirectory = rootDirectory;
}

public string GetContent(string id)
{
var result =
File.ReadAllText(string.Format("{0}/{1}{2}", _rootDirectory, id, Extension));
return result;
}
}
}
[/csharp]

IDataRepository

A similar story for the IDataRepository file system implementation:

[csharp]namespace CreateFlatFileWebsiteFromRazor
{
internal class FileSystemDataRepository : IDataRepository
{
private readonly string _rootDirectory;
private const string Extension = ".json";

public FileSystemDataRepository(string rootDirectory)
{
_rootDirectory = rootDirectory;
}

public string GetData(string id)
{
var results =
File.ReadAllText(string.Format("{0}/{1}{2}", _rootDirectory, id, Extension));
return results;
}
}
}
[/csharp]

IUploader

Likewise for the file system implemention of IUploader:

[csharp]namespace CreateFlatFileWebsiteFromRazor
{
internal class FileSystemUploader : IUploader
{
private readonly string _rootDirectory;
private const string Extension = ".html";

public FileSystemUploader(string rootDirectory)
{
_rootDirectory = rootDirectory;
}

public void SaveContentToLocation(string content, string location)
{
File.WriteAllText(
string.Format("{0}/{1}{2}", _rootDirectory, location, Extension), content);
}
}
}
[/csharp]

All pretty simple stuff.

Blob Storage

All I’m doing here is changing those implementations to use blob storage instead. In order to do this it’s worth having a class to wrap up the common functions such as getting references to your storage account. I’ve given mine the ingenious title of BlobUtil:

[csharp]class BlobUtil
{
public BlobUtil(string cloudConnectionString)
{
_cloudConnectionString = cloudConnectionString;
}

private readonly string _cloudConnectionString;

public void SaveToLocation(string content, string path, string filename)
{
var cloudBlobContainer = GetCloudBlobContainer(path);
var blob = cloudBlobContainer.GetBlockBlobReference(filename);
blob.Properties.ContentType = "text/html";

using (var ms = new MemoryStream(Encoding.UTF8.GetBytes(content)))
{
blob.UploadFromStream(ms);
}
}

public string ReadFromLocation(string path, string filename)
{
var blob = GetBlobReference(path, filename);

string text;
using (var memoryStream = new MemoryStream())
{
blob.DownloadToStream(memoryStream);
text = Encoding.UTF8.GetString(memoryStream.ToArray());
}
return text;
}

private CloudBlockBlob GetBlobReference(string path, string filename)
{
var cloudBlobContainer = GetCloudBlobContainer(path);
var blob = cloudBlobContainer.GetBlockBlobReference(filename);
return blob;
}

private CloudBlobContainer GetCloudBlobContainer(string path){
var account = CloudStorageAccount.Parse(_cloudConnectionString);
var cloudBlobClient = account.CreateCloudBlobClient();
var cloudBlobContainer = cloudBlobClient.GetContainerReference(path);
return cloudBlobContainer;
}
}
[/csharp]

This means that the blob implementations can be just as simple.

IContentRepository – Blob Style

Just connect to the configured storage account, and read form the specified location to get the Razor view:

[csharp]class BlobStorageContentRepository : IContentRepository
{
private readonly BlobUtil _blobUtil;
private readonly string _contentRoot;

public BlobStorageContentRepository(string connectionString, string contentRoot)
{
_blobUtil = new BlobUtil(connectionString);
_contentRoot = contentRoot;
}

public string GetContent(string id)
{
return _blobUtil.ReadFromLocation(_contentRoot, id + ".cshtml");
}
}
[/csharp]

IDataRepository – Blob style

Pretty much the same as above, except with a different “file” extension. Blobs don’t need file extensions, but I’m just reusing the same files from before.

[csharp]public class BlobStorageDataRespository : IDataRepository
{
private readonly BlobUtil _blobUtil;
private readonly string _dataRoot;

public BlobStorageDataRespository(string connectionString, string dataRoot)
{
_blobUtil = new BlobUtil(connectionString);
_dataRoot = dataRoot;
}

public string GetData(string id)
{
return _blobUtil.ReadFromLocation(_dataRoot, id + ".json");
}
}
[/csharp]

IUploader – Blob style

The equivalent for saving it is similar:

[csharp]class BlobStorageUploader : IUploader
{
private readonly BlobUtil _blobUtil;
private readonly string _outputRoot;

public BlobStorageUploader(string cloudConnectionString , string outputRoot)
{
_blobUtil = new BlobUtil(cloudConnectionString);
_outputRoot = outputRoot;
}
public void SaveContentToLocation(string content, string location)
{
_blobUtil.SaveToLocation(content, _outputRoot, location + ".html");
}
}
[/csharp]

Worker Role

And tying this together is a basic worker role which looks all but identical to the console app:

[csharp]public override void Run()
{
var cloudConnectionString =
CloudConfigurationManager.GetSetting("Microsoft.Storage.ConnectionString");

IContentRepository content =
new BlobStorageContentRepository(cloudConnectionString, "content");

IDataRepository data =
new BlobStorageDataRespository(cloudConnectionString, "data");

IUploader uploader =
new BlobStorageUploader(cloudConnectionString, "output");

var productIds = new[] { "1", "2", "3", "4", "5" };
var renderer = new RenderHtmlPage(content, data);

foreach (var productId in productIds)
{
var result = renderer.BuildContentResult("product", productId);
uploader.SaveContentToLocation(result, productId);
}
}
[/csharp]

The Point?

By setting the output container to be public, the html files can be browsed to directly; we’ve just created an auto-generated flat file website. You could have the repository implementations access the local file system and the console app access blob storage; generate the html locally but store it remotely where it can be served from directly!

Given that we’ve already created the RazorEngine logic, the implementations of content location are bound to be simple. Swapping file system for blob storage is a snap. Check out the example code over on github

Next up

There’s a few more stages in this master plan, and following those I’ll swap some stuff out to extend this some more.

Generate a Flat File Web Site using RazorEngine and RazorMachine

Razor and ASP.Net

As a normal person you’d probably be happy with how Razor template files are used within MVC; there’s a nice convention for where they live – they’ll be in a Views folder within your project most likely – and you refer to them either by name or sometimes just by convention – what’s that? You have an ActionResult method called “Index”? I’ll go fetch the “Index” view from the folders I normally expect the cshtml files to live in for ya then.

The way this works is fantastic; development can steam ahead without the pain and confusion of all of the possible ways you could do it wrong when choosing webforms and .aspx files.

Of course, the MS implementation of an MVC framework in itself is a wonderful thing; all but enforcing the separation of concerns that is just so easy to ignore in webforms.

Razor outside of ASP.Net

But what about when you want to dynamically generate html without a process being hosted as a website? One big use case for this is email generation; sure, you could host an MVC web API and have the content generation process constantly call it, but that seems a little inefficient.

RazorEngine and RazorMachine

There are a few solutions to this; you can actually hand roll your own (I might get onto that in a future post) or you can try out some reasonably well known open source solutions like RazorEngine:

A templating engine built on Microsoft’s Razor parsing engine, RazorEngine allows you to use Razor syntax to build dynamic templates:

[csharp]string template = "Hello @Model.Name, welcome to RazorEngine!";
string result = Razor.Parse(template, new { Name = "World" });
[/csharp]

and RazorMachine:

RazorMachine is a robust and easy to use .Net Razor v2/v3 template engine. The master branch uses Razor v3. This implementation supports layouts (masterpages) and a _viewStart construct, just like MVC does support these features. The RazorEngine works independently from MVC. It only needs the System.Web.Razor reference. It almost works exacly like Asp.Net MVC

[csharp]var rm = new RazorMachine();
var result =
rm.Execute("Hello @Model.FirstName @Model.LastName", new {FirstName="John", LastName="Smith"});
[/csharp]

There’s a short stackoverflow answer comparing them (and RazorTemplates, another similar OSS solution) too.

Getting stuck in

Create a new project and use some nuget awesomeness

[powershell]Install-Package razormachine[/powershell]

Then, if you don’t already, add references to

[csharp]system.web.helpers // for json.decode
microsoft.csharp // for dynamic types
[/csharp]

If you want to debug this functionality via a console app running from VisualStudio, you may need to uncheck “enable visual studio hosting process” in Project -> Properties -> Debug

If you want to run this outside of Visual Studio, you can just run the compiled exe (bin/debug) as admin.

If you’re using a test runner then you might be fine as is. I can’t actually remember the issue I was having as I now can’t recreate it, but I think it might have been around using dynamic models and Json decoding.

RazorEngine

This is the core bit of functionality for a basic use case for RazorEngine:

[csharp]var model = Json.Decode("{\"Description\":\"Hello World\"}");
var template = "<div class=\"helloworld\">@Model.Description</div>";
const string layout = "<html><body>@RenderBody()</body></html>";

template = string.Format("{0}{1}", "@{Layout=\"_layout\";}", template);

using (var service = new TemplateService())
{
service.GetTemplate(layout, null, "_layout");
service.GetTemplate(template, model, "template");

var result = service.Parse(template, model, null, "page");

Console.Write(result);
Console.ReadKey();
}
[/csharp]

Your output should be:

[csharp]<html><body><div class="helloworld">Hello World</div></body></html>
[/csharp]

Pretty easy, right?

RazorMachine

Here’s the equivalent in RazorMachine

[csharp]var model = Json.Decode("{\"Description\":\"Hello World\"}");
var template = "<div class=\"helloworld\">@Model.Description</div>";
const string layout = "<html><body>@RenderBody()</body></html>";

var rm = new RazorMachine();
rm.RegisterTemplate("~/shared/_layout.cshtml", layout);

var renderedContent =
rm.ExecuteContent(string.Format("{0}{1}", "@{Layout=\"_layout\";}", template), model);
var result = renderedContent.Result;

Console.Write(result);
Console.ReadKey();
[/csharp]

Again, same output:

[html]<html><body><div class="helloworld">Hello World</div></body></html>
[/html]

Notice that in both of them you have to lie about there being a layout file existing somewhere; in RazorEngine you give it a name:

[csharp]template = string.Format("{0}{1}", "@{Layout=\"_layout\";}", template);[/csharp]

then refer to that name when adding the template:

[csharp]service.GetTemplate(layout, null, "_layout");[/csharp]

In RazorMachine you register the template as a dummy virtual file:

[csharp]rm.RegisterTemplate("~/shared/_layout.cshtml", layout);[/csharp]

then refer back to it as you would normally do within ASP.Net MVC when executing the content:

[csharp]var renderedContent =
rm.ExecuteContent(string.Format("{0}{1}", "@{Layout=\"_layout\";}", template), model);[/csharp]

Differences

I’ve found it easier to process sub templates (such as @Include) within RazorEngine, as I just recursively scan a file for that keyword and add the corresponding template to the service, e.g. look at the ProcessContent and ProcessSubContent methods below:

[csharp]public class RenderHtmlPage
{
private readonly IContentRepository _contentRepository;
private readonly IDataRepository _dataRepository;

public RenderHtmlPage(IContentRepository contentRepository,
IDataRepository dataRepository)
{
_contentRepository = contentRepository;
_dataRepository = dataRepository;
}

public string BuildContentResult(string page, string id)
{
using (var service = new TemplateService())
{
// get the top level razor template, e.g. "product"
// equivalent of "product.cshtml"
var content = GetContent(page);
var data = GetData(id);

ProcessContent(content, service, data);
var result = service.Parse(content, data, null, page);

return result;
}
}

private void ProcessContent(string content,
TemplateService service,
dynamic model)
{
// does the string passed in reference a Layout at the start?
const string layoutPattern = @"@\{Layout = ""([a-zA-Z]*)"";\}";

// does the string passed in reference an Include anywhere?
const string includePattern = @"@Include\(""([a-zA-Z]*)""\)";

// recursively process the Layout
foreach (Match match in Regex.Matches(content, layoutPattern,
RegexOptions.IgnoreCase))
{
ProcessSubContent(service, match, model);
}

// recursively process the @Includes
foreach (Match match in Regex.Matches(content, includePattern,
RegexOptions.IgnoreCase))
{
ProcessSubContent(service, match, model);
}
}

private void ProcessSubContent(TemplateService service,
Match match,
dynamic model)
{
var subName = match.Groups[1].Value; // got an include/layout match?
var subContent = GetContent(subName); // go get that template then
ProcessContent(subContent, service, model); // recursively process it

service.GetTemplate(subContent, model, subName); // add it to the service
}

private string GetContent(string templateToLoad)
{
// hit the filesystem, db, API, etc to retrieve the razor template
return _contentRepository.GetContent(templateToLoad);
}

private dynamic GetData(string dataToLoad)
{
// hit the filesystem, db, API, etc to return some Json data as the model
return Json.Decode(_dataRepository.GetData(dataToLoad));
}
}
[/csharp]

Why is this useful?

I’m not going to go into the details of either RazorMachine or RazorEngine; there’s plenty of documentation up on their respective websites already. I’ve used @Includes in the examples above due to its simplicity; the libraries have differing support for things like @Html.Partial and also can be extended.

Unfortunately, the html helpers (like @Html.Partial) need to have an HttpContext and run inside of ASP.Net MVC; which is what I’m trying to avoid for now.

If you pull down my initial teeny solution from github and look at the tests you’ll notice the content of the template, layout, and model are either strings or coming from the filesystem; not related to the structure of the project or files in the project or anything like that.

This means we can deploy a rendering process that returns rendered html based on strings being passed to it. Let’s play with this concept a bit more.

Flat File Web Page Generation

Say you wanted to “host” a website directly within a CDN/cache, thus avoiding the hosting via the normal route of servers and related infrastructure. Sure, writing flat html in a text editor is a solution, but what if you wanted to still be able to structure your pages into common modules, write C# code to manage the logic to dynamically combine them, and use Razor syntax and views for defining the front end?

This next section plays on this concept a bit more; we’ll write a small app that accesses a couple of directories – one for Razor files, one for data files – and generates a flat website into a third directory.

I will then expand on this concept over a series of posts, to make something more realistic and potentially useful.

Command Line & FileSystem FTW

I’ve created another repo up on github for this section, but cutting to the chase – here is the guts of demo console app:

[csharp]const string workingRoot = "../../files";
IContentRepository content =
new FileSystemContentRepository(workingRoot + "/content");

IDataRepository data =
new FileSystemDataRepository(workingRoot + "/data");

IUploader uploader =
new FileSystemUploader(workingRoot + "/output");

var productIds = new[] {"1", "2", "3", "4", "5"};
var renderer = new RenderHtmlPage(content, data);

foreach (var productId in productIds)
{
var result = renderer.BuildContentResult("product", productId);
uploader.SaveContentToLocation(result, productId);
}
[/csharp]

The various FileSystemXX implementations either just read or write files from/to the file system. Natch.

So what we’ve got here is an implementation of the RazorEngine methods I pasted in above wrapped in a RenderHtmlPage class, being called for a number of “productIds”; these happen to exist as json files on disc, e.g. “1.json”.

Each file is being combined with whatever Razor templates are listed in the product cshtml file and its referenced @Includes. The resulting html is then saved back to the file system.

So with these views in files/content:
razorengine-flat-file-website-views

And these json files in files/data:
razorengine-flat-file-website-jsondata

We get these html files generated in files/output:
razorengine-flat-file-website-htmloutput

Hopefully you can see where this is leading; we can keep Views in one place, get the model data from somewhere else, and have the extremely generic rendering logic in another place.

The Theory

With this initial version we could take an existing ASP.Net MVC website (assuming it didn’t use any html helpers in the views..) and process it offline with a known dataset to create a readonly version of the website, ready to serve from a filesystem.

Next Up

I’ll take this concept and run with it across various implementations, gradually ending up on something that might even be useful!

Programming-MuddyFunster!

I recently saw the website programming-m*therf*cker.com (not linking to it as people’s nsfw alerts might go mental with this post) and replied with the comment below. Have a look at the site and the values it extols, then let me know if I’m being fair!

Man, that site annoys me. So long as he’s talking about programming as a hobby, then that’s fine. But all of those things he’s against are nothing to do with managers but to do with devs making high quality software and being professional.

The site also makes me happy, in that if there are people with that same attitude out there putting software into production systems then I’ll always be able to find gainful employment fixing their stuff…