Backup Backup Backup

Whilst out at a bar in Camden recently a friend started hunting around our area with an increasingly panic stricken look on his face. Turns out that his personal laptop had been taken from under our seats. Six years of his personal dev work lost.

The response from every technical person he told? Should’ve backed up. Also, you’ve got Prey on there, right?.. no?.. Ah..

Not exactly the most compassionate response, but for those of us with a half decent backup solution all you need worry about is getting the insurance paid out on the hardware itself.

My backup solution is pretty minimal; the only things I care about are photos and videos, documents (work and home), and coding fun.

First up – Cloud

Photos

My photos are usually taken on my Samsung Galaxy S3 so exist in several places almost by default. The SG3 is an android device, so my photos are automatically synced up to my google drive using Google Plus Instant Upload.

I have a dropbox account and the android dropbox app so they’re also being synced to dropbox; I have dropbox on my laptops at home and on my work PC, so the photos are downloaded onto those devices too.

I have the Jungledisk agent on all of my PCs which also backup those photos to my jungledisk account. It’s going to be pretty tricky to lose them all.

Documents

Documents are handled by both dropbox and Windows Live Mesh; as soon as I close a document on my office PC it’s synced via Live Mesh to SkyDrive and then downloaded again to my home PC.

Again, this is also backed up to Jungledisk as separate PC backups.

Coding fun

This also uses Windows Live Mesh, and Dropbox, and Jungledisk, but the ones I really like are also pushed up to github.

Pricey, right?

The cost of this security? Not much at all.

  • Live Mesh comes with Windows 7.

  • Dropbox is free and if you by a SG3 you get 50GB free for a year, plus 500MB free for every 500MB of photos you backup to dropbox.

  • Jungledisk costs me around £10 a month, which given that I also use it to automatically back up everything in my and my wife’s laptops’ “My Documents” folders to a secure cloud based solution, including all iTunes, iPhoto, etc folders, I think that’s money well spent; obviously some of that is also on iCloud too!

Physical? Sure!

I’m not relying on internet access to get my files back either. I have both local network and USB attached backup solutions.

External HDD

Western Digital My Passport

This tiny 500GB Western Digital Passport attaches to whichever machine I’m working on at the time and constantly backs up my entire D drive (non-OS) securely and with rollback capabilities.

NAS

LaCie LaCinema

I also have everything in key folders on each laptop within my home network being copied over to a 1TB LaCie LaCinema HD HDMI Media Centre/NAS, which is an amazing bit of kit in its own right. You can now get a 2TB version for about £160.

It Works For Me ©

I’m not saying this is what everyone should do, as requirements will obviously differ. But this works for me, and the last time one of my laptops crashed and burned I was quite happy at the chance to get a shiny new one, not having lost any important data at all.

Applications

No need to back up any applications since it’s so easy to rebuild an entire Windows system using things like Chocolatey, WebPi, and Ninite.

Headless VirtualBox

VirtualBox

I tend to create a lot of proofs of concept, be it applications or servers, and this process will usually start with spinning up a VM. For this I prefer to use Oracle’s VirtualBox mainly because it’s free and easy to use.

However, using VirtualBox each time I want to start and stop the VM isn’t great as it will open up the VM in a new window and you need to have both the VM window and VirtualBox running on your machine at the same time.

Luckily you can use the “start” command to run VirtualBox’s VBoxHeadless application to boot up a VM without starting VirtualBox itself and without having a window open, other than the command line.

start "Build Server VM" /D"C:\Program Files\Oracle\VirtualBox\" /MIN VBoxHeadless.exe --startvm "Server 2008 32"

Just make sure the name of the VM at the end of the command is the same as that within VirtualBox:

VirtualBox dashboard

You can then access this via RDP (if your VM is windows) or SSH (if it’s Linux), as can anyone else on your network, should you have configured its network settings correctly.

WebForms ScriptManager Vs MVC – FIGHT!

If you’ve tried to squeeze MVC into a WebForms project which uses ScriptManager elements for AJAX functionality, be sure to add some hardcore IgnoreRoute entries in your route registration section.

If you don’t then you’ll find the calls to your asmx webservice that ScriptManager creates will receive 404 errors looking for asmx/js or asmx/jsdebug that contain an HTTPException which looks like:

The controller for path blah.asmx/js was not found or does not implement IController

or if you’re in debug mode

The controller for path blah.asmx/jsdebug was not found or does not implement IController

This basically means that the pattern {folder}/{file}.asmx/{something} isn’t matching a route. Since it shouldn’t match one then you need to make sure you add in an exception.

Ignore a specific file type

This one didn’t actually work for me as expected, but is worth listing here:

routes.IgnoreRoute("{resource}.asmx/{*pathInfo}");

Ignore an entire folder

This brute force attack worked for me:

routes.IgnoreRoute("{folder}/{*pathInfo}", new { folder = "WebServices" });

Strangeness

I didn’t need to add in the IgnoreRoute on one IIS7 instance but did on another IIS7 server. Not sure why, probably due to HTTPHandler configuration within IIS itself?

Creating a SublimeText plugin to publish markdown to WordPress

I’ve been using SublimeText for a while now as a no-frills text editor; it has a nice Zen mode which hides all of the tabs and menus and pushes it to full screen which I’ve found perfect for taking notes during seminars:

SublimeText2 Zen mode

I have been trying to take notes using the markdown sytnax, but have found the process between taking the notes in markdown, converting them to HTML, and publishing the HTML to a blog a bit of a pain.

Given that SublimeText supports extension via plugin scripts written in Python, I’ve knocked together an extension which will

  1. convert the markdown to html
  2. push the html directly to wordpress

Before I get into the details of the script, maybe a little background would be useful:

What’s Markdown?

A lightweight markup language for writing human-readable content which can be compiled into HTML. Being able to write a lovely HTML blog post in a very basic text editor instead of the bloated Windows Live Writer is extremely refreshing.

For example, to make a line a <h1> header, prefix it with a #. For <h2>, use ##. For <h3> ###. Easy.

Want a link? Try surrounding it with square brackets and putting the url just after it in round brackets: [Something like this](which links here). It’s a really easy “language” and actually looks decent enough to just read on its own.

To include an image you just need to write ![alt text](image url)

A syntax reference page can be found here.

One problem I do have is the basic support for code highlighting within markdown is poor, obviously, since HTML only has a single <code> block. The Markdown generated HTML conflicts with my wordpress Syntaxhighlighter plugin, hence the ugly clode blocks in this post.

There is support for “fenced-code-blocks” wihtin many implementations of markdown, and with the library that I’m also using, but this is still not playing nicely with Syntaxhighlighter. I’m sure I’ll figure something out eventually – please bear with me until then.

I use the fantastic python-markdown2 library to convert markdown to HTML on the fly; I just had to copy the lib/markdown2.py file from the github repo into the sublimetext “plugins” directory – I created a Markdown subdirectory for it – and reference it from my script. i.e.,

post_content = str(markdown2.markdown(post_content,extras=["code-friendly"]))

What’s Python?

Python

A functional scripting language. I found it really quite tricky to work with and had to install PythonShell as a powershell-ish window for testing out python commands before putting them to work in the plugin script. What’s not great fun is that SublimeText2 installs a different version of Python to PythonShell to version 1 of SublimeText to any version you explicitly install yourself.

Some commands and syntax will work in one version of python and not others

One great example is setting up a proxy; the syntax for this is completely different between versions of python and whatever example code you’re reading might be using the syntax that your version of python doesn’t support.

Working with a functional language took a lot of getting used to; I’d want to return an object from a method but had to instead return a comma delimited list of base types. It’s all a little odd to me, and my messy code shows that!

What’s WordPress?

Wordpress

A popular free blog engine that’s used as a CMS for some small businesses. It has an XML RPC API which conforms to the MetaWeblog spec, and you can really dig into this by looking through the php files that make up a WordPress installation.

Since I’m using Amazon EC2 as my wordpress host I can use Kitty to SSH in and browse to /var/www/html/xmlrpc.php to see the metaweblog service api itself. That file references wp-includes/class-wp-xmlrpc-server.php which contains all of the underlying functionality for the api implementation. This is great reference material to make sure you’re passing the correct values of the correct types to the correct endpoints.

Details

The sublimemarkpress plugin is intended to:

  1. check the contents of a config file for your blog access details
  2. scan the contents of the active window to get the posts details, such as post ID (to make it an Update instead of a Create), tags, and status
  3. convert the text from Markdown to HTML
  4. Push the HTML to the metaweblog endpoint with the correct blog/post details

1. Getting the Blog setup details from a config file

The plugin relies on a settings file called “sublimemarkpress.sublime-settings” using the structure:

{
    "xmlrpcurl": <URL to xml rpc endpoint>,
    "username": <username>,
    "password": <password>
}

to read it:

s = sublime.load_settings("sublimemarkpress.sublime-settings")
mbURL = s.get("xmlrpcurl")
mbUsername = s.get("username")
mbPassword = s.get("password")

2. Get the text and strip out the blog post data

The plugin expects the top of your text file/active window to have optional tags to define blog post details:

<!-- 
#post_id:<id of existing post - optional>
#tags:<comma delimited list of post tags - optional>
#status:<draft or publish - optional>
-->

To get the entire contents of the active window:

all_lines_in_page = self.view.lines(sublime.Region(0, self.view.size()))

Then to extract the header details:

post_id, tags, status, has_header_content = self.GetHeaderContent(all_lines_in_page, header_lines)

where GetHeaderContent is the hack-y:

def GetHeaderContent(self, all_lines_in_page, header_lines):
    page_info = {"has_header_content":False,"post_id":None, "tags":"", "status":""}

    if self.view.substr(all_lines_in_page[0]).startswith("<!--"):
        page_info["has_header_content"] = True
        self.MoveCurrentLineToHeader(header_lines, all_lines_in_page)

        # post_id
        if self.view.substr(all_lines_in_page[0]).startswith("#post_id"):
            page_info["post_id"] = self.view.substr(all_lines_in_page[0]).split(":")[1]
            self.MoveCurrentLineToHeader(header_lines, all_lines_in_page)

        #post tags
        if self.view.substr(all_lines_in_page[0]).startswith("#tags"):
            page_info["tags"] = self.view.substr(all_lines_in_page[0]).split(":")[1]
            self.MoveCurrentLineToHeader(header_lines, all_lines_in_page)

        #post status
        if self.view.substr(all_lines_in_page[0]).startswith("#status"):
            page_info["status"] = self.view.substr(all_lines_in_page[0]).split(":")[1]
            self.MoveCurrentLineToHeader(header_lines, all_lines_in_page)

        self.MoveCurrentLineToHeader(header_lines, all_lines_in_page) # removes the closing comment tag
    return page_info["post_id"],page_info["tags"],page_info["status"],page_info["has_header_content"]

def MoveCurrentLineToHeader(self, header_lines, all_lines_in_page):
        header_lines.insert(len(header_lines),all_lines_in_page[0])
        all_lines_in_page.remove(all_lines_in_page[0])

3. Convert the rest from Markdown to HTML

As mentioned earlier, thanks to the great python-markdown2 library, this is simply a case of calling the “markdown” method, passing in the content to process:

post_content = str(markdown2.markdown(post_content,extras=["code-friendly"]))

I actually do a test to see if the markdown library can be imported first, then if it fails I don’t even try to convert from markdown to HTML:

can_markdown = False
try: 
    import markdown2 # markdown
    can_markdown = True
except ImportError:
    can_markdown = False

4. Post to metaweblog api

Build up the request using the blog details, post details, and HTML content:

content = self.BuildPostContent(self.view, {"content": post_content, "title": title, "tags": tags, "status": status})

def BuildPostContent(self, view, page_data):        
    return {"description": page_data["content"], "post_content": page_data["content"], "title": page_data["title"], "mt_keywords": page_data["tags"], "post_status": page_data["status"]}

Then submit it to the api:

proxy = xmlrpclib.ServerProxy(mbURL)

if post_id == None:
    post_id = proxy.metaWeblog.newPost(blog_id, mbUsername, mbPassword, content)
else:
    proxy.metaWeblog.editPost(post_id, mbUsername, mbPassword, content)

Extra bits and making it all work

To execute a command from a plugin within SublimeText, firstly you need to import the sublimetext library at the top:

import sublime, sublime_plugin # sublime

then name your class “something<command>” and have it take the “sublime_plugin.TextCommand” parameter:

class PublishCommand(sublime_plugin.TextCommand):

Then to run it you need to hit ctrl+’ to bring up the command window and type:

view.run_command("<name of the class minus the 'Command' suffix>")

e.g. for my plugin:

view.run_command("publish")

You can try this out with loads of the plugins included with a SublimeText install

That’s about it

All of this is in a small github repo. I’d appreciate it if you want to fork the repo and help show me how python should actually be structured. And written.

Known Issues

Categories

These need to be requested, looped through and matched against those associated with a post, added if those specified don’t exist, and the IDs associated with the post. Couldn’t be arsed.

As such, when you post to your blog you’ll see the entry is “uncategorised” and you’ll have to manually edit this.

Images

Since I’m just fooling around with a text editor, uploading images is still a little out of scope.

Random reference material

  • pyblog – for helping to work out the guts of how python interacts with the metaweblog api
  • I used Fiddler & Windows Live Writer for investigating traffic and finding out what the actual parameter being used to specify “tags” is within the metaweblog api: mt_keywords

CSS woes in IE8

Having trouble getting clever CSS to run in IE8? Trying to use webfonts/@font-face and only some of them are being applied? Make sure you don’t have css psuedo-selectors in there, else IE8 dies.

i.e., if you have a list of classes/css selectors having a style applied to them, such as:
[css].modal h2, h1:not(.title), .prod_message .figure h2 {
color:orange;
}[/css]
then the entire section is ignored in IE8 due to the
[css]h1:not(.title)[/css]
entry.

I’ve not been able to track this one down for MONTHS and suddenly I get a new brand of coffee and it all makes sense. Come back to us, IE8, you are once again welcomed.