Building your first Botframework based Cortana Skill

Hi. I'm Cortana.

At //BUILD 2017 Microsoft announced support for Cortana Skills and connecting a Cortana Skill into a Bot Framework chatbot; given the number of chatbots out there using Microsoft Bot Framework, this is an extremely exciting move.

In this article I’ll show you how to create your first Cortana Skill from a Bot Framework chatbot and make it talk!


If you’re not already familiar with Cortana, this is Microsoft’s “personal assistant” and is available on Windows 10 (version 1607 and above) and a couple of Windows phones (Lumia 950/950 XL), a standalone speaker – like an Amazon Echo – and a plethora of devices that can run the Cortana app, including iOS and Android and plenty of laptops.

Cortana all the things, Derrick.

You’re going to be seeing a lot more of this little box of tricks (“Bot” of tricks? Box of bots?.. hmm…), so you might as well get in on the act right now!

Bot Framework

I’m going to assume you have some knowledge of Microsoft’s Bot Framework for building multichannel chatbots. In case you’re not, then you should pop off now and familiarise yourself with these (but mainly just the first one):

Done that? Ok, great – now we can get started! I’m going to use the demo bot I created and have been using for most of my Bot Framework articles:

rposb demo bot - markdown

The Cortana channel

The first thing we’ll do is head over to the Bot Framework developer centre at Log in, and choose the bot you want to power up with Cortana awesomeness!

bot listing

Now scroll down to the list of available channels and you should find a shiny new one called Cortana:

channel listing

It’s the pretty blue circle, not the pretty blue envelope, pretty blue speech bubble, pretty blue square speech bubble smiley thing, or the other pretty blue things…

You’ll now be taken to the channel configuration screen; if you’ve set up a bot on many channels before, you probably know what to expect. Fortunately this is a very short setup screen, with only one section you really need to pay attention to:

Invocation name

This is what you’ll have to say in order to get Cortana to invoke your skill. Don’t make this something hard to say or hard to understand etc etc. I’ve gone for “demo bot”.

The rest of the form looks like this:

cortana channel setup

Notice “Request user profile data” at the end? This is kinda cool and I’ll get on to it in another article; it allows your bot to request permission to access certain information about the user, like when you do Google OAuth and see a prompt saying “this site is requesting access to: your email address”

Leave it blank for now, hit “Save” and it will appear in your list of connected channels for that bot:

connected channel listing

Become an American!

If you’re already an American, then by now you should have achieved the bare minimum necessary to have your Bot Framework chatbot accessible through Cortana.

If, like me, you’re not so fortunate, then you need to pretend a bit. Cortana is currently limited to the en-US locale, so you may need to configure your pc to be U.S. friendly.

Tap the Windows key and type “Language” to open “Region and Language settings“:

Region and language settings

From here you’ll need to change your Region to “United States”, tap “Add a language” and search for “en-us” to add “English (United States)”.

You’ll need to restart and let your machine download the various updates and language packs necessary.

Hey Cortana!

Ok, now you should have done the bare minimum necessary to connect your bot to Cortana (at least for you).

Make sure you’re logged in to your pc using the same account that you use for your Bot Framework/Cortana dashboard configuration, otherwise your Cortana app won’t be able to find your skill.

Give it a go! In my case (because the invocation name for my bot is “demo bot”) I just say “Hey Cortana; ask demo bot hi”. Yeah, I know the structure doesn’t make sense, but Cortana seems to need you to say “ask” otherwise it thinks you’re sending a message to someone if you say “tell”.

Anyway, if it’s working you should see something like this:


Since this is the first time we’ve attempted to talk to the bot, we’re also given the permission prompt:

Bot permission prompt

Since I didn’t request any extra information from the user, the permission is just for “your request”, i.e. “hi”. I’m ok with that, so I’ll tap “Agree”.

Now the request is passed to my bot, processed, and the response is sent back:


Wohoo! Now I’ll try one of the other commands my bot can handle – “markdown”:


Summary so far

Without much effort, we’ve taken an existing Bot Framework chatbot and exposed it via the Cortana channel. But it’s mute; no voice.

Before we make it talk, let’s investigate the Cortana dashboard a bit.

Cortana Dashboard

Go back to the Bot Framework dashboard and tap the “Manage in Cortana dash board” link to be taken to – surprisingly enough – the Cortana Dashboard:

Cortana dashboard

Deployment Ring

We already have some interesting info here; Deployed to self and Deploy to group. Right now, the only person who can use your skill is you; more specifically, the same Microsoft sign in account that your bot belongs to. In order to use your skill with Cortana, you must be logged in to Cortana using the same account.

If you want to share this skill with someone else, you can just tap “Deploy to group”, add in their Microsoft sign in email address, and you’ll be given a URL to send them which will give them access to your skill! Cool, huh?

Here I’m sharing the skill with .. um.. me!

Cortan deploy to group

… Thinky thinky thinky…
Cortana deploying to group

Ooh! I haz access too!
Cortana deployed group

Great for testing with a specific group of people.


Another point of interest is the “Debug” link; tap that and enable debug mode on the following page:

debug mode

If you talk to your bot now you’ll see an extra section in the Cortana response:

cortana haz debugz

This contains an awful lot of interesting info, such as the request containing whether the device has a screen:

    "entities": [
        "type": "Intent",
        "name": "None",
        "entities": []
        "type": "DeviceInfo",
        "supportsDisplay": "true"

That way, you can cater your bot response to be more audio based instead of visual, or vice versa.

What does your bot sound like?

Lastly, let’s make a couple of tiny changes to give our bot a voice. Right now you can talk to your bot, but you won’t hear anything in the response. Let’s fix that.

Open up your Bot Framework project and update to the latest version of Microsoft,Bot.Builder via Nuget:

 > Update-Package Microsoft.Bot.Builder

Now wherever you bot returns a reponse you can either:

  1. Use a new method SayAsync on the context:

    await context.SayAsync(text: "I'm text", speak: "I'm voice");
  2. If you usually create a reply from the incoming message activity using message.CreateReply then you can explicitly set the Speech property:

    var reply =  message.CreateReply();
    reply.Text = "This is text";
    reply.Speak = "This is speech";
  3. Or you could use this handy extension class:

    public static class MessageActivityExtension
        public static void Say(
                this IMessageActivity activity,
                string text,
                string speech = null, 
                string inputHint = InputHints.AcceptingInput)
            activity.Text = text;
            activity.Speak = speech ?? text;
            activity.InputHint = inputHint;

    which allows you to do something like:

    var reply =  message.CreateReply();
    reply.Say("This is text", "This is speech");

I’ll return to InputHints in a subsequent article

Go through your existing chatbot and add in a few Speech properties to the reponse, redeploy, and ask Cortana again; suddenly your bot will be able to speak! BOT EVOLUTION! Aw yeah!


In this article I’ve shown you how to connect your Bot Framework chatbot to the Cortana channel, how to set up your PC to set up Cortana and connect back to your bot, and how to give your bot a voice.

There’s still so much cool stuff to share, so stay tuned for more Cortana articles!

Involved in a startup? Read this!

Having been the VP of Engineering at a startup, I understand a lot of the challenges. The technical ones relating to the solution you think you need to build, more technical ones relating to the solutions the investors want you to build, the development process to best fit a rapidly changing product, team, requirements, and priorities, as well as managing the team through uncertain terrain.

They’re the fun ones. The easy ones! Especially given how talented my dev team was.

The founder had the difficult challenges; define a product that could be a success, iterate that idea based on extensive user testing, and most importantly, ensure there was funding.

Luckily, our founder was as talented at soliciting funds as we were at building epic tech!

If you are involved in a startup, perhaps Just Eat’s Accelerator programme can help with both types of challenge!

Continue reading

Receiving Images to Your Skype Botframework Bot (v2!)

If you’re getting a “403” HTTP error when attempting to receive an image sent to your Skype bot, and the previous use of message.ServiceUrl to create a ConnectorClient didn’t work, try this more verbose version which explicitly sets the authorization header:

byte[] data;

if (image.ContentUrl != null)
    using (var connectorClient 
        = new ConnectorClient(new Uri(message.ServiceUrl)))
        var token = 
            await (connectorClient.Credentials as MicrosoftAppCredentials)

        var uri = new Uri(image.ContentUrl);

        using (var httpClient = new HttpClient())
            if (uri.Host.EndsWith("") 
                && uri.Scheme == Uri.UriSchemeHttps)
                    .Authorization = 
                        new AuthenticationHeaderValue("Bearer", token);

                    .Add(new MediaTypeWithQualityHeaderValue("application/octet-stream"));

            // Get the image in a byte[] variable
            data = await httpClient.GetByteArrayAsync(uri);

Generating Image Hashtags using Microsoft’s Computer Vision API

Whatever your social media tool of choice is these days, it’s almost guaranteed to be filled with images and their associated hashtags #sorrynotsorry #lovelife #sunnyday

Sometimes coming up with those tags is more work than perfectly framing your latest #flatlay shot.

In the age of amazing image recognition tech, it must be possible to create something that can help us out and give us more time to move that light source around to cast the right shadow over your meal.

Turns out, it is possible! Yay! (of course..)

In this article I’ll show you how to automatically generate image hashtags via a chatbot using Microsoft’s Computer Vision API.

Continue reading

Sentiment Analysis using Microsoft’s Cognitive Services

Now that we are making more conversational interfaces thanks to technology like botframework, interaction with the user is no longer limited to a tap on a link or a button.

Having written language as the primary form of interaction with our systems gives significant difficulties in terms of intent understanding, but also gives great opportunities for further understanding of the user.

Intent understanding has already been tackled by the likes of LUIS; what about the user’s sentiment?

In this article I’m going to introduce Microsoft’s Text Analysis API and show you how to easily get sentiment analysis for a message coming in to your bot.

Continue reading

MVP led TechDays Online: Videos!

As part of Microsoft’s recent Tech Days Online, I was very pleased to be able to record a couple of short videos about botframework, LUIS, the QnA Maker, and how I have been working with JustEat to use these technologies in their Customer Help chatbot solution.

Unfortunately I wasn’t able to attend the live TechDays sessions, so instead of an hour or two of my dulcet tones you only have the pleasure of ten minutes; feel free to replay those minutes as many times as you like!

First up, a ten minute session on the JustEat Customer Care chatbot implementation:

Continue reading

5 Cool Chatbots: Feb 2017 Edition

ChristopherBot (Facebook)

Never forget your homework again

I have to include this bot first; it’s received a lot of press over the past few weeks, and rightly so. A great little concept from a 14 year old schoolboy who was forever forgetting about his homework. He created a Facebook messenger chatbot in Ruby and hosted on Heroku to help him (and you!) keep track of work that’s pending.


Read more about ChristopherBot on the BBC and try it out over at You can view the code (mainly Ruby) over on GitHub – annoyingly good code from someone so young! He puts me to shame..

Continue reading

Virtual Shop Assistant Chatbot with Amazing Image Recognition

There has been some significant progress in “deep learning”, AI, and image recognition over the past couple of years; Google, Microsoft, and Amazon each have their own service offering. But what is the service like? How useful is it?

Everyone’s having a go at making a chatbot this year (and if you’re not, perhaps you should contact me for consultancy or training!) – and although there are some great examples out there, I’ve not seen much in the e-commerce sector worth talking about.

In this article I’m going to show you a cool use case for an image recognition e-commerce chatbot via a couple of clever APIs wired together by botframework.

Continue reading

MVP led TechDays Online

MVP led TechDays Online

On February 20th to 22nd you’ll be able to join MVPs like me at the 3 day online conference UK Tech Days Online.

This is a chance for MVPs to talk about some of the cool new MS tech that’s out there, show you how to use it yourself, and give a few case studies. This year it’s all about AI, Bots, Data Science, and Azure OSS – awesome stuff!

Since I’m not able to attend this on the day of the event, my fellow Bot MVP, Gary Pretty, will be accompanied by the talented James Mann (no doubt a Bot MVP in the next round of awards!) for a couple of sessions on Day 1 all about BotFramework, LUIS, and the various features and tooling around that ecosystem.

So that you don’t miss out on my handsome visage (!), I’m pre-recording a short session at Microsoft over the next couple of weeks, which will hopefully be played here and there throughout the conference, where I give a very brief case study for the JustEat Help chatbot; it’ll be a tasty Bot infomercial snack – don’t miss out!

Full Schedule

20 February 2017

  • 10.00am: Data, data, data – How and where to store it on Azure?
  • 11.00pm: Conversational UI using the Microsoft BOT Framework
  • 12.00pm: Microsoft Bot Framework and Cognitive Services: Make your BOT smarter!
  • 1.00pm: The best kept secret, Document DB.
  • 2.00pm: Let’s discuss Server-less.
  • 3.00pm: Keynote: Dr Mike Rys What is an Azure Data Lake?

21 February 2017

  • 10.00am: Creating a PHP-MySQL web app in Azure App Service and deploying using FTP.
  • 11.00am: Gain profit from Azure app service tooling as an OSS developer.
  • 12.00pm: Dockerizing Your Cross-Plat .NET Development.
  • 1.00pm: Communication Driven Development.
  • 2.00pm: Monitoring Linux in Azure with Microsoft Operations Management Suite Log Analytics.
  • 3.00pm: The Open Source World of Xamarin.

22 February 2017

  • 10.00am: Bootstrapping blockchain.
  • 11.00am: How IOT and data is changing lives.
  • 12.00pm: An introduction to Quantum Computing.
  • 1.00pm: Social Scientist Professor Bradley Love from University College London and the Alan Turing Institute
  • 2.00pm: Microsoft Regional Directors panel.

If this whets your appetite for nerd knowledge, pre register now at

Custom BotFramework Intent Service Implementation

Developing a chatbot with language understanding capabilities is a huge leap from basic pattern recognition on the input to match to specific commands.

If you have a botframework chatbot, you’re currently limited to using LUIS as your NLP (Natural Language Processing) provider via the various Luis classes: LuisDialog, LuisModelAttribute, and LuisService.

If you’re trying to compare alternative NLP services, such as or or even Alexa, then implementing support for another NLP service in Botframework for this can be a bit tricky.

In this article I’ll show you one approach to decoupling your botframework bot from a specific NLP solution.

Continue reading