Receiving Images to Your Skype Botframework Bot (v2!)

If you’re getting a “403” HTTP error when attempting to receive an image sent to your Skype bot, and the previous use of message.ServiceUrl to create a ConnectorClient didn’t work, try this more verbose version which explicitly sets the authorization header:

byte[] data;

if (image.ContentUrl != null)
{
    using (var connectorClient 
        = new ConnectorClient(new Uri(message.ServiceUrl)))
    {
        var token = 
            await (connectorClient.Credentials as MicrosoftAppCredentials)
                .GetTokenAsync();

        var uri = new Uri(image.ContentUrl);

        using (var httpClient = new HttpClient())
        {
            if (uri.Host.EndsWith("skype.com") 
                && uri.Scheme == Uri.UriSchemeHttps)
            {
                httpClient
                    .DefaultRequestHeaders
                    .Authorization = 
                        new AuthenticationHeaderValue("Bearer", token);

                httpClient
                    .DefaultRequestHeaders
                    .Accept
                    .Add(new MediaTypeWithQualityHeaderValue("application/octet-stream"));
            }

            // Get the image in a byte[] variable
            data = await httpClient.GetByteArrayAsync(uri);
        }
    }
}

Generating Image Hashtags using Microsoft’s Computer Vision API

Whatever your social media tool of choice is these days, it’s almost guaranteed to be filled with images and their associated hashtags #sorrynotsorry #lovelife #sunnyday

Sometimes coming up with those tags is more work than perfectly framing your latest #flatlay shot.

In the age of amazing image recognition tech, it must be possible to create something that can help us out and give us more time to move that light source around to cast the right shadow over your meal.

Turns out, it is possible! Yay! (of course..)

In this article I’ll show you how to automatically generate image hashtags via a chatbot using Microsoft’s Computer Vision API.

Continue reading

Sentiment Analysis using Microsoft’s Cognitive Services

Now that we are making more conversational interfaces thanks to technology like botframework, interaction with the user is no longer limited to a tap on a link or a button.

Having written language as the primary form of interaction with our systems gives significant difficulties in terms of intent understanding, but also gives great opportunities for further understanding of the user.

Intent understanding has already been tackled by the likes of LUIS; what about the user’s sentiment?

In this article I’m going to introduce Microsoft’s Text Analysis API and show you how to easily get sentiment analysis for a message coming in to your bot.

Continue reading

MVP led TechDays Online: Videos!

As part of Microsoft’s recent Tech Days Online, I was very pleased to be able to record a couple of short videos about botframework, LUIS, the QnA Maker, and how I have been working with JustEat to use these technologies in their Customer Help chatbot solution.

Unfortunately I wasn’t able to attend the live TechDays sessions, so instead of an hour or two of my dulcet tones you only have the pleasure of ten minutes; feel free to replay those minutes as many times as you like!

First up, a ten minute session on the JustEat Customer Care chatbot implementation:

Continue reading

Virtual Shop Assistant Chatbot with Amazing Image Recognition

There has been some significant progress in “deep learning”, AI, and image recognition over the past couple of years; Google, Microsoft, and Amazon each have their own service offering. But what is the service like? How useful is it?

Everyone’s having a go at making a chatbot this year (and if you’re not, perhaps you should contact me for consultancy or training!) – and although there are some great examples out there, I’ve not seen much in the e-commerce sector worth talking about.

In this article I’m going to show you a cool use case for an image recognition e-commerce chatbot via a couple of clever APIs wired together by botframework.

Continue reading

Custom BotFramework Intent Service Implementation

Developing a chatbot with language understanding capabilities is a huge leap from basic pattern recognition on the input to match to specific commands.

If you have a botframework chatbot, you’re currently limited to using LUIS as your NLP (Natural Language Processing) provider via the various Luis classes: LuisDialog, LuisModelAttribute, and LuisService.

If you’re trying to compare alternative NLP services, such as kitt.ai or wit.ai or even Alexa, then implementing support for another NLP service in Botframework for this can be a bit tricky.

In this article I’ll show you one approach to decoupling your botframework bot from a specific NLP solution.

Continue reading

Sending Proactive Botframework Messages

Having a botframework chatbot up and running and responding to user messages is one thing, but how can you send a new message to bring the user back into the conversation if they haven’t just sent a new message for you to reply to?

The botframework documentation and other tutorials will point you towards using Azure Functions and the new ActivityType.Trigger to handle this which, although being a great use case for Azure Functions, make the underlying implementation harder to understand. It also means you couldn’t easily implement this on AWS, for example.

In this article I’ll show you how to easily implement Proactive Botframework Messaging just using BotFramework fundamentals.

Continue reading

Connecting Alexa to a Botframework Chatbot

In the previous article we dissected an Alexa Skill down to the JSON request and Response, and pointed it to an HTTPS endpoint (your laptop) to get a basic end to end Skill working.

In this article I’ll show you how to link that skill into your botframework chatbot.

Creating a botframework reply

Let’s dip back into BotFramework in order to create something that can respond to the incoming request.

Calculating a Chinese Zodiac animal based on the year is really simple; just get the remainder from dividing by 12 and apply a switch:

Continue reading

Receiving files sent to your botframework chatbot

We’ve already looked at how a botframework bot receives messages, and even how to save those messages.

In this article I’ll show you how to handle files that are sent to your botframework chatbot.

When a user interacts with your bot, unless they’re responding to a prompt, they will cause the Message controller’s Post method to fire with an activity.

This will send a message through to your underlying IDialog or LuisDialog implementation.

MessageReceived

The method that receives the message will have the signature (though the parameter names and method name could be different):

Continue reading

Transcribing messages in BotFramework

Now that you’ve deployed your well structured, exception handling, language understanding chatbot, how do you know just what people are saying to it?

Sure, you could copy and paste some logging code all over the place, but there must be a cleaner way.

In this article I’ll show you a few simple tricks to be able to save each message going to and from your botframework chatbot.

IDialog

Let’s start off by saving messages going to any dialog that implements IDialog<T>.

In order to implement IDialog<T> you only need to implement the StartAsync method; however, this isn’t much use on its own, so let’s get the dialog into a conversation loop by adding in a MessageReceivedAsync method and calling that from StartAsync and from itself:

Continue reading