Browser Super Powers: getUserMedia

In case you didn’t already believe it, your Web Browser has super powers. No longer is it something to merely display a document marked up with hypertext.

No longer is it limited to the read-only text and images of the olden days (aka the last two decades or so). Oh no. Now that the browser wars have cooled down, and the commons group are collaborating and updating the W3C standards rapidly enough for the eager-beaver browser vendors, we’re seeing new functionality quickly adopted across all major browser.

One of these super powers allows us to access the user’s microphone and camera (with their permission) using a one-liner:

var promise = navigator.mediaDevices.getUserMedia(constraints);

where constraints define the device preferences, such as:

{
  audio: true,
  video: {
    width: { min: 1024, ideal: 1280, max: 1920 },
    height: { min: 776, ideal: 720, max: 1080 },
    facingMode: "user"
  }
}

Here we’re requesting permission to access the device’s microphone and the camera, with a minimum and maximum requirement around the camera resolution, as well as defining a preference for the front-facing camera if available (facingMode).

This is all just plain old javascript too! At the time of writing, you only need to worry about ye olde IE and Opera Mini not supporting it.

Don’t believe me? Go to a website that uses HTTPS, open your browser’s Developer Tools and paste this in to the console:

var constraints = { audio: true, video: true }; 

navigator.mediaDevices.getUserMedia(constraints)
.then(function(mediaStream) {
    var video = document.createElement('video');
    document.getElementsByTagName('body')[0].appendChild(video);
    video = document.querySelector('video');
    video.srcObject = mediaStream;
    video.onloadedmetadata = function(e) {
        video.play();
    };
})
.catch(function(err) { console.log(err.name + ": " + err.message); });

You’ll be prompted for permission to access your devices:
browser requesting permission to access camera

If you allow, then you can scroll to the bottom of the page and see your lovely face appear in a dynamically generated video tag:

dynamically added video element with my pretty face in it
Amazing, right?

Web browsers are getting closer to native apps in what they can achieve, and getUserMedia (aka Stream API) is just one example of this.

Getting Browser Location in Chrome and Deprecating Powerful Features on Insecure Origins

A while ago I created the most useful web page I’ve ever written; an extremely basic page to list the bus arrival times, based on your current location anywhere in London!

I have used this page several times a day on an almost daily basis since I created it, as does my wife; every morning we need to know when the next bus will arrive near our house in order to know when to rush out with our children to take them to school.

I used it every morning to check if I needed to rush out the door to catch the rare “express” bus that would get me to the tube much quicker than the usual one.

I discovered just how useful it is whilst in a pub, deciding whether I had time for another drink before heading home; a quick glance at my phone and I could find out that the next bus home wasn’t for 20 minutes – plenty of time!

However, recently it stopped working for me. It still worked just fine on my wife’s iPhone (and hence Safari), but not on my Nexus (Chrome). It works on my laptop too, or at least appeared to.

So what’s going on?

Continue reading

HTML5 input type “email” validation

The HTML5 “email” input type is great for short-cutting the usual email validation javascript mess. Just by defining your input box as “type=’email'” you get validation for free (in modern browsers).

html5_email_type_validation

But how do you hook into this? Perhaps change the message or do some other actions at the same time? Check out the “invalid” event:

[javascript]
$("#email-input").on(‘invalid’, function (e) {
// do stuff
});

[/javascript]

Or if you’re old school

[javascript]
document.getElementById(’email-input’).addEventListener(‘invalid’, function(e){
// do stuff
});
[/javascript]

This returns a ValidtyState object on which you have the following properties:

  • badInput
  • customError
  • patternMismatch
  • rangeOverflow
  • rangeUnderflow
  • stepMismatch
  • tooLong
  • typeMismatch
  • valid
  • valueMissing

So you can work with it like so:

[javascript]
$("#email-input").on(‘invalid’, function (e) {
if (e.target.validity.typeMismatch) {
alert("enter a valid email address!");
}
});
[/javascript]

Or you could even override the default message that appears in the tooltip:

[javascript]
$("#email-input").on(‘invalid’, function (e) {
if (e.target.validity.typeMismatch) {
e.target.setCustomValidity("enter a valid email address!");
}
});
[/javascript]

Clever stuff, eh?

#velocityconf notes part 2

Yoav Weiss did a great session on responsive images and techniques; this scared me a little as he’s covering a lot of content that could contradict the talk I’m doing later in the week!

image

He’s mentioned some great pertinent points that I’ll reference back to.

Other things he talked about:

LQIP, which sounds like a reintroduction of the ancient lowsrc attribute.

He’s really having a pop at Mobify and their image loading hack script, which he calls Bat-Shit-Loco-Insane ™. Nice. Explaining how it works feels like the whole room has just facepalmed.

He also talks about compressive images. Luckily nothing that completely ruins my stuff.. whew. I’ll just have to update my notes a little bit.

Each session I’m going to gives me ideas on changes I should make to our Friday session!

Smart TV 101 : Part #3 – Deploying to TV

I’m committing to doing 12 months of “101”s; posts and projects themed at beginning something new (or reasonably new) to me. January was all about node development awesomeness. February is all about Smart TV apps.

Deploying to a TV

Now that we’ve got a basic Smart TV app this post will investigate how to get that app on to the TV itself.

Packaging using the IDE

During the initial installation of the IDE you will have been asked to installed Apache; this is what it’s all been leading up to! You actually just need a web server on your home network somewhere; doesn’t have to be apache, doesn’t have to be on your developer pc.

Prerequisites

Make sure you’ve configured your Server settings within the IDE preferences:
samsung-packaging-server-prefs
samsung-packaging-server-prefs-root

The packaging process will drop a zip file into a Widget/ subdirectory of this directory.

Initiating package creation

As for actually creating the package, if you’re using the Eclipse IDE then you’re spoiled for choice: highlight the project in your project explorer and then either

  1. Click the Samsung App Packaging button
    samsung-app-packaging-1
  2. Click the Samsung Smart TV SDK menu, then click App Packaging
    samsung-smart-sdk-packaging-menu
  3. Right click the project in project explorer, Samsung Smart TV SDK, Packaging
    samsung-sdk-packaging-context-menu

Whichever you do you’ll end up with the same results:
samsung-packaging-dialog
samsung-packaging-confirmation

Results

Assuming you’ve set up the server settings in your preferences then you’ll end up with:

  1. a zip file placed within the SDK installation’s Package/ directory
    samsung-package-sdk-dir
  2. the same zip file placed in a Widget/ subdirectory on your configured server
    samsung-package-widget-dir
  3. a new (or updated) widgetlist.xml file in the root of your configured server’s directory

    samsung-widgetlist-xml

Make sure that you can browse to this file and that Apache is running by opening a browser and putting in http://<your development pc’s IP>/widgetlist.xml

Anatomy of a package and a widgetlist

So what is a package made of? Looking at the image above for the the zip file that’s created you’ll see that it looks almost identical to the contents of your application within the workspace:
samsung-app-workspace

So essentially the packaging step is zipping up your project directory, putting it into a specified web server subdirectory, and updating an XML file. Obviously, you shouldn’t need an IDE or SDK to do this sort of thing and I’ll be getting on to this development & deployment process without using Eclipse or installing Apache in a later post.

Deploying!!

Now that we have a package it’s time to load it on to your Smart TV. For this post I’ll be talking about deploying from the development pc via your home network, and in a later post will be talking about loading in packages externally.

TV setup

Make sure your TV is connected to your network and that your development pc’s Windows Firewall is off (or at least configured to allow local network traffic).

  • Turn on the TV
  • Go to your app hub/Smart Hub
  • Press the Login button
  • Create an account using the username “develop” and set a password

developer account

After you’ve successfully created the develop user you need to

  • Open the Settings menu
  • Open the new Development sub menu
  • Choose Setting server IP and enter the IP of your development PC
  • Choose User application synchronisation to check the apps that are listed in widgetlist.xml and install (or update) them all

download dev app

You should now find your application on the App Hub screen with a little red “user” banner over it; select it to run it, just like any other app.

asos-app-running-on-smart-tv-1

Smart TV 101 : Part #2 – App Development

I’m committing to doing 12 months of “101”s; posts and projects themed at beginning something new (or reasonably new) to me. January was all about node development awesomeness. February is all about Smart TV apps.

SDK

There is a wonderfully detailed SDK document for the current latest version (v4.0) which provides the environment to develop and test apps for the 2011, 2012, and 2013 series of TVs.

This consists of an IDE (a version of Eclipse), a selection of emulators for the three series of TVs it supports, automated test tools, app packaging facilities, and a few other tools.

There are examples and tutorials for projects ranging from gesture recognition, voice recognition, adaptive video streaming, through to advertisment embedding.

Developing gesture recognition apps for the 2013 Smart TV series

IDE – Ecplise

I’ve never been a fan of Eclipse as an IDE, but I’m stuck with it at the moment since it’s part of the Samsung SDK! To be fair, it does integrate into app development process quite well.

Once you’ve downloaded it from the SamsungDForum website and installed it you can create one of three types of application:

  1. Basic – for the less codey-types, using as visual editor. A bit like Visual Studio in design mode.
  2. Javascript – for writing the css, html, and js code yourself; this is the one I’ll be using
  3. Flash – strangely enough, for embedding flash into your app

ecplise-1

Within this flavour of Eclipse is the facility to launch the current application under development directly in an emulator, and also the ability to create a package for deployment (to be covered in the next post).

Emulator

As with any project in which you’re developing an application which will be running on a system that is different to the one on which you’re developing – such as iPhone or Android apps – you’re going to need a solid emulator.

The Samsung ones are actually reasonably good. There are some reasonably advanced debugging and testing facilities built into the SDK but even just having any javascript alert display within a debug window is extremely useful.

Smart TV Emulator

Developing a basic app

Right, let’s get down to business.

  1. Install the SDK
  2. Open up Eclipse
  3. Create a new Javascript app
  4. Make sure you’ve selected the project in the file explorer tab (i.e., not one of the js or html files)
  5. Click the Samsung Smart TV menu and select Open current project in Emulator

aaaaannnnd

samsung-emulator-1

WOW! Nothing!

Ok, let’s make it do something.

Add in a new div, give it an id, and whack in some text. This still won’t actually appear so edit the css and give it a height, width, and garish background colour.

There’s still one thing that you may need to check; I believe that this is now part of the standard base project, but in previous versions of the SDK you had to edit the Main.onLoad event and wire up a call to let the application manager know it was now ok to start showing things:
[js]widgetAPI.sendReadyEvent[/js]

My resulting HTML looks a bit like:
[html highlight=”8,9″]<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>AlrightMate</title>

<!– TODO : Common API –>
<script type="text/javascript" language="javascript" src="$MANAGER_WIDGET/Common/API/Widget.js"></script>
<script type="text/javascript" language="javascript" src="$MANAGER_WIDGET/Common/API/TVKeyValue.js"></script>

<!– TODO : Javascript code –>
<script language="javascript" type="text/javascript" src="app/javascript/Main.js"></script>

<!– TODO : Style sheets code –>
<link rel="stylesheet" href="app/stylesheets/Main.css" type="text/css">

<!– TODO: Plugins –>

</head>

<body onload="Main.onLoad();" onunload="Main.onUnload();">
<div id="content">Alright mate?</div>

<!– Dummy anchor as focus for key events –>
<a href="javascript:void(0);" id="anchor" onkeydown="Main.keyDown();"></a>

<!– TODO: your code here –>
</body>
</html>[/html]

and the autogenerated Main.js script has this onLoad method:
[js]Main.onLoad = function()
{
// Enable key event processing
this.enableKeys();
widgetAPI.sendReadyEvent();
}[/js]

Notice the $MANAGER_WIDGET files referenced in the head; these files allow access to common object modules and are on the TV itself and installed as part of the SDK.

Try running the emulator again –

samsung-emulator-2

Stonking.

Developing a slightly less basic app

Using the API created in my January posts on nodejs I’m going to create a tv app which will display the results of a product search on the Asos catalogue.

The main.js file now has an updated onload method, which makes a call to the API and then passes the returned data to a new method:
[js]Main.onLoad = function()
{
var URL = "http://rposbo-basic-node-api.apphb.com/products/socks?key=" + api_key;

if (this.XHRObj != null){
this.XHRObj.destroy();
}
this.XHRObj = new XMLHttpRequest();

if (this.XHRObj) {
alert("got XHR");
this.XHRObj.onreadystatechange = function () {
alert("State changed to " + Main.XHRObj.readyState);
if (Main.XHRObj.readyState == 4) {
alert("got data");
Main.recieveData();
}
};
this.XHRObj.open("GET", URL, true);
this.XHRObj.send(null);
}

// Enable key event processing
this.enableKeys();
widgetAPI.sendReadyEvent();
};[/js]

The new recieveData method which loops through the returned product data and creates some basic html elements to display the image and title in a list item:
[js]Main.recieveData = function () {

alert("alerting data…");
var data = JSON.parse(this.XHRObj.responseText);
for(var i=0; i<data.products.length; i++)
{
var product = data.products[i];
alert("adding " + product.title);

// image
var productImg = document.createElement("img");
productImg.setAttribute("src", product.image);

// text
var title = document.createTextNode(product.title);

// link containing both
var link = document.createElement("a");
link.appendChild(productImg);
link.appendChild(title);

// list item containing link
var listItem = document.createElement("li");
listItem.appendChild(link);

document.getElementById(‘listing’).appendChild(listItem);
}
};[/js]

No jQuery here, since I don’t want to have to load it up locally on to the tv and waste precious memory.

The single html file now looks like
index.html
[html highlight=”12″]<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>rposboBasicTvApp</title>

<!– TODO : Common API –>
<script type="text/javascript" language="javascript" src="$MANAGER_WIDGET/Common/API/Widget.js"></script>
<script type="text/javascript" language="javascript" src="$MANAGER_WIDGET/Common/API/TVKeyValue.js"></script>

<!– TODO : Javascript code –>
<script language="javascript" type="text/javascript" src="app/javascript/key.js"></script>
<script language="javascript" type="text/javascript" src="app/javascript/Main.js"></script>

<!– TODO : Style sheets code –>
<link rel="stylesheet" href="app/stylesheets/Main.css" type="text/css">

<!– TODO: Plugins –>

</head>

<body onload="Main.onLoad();" onunload="Main.onUnload();">

<div id="listing"></div>

<!– Dummy anchor as focus for key events –>
<a href="javascript:void(0);" id="anchor" onkeydown="Main.keyDown();"></a>

<!– TODO: your code here –>
</body>
</html>[/html]
The highlighted line is just where I define my API key and refer to it in Main.js.

This subtly changed code now looks something like:
asos-tv-emulator

Next up – deploying to a TV

We’ve got a basic app, now it’s time to get it on to the TV!

The code from this post is available on github

jquery cookies

For anyone else having fun working with cookies in jquery who decided to use jquery.cookie as their plugin of choice; if you create a cookie
[javascript]
$.cookie(‘ro’,’awesome’);
[/javascript]
And then try to change it
[javascript]
$.cookie(‘ro’,’lame’);
[/javascript]
you may actually end up with a cookie that contains BOTH values.

Instead, try jquery.cookies (plural!) which does seem to update correctly. Or just write your own plugin; cookie manipulation isn’t that tough!