Handwriting Recognition as a Service

At a recent parent-teacher conference, my son’s teacher voiced his struggle in reading my son’s handwriting. This got me thinking about whether and how technology could assist my son and others like him. In consideration of an educational tablet app that enables people to improve their handwriting, I came across two Handwriting Recognition As A Service (HRAAS) APIs. One is free using Google Input Tools, and one is paid from Vision Objects.

I’m still waiting for Vision Objects to get back to me about a request for more information on their products. Unlike almost every other software company these days, they don’t seem to expose documentation on their products to the public. They want to know who you are and some details about your product before they let you see their APIs.

As for Google, I installed the Google Input Tools for Chrome extension, and I was able to sniff their XHR traffic by debugging the extension. I wrote a “J” in their writing area (using the mouse with Chrome on a PC) as depicted here:

google input tools

Here’s the HTTP POST request body that the extension makes to https://inputtools.google.com/request?itc=en-t-i0-handwrit&app=chext for the input above:

{
    "app_version": 0.4,
    "api_level": "537.36",
    "device": "5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36",
    "input_type": 0,
    "options": "enable_pre_space",
    "requests": [{
        "writing_guide": {
            "writing_area_width": 425,
            "writing_area_height": 194
        },
        "pre_context": "",
        "max_num_results": 10,
        "max_completions": 0,
        "ink": [
            [
                [92, 91, 91, 91, 91, 90, 90, 90, 89, 89, 88, 87, 86, 85, 85, 84, 83, 82, 81, 80, 79, 78, 77, 77, 76, 75, 75, 74, 72, 72, 71, 70, 69, 67, 66, 64, 61, 59, 56, 55, 53, 51, 49, 46, 44, 41, 39, 36, 34, 32, 31, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30],
                [55, 56, 58, 61, 64, 70, 73, 77, 80, 84, 90, 94, 98, 101, 104, 106, 108, 113, 116, 119, 122, 126, 128, 129, 131, 133, 134, 137, 139, 140, 142, 142, 144, 145, 146, 147, 147, 148, 149, 149, 149, 149, 149, 149, 148, 147, 144, 141, 137, 134, 131, 127, 123, 118, 109, 104, 101, 99, 98, 97, 96, 95],
                [0, 187, 203, 203, 219, 234, 250, 250, 265, 265, 281, 297, 297, 312, 328, 343, 343, 359, 375, 375, 390, 406, 406, 421, 437, 437, 453, 468, 484, 484, 499, 515, 515, 531, 531, 546, 562, 562, 577, 593, 609, 609, 624, 640, 640, 655, 671, 671, 687, 702, 702, 718, 733, 749, 765, 765, 780, 796, 796, 811, 843, 858]
            ]
        ]
    }, {
        "feedback": "∅[deleted]",
        "select_type": "deleted",
        "ink_hash": "18d06cfd82f0175f"
    }]
}

and here’s the response:

[
    "SUCCESS",
    [
        [
            "05c9c4d707af74e1",
            [
                "J",
                "j",
                "I",
                "ij",
                "ji",
                "li",
                "il",
                "Ji",
                "ii",
                "jr"
            ],
            [],
            {
                "is_html_escaped": false
            }
        ],
        [
            "18d06cfd82f0175f",
            [],
            [],
            {
                "is_html_escaped": false
            }
        ]
    ]
]

The next step is to see how hard it is to generate the list of coordinates for the request. I don’t know if it’s still relevant, but there’s an old version of the deobfuscated code at http://ctrlq.org/code/19205-google-handwriting-api.

Advertisements

About jmjpro

Software Development Manager java, networking, xml
This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s