three ajax problems in my Phonegap app

I’m working on a Phonegap application to run on tablet computers on the walls in my house. I’d been making good progress and enjoying it – using CoffeeScript and LESS for the first time in anger and trying out Sublime (with some plugins) which I’m also very happy with.  But yesterday I started on what I thought would be the easiest bit – making REST calls to the Indigo and  Vera 3 Z-wave controllers.

I inserted some JQuery Ajax code from the CoffeeScript website into an onclick, changed the url to the one for Indigo, and tried it out – fully expecting a clunk as an Aeon relay unit sitting on my desk switched over. But nothing.

$(document).ready ->
    $.ajax http://192.168.0.136:8176/actions/Cellar%20Lights%20On?_method=execute',
        type: 'GET'
        dataType: 'xml'
        error: (jqXHR, textStatus, errorThrown) ->
            $('body').append "AJAX Error: #{textStatus}"
        success: (data, textStatus, jqXHR) ->
            $('body').append "Successful AJAX call: #{data}"

I looked at the error code received and it simply says:

AJAX Error: error

I also added code to output the “errorThrown” value but it was blank as was jqXHR.status

So, since I’d done this lots of times before and was actually just copying the url from an Android native app which was working fine, I started working though all the things I could think of in this order:

  • maybe I malformed the CoffeeScript somehow and stopped it working – but it’s just copied from the CoffeeScript site and I could see nothing wrong
  • maybe I need some extra parameters to the ajax call to make it work – I read through lots of Jquery docs and questions and stumbled on the crossDomain option – maybe this was it! So I started reading up about cross-domain problems and generally the advice is that if you are running from file:// (which I was in the desktop browser where I was developing) or in PhoneGap (which I’d tried a couple of times) it shouldn’t be a problem
  • there is security on the REST call to the Indigo server because I see “access denied” in the Indigo log when I attempt to call it

So I switched the url to the Vera3 one that I knew worked in Android – and the lights went on – Yay! 

$(document).ready ->
    # Advanced Settings
    $.ajax http://192.168.0.206:3480/data_request?id=lu_action&serviceId=urn:micasaverde-com:serviceId:HomeAutomationGateway1&action=RunScene&SceneNum=1&output_format=xml',
        type: 'GET'
        dataType: 'xml'
        error: (jqXHR, textStatus, errorThrown) ->
            $('body').append "AJAX Error: #{textStatus}"
        success: (data, textStatus, jqXHR) ->
            $('body').append "Successful AJAX call: #{data}"

But I still See Ajax Error: error added to the body text.

I looked in the network console on Chrome and saw this:

image

But I couldn’t find much about what (canceled) means.

And I tried the same code on Phonegap on an Android emulator and also on an Android device and in both cases I see the same error code. And in these cases it doesn’t work at turning the lights on/off either – although the logging in Vera 3 is so poor it’s hard to find if the request was received or not.

So to recap. I now seemed to have three problems:

  • a return value indicating error even though the REST request seems to succeed (when run on a desktop browser)
  • authentication is required on the Indigo server which is preventing the server executing the command
  • there is a further difference when running on Android (or emulator) as the REST command doesn’t succeed

assessing the communication flow

To investigate the first problem further I decided I needed lower level info about what was going on. I was going to use WireShark (which I’m fairly familiar with) but discovered chrome://net-internals/#events which shows:

643: URL_REQUEST
http://192.168.0.206:3480/data_request?id=lu_action&serviceId=urn:micasaverde-com:serviceId:HomeAutomationGateway1&action=RunScene&SceneNum=1&output_format=xml
Start Time: 2013-10-24 09:53:01.355
t=1382604781355 [st=  0] +REQUEST_ALIVE  [dt=178]
t=1382604781355 [st=  0]   +URL_REQUEST_START_JOB  [dt=177]
                            --> load_flags = 135151872 (DO_NOT_SAVE_COOKIES | DO_NOT_SEND_AUTH_DATA | DO_NOT_SEND_COOKIES | ENABLE_LOAD_TIMING | MAYBE_USER_GESTURE | VERIFY_EV_CERT)
                            --> method = "GET"
                            --> priority = 2
                            --> url = "http://192.168.0.206:3480/data_request?id=lu_action&serviceId=urn:micasaverde-com:serviceId:HomeAutomationGateway1&action=RunScene&SceneNum=1&output_format=xml"
t=1382604781355 [st=  0]      HTTP_CACHE_GET_BACKEND  [dt=0]
t=1382604781355 [st=  0]      HTTP_CACHE_OPEN_ENTRY  [dt=2]
t=1382604781357 [st=  2]      HTTP_CACHE_ADD_TO_ENTRY  [dt=0]
t=1382604781357 [st=  2]      HTTP_CACHE_READ_INFO  [dt=0]
t=1382604781357 [st=  2]     +HTTP_STREAM_REQUEST  [dt=2]
t=1382604781359 [st=  4]        HTTP_STREAM_REQUEST_BOUND_TO_JOB
                                --> source_dependency = 647 (HTTP_STREAM_JOB)
t=1382604781359 [st=  4]     -HTTP_STREAM_REQUEST
t=1382604781359 [st=  4]     +HTTP_TRANSACTION_SEND_REQUEST  [dt=1]
t=1382604781359 [st=  4]        HTTP_TRANSACTION_SEND_REQUEST_HEADERS
                                --> GET /data_request?id=lu_action&serviceId=urn:micasaverde-com:serviceId:HomeAutomationGateway1&action=RunScene&SceneNum=1&output_format=xml HTTP/1.1
                                    Host: 192.168.0.206:3480
                                    Connection: keep-alive
                                    Accept: application/xml, text/xml, */*; q=0.01
                                    Origin: null
                                    User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36
                                    Accept-Encoding: gzip,deflate,sdch
                                    Accept-Language: en-US,en;q=0.8
t=1382604781360 [st=  5]     -HTTP_TRANSACTION_SEND_REQUEST
t=1382604781360 [st=  5]     +HTTP_TRANSACTION_READ_HEADERS  [dt=171]
t=1382604781360 [st=  5]        HTTP_STREAM_PARSER_READ_HEADERS  [dt=171]
t=1382604781531 [st=176]        HTTP_TRANSACTION_READ_RESPONSE_HEADERS
                                --> HTTP/1.1 200 OK
                                    content-Type: xml
t=1382604781531 [st=176]     -HTTP_TRANSACTION_READ_HEADERS
t=1382604781531 [st=176]      HTTP_CACHE_WRITE_INFO  [dt=0]
t=1382604781531 [st=176]      HTTP_CACHE_WRITE_DATA  [dt=1]
t=1382604781532 [st=177]      HTTP_CACHE_WRITE_INFO  [dt=0]
t=1382604781532 [st=177]   -URL_REQUEST_START_JOB
t=1382604781532 [st=177]    HTTP_TRANSACTION_READ_BODY  [dt=0]
t=1382604781532 [st=177]    HTTP_CACHE_WRITE_DATA  [dt=0]
t=1382604781532 [st=177]    HTTP_TRANSACTION_READ_BODY  [dt=1]
t=1382604781533 [st=178]    HTTP_CACHE_WRITE_DATA  [dt=0]
t=1382604781533 [st=178] -REQUEST_ALIVE

Which seems to me to indicate that it should have worked …

But I still get the message:

AJAX Error: error

so now I’ve found out about CORS

My good friend Andy Chitty is up this weekend and he suggested that there might be something that can be done on the server to allow a cross-domain call. This apparently is called Cross-Origin Resource Sharing (CORS)

The deal is that the server needs to add the following into the header of its response:

Access-Control-Allow-Origin: *

The star means allow all domains to access the server.

Andy put this into a server app he’s working on and, lo and behold, it works cross domain without a hitch.

So what seems to be the case is that the browser makes the request – I guess it would have to because the server might return “allow cross domain” in its header (which unfortunately the Vera3 and Indigo don’t seem to) – and the server will enact the REST “command” (as Vera 3 does) – but the browser stops you getting at the reponse!

So I switched to the Android problem – since the lights do go on and off and I’m not that worried about getting a response back from the server – yet.

delving deeper into the Android problem

I’ve found that it doesn’t work on the emulator or the Android tablet so I started to look further into Android configuration, permissions and whitelisting.

Basically a PhoneGap application can decide unilaterally that access to any domain is acceptable. So I took the view that I should enable everything for now and added

; to /code/res/xml/cordova.xml

as suggested by the documentation but that didn’t work at all.

So I edited the config.xml file in the root of the project and added the “access origin” statement to that – and it worked to the extent that the lights now turn on and off.

I now realise that all of the guff in the PhoneGap/Cordova platforms folder is dynamically generated and shouldn’t be touched – I think – maybe?

And now I get a different error message added to the body text:

AJAX Error: parsererror

Which is progress – of a kind??

I then changed the the dataType: in the ajax call to “text” to avoid what seems to be an error either in the xml produced by the Vera 3 server or the parser in Jquery and it now returns OK which is great news.

trying to get debug capability back for the desktop browser

So now I have some code which is essentially the same code I started with but using dataType: “text” in the JQuery ajax call. And it works on PhoneGap but not in the desktop browser. And accessing Indigo server has the added problem of digest authentication.

For now though it would be great to find a way to get my debug capability back by having the desktop browser work. I found couple of different potential methods for disabling the “cross-origin” limit on the desktop. One is to use a plug-in for a browser – I found these but none seem to work for me (edit: I realised I wasn’t pressing the red CORS button placed on the Chrome toolbar by the 3rd one below and that now works!) and it seems tough to tell what is going on as the developer tools don’t seem to show what the plug-ins are doing.

  1. ForceCORS for Firefox https://addons.mozilla.org/en-US/firefox/addon/forcecors/
  2. JetBrains CorsControl http://blog.jetbrains.com/webstorm/2013/06/cors-control-in-jetbrains-chrome-extension/
  3. Allow-Control-Allow-Origin: * from vitvad https://github.com/vitvad/Access-Control-Allow-Origin/blob/master/README.md

So I’ve found another option using Fiddler and I’ve installed the FiddlerHook plugin too and followed the instructions to enable it.

There is an explanation of how to do inject the “Access-Control-Allow-Origin” into the HTTP response here:  http://blog.caplin.com/2010/03/19/using-fiddler-to-help-develop-cross-domain-capable-javascript-web-applications/ – basically what you do is to change the Custom Rule (select  Rules | Customize Rules in Fiddler UI). This brings up a file and you modify the OnBeforeResponse() function as follows:

static function OnBeforeResponse(oSession: Session)
{
	oSession.oResponse.headers.Add("Access-Control-Allow-Origin", "*");
}

Running the request on the Indigo server I now see:

image

And the I see the OK response from the server - Yay!

back to the authentication problem on the Indigo server

To address this problem I’ve tried Basic authentication using JQuery’s username: and password: ajax options and also using “headers:” to add authentication to the headers. Neither worked and, reading up on the Indigo server, it seems it uses “digest” authentication so the failure isn’t that surprising.

I see that there is a jquery plugin for digest authentication https://code.google.com/p/digestj/ but it includes the server code and I'm thinking of trying to roll my own with CoffeeScript but after all this I’m really keen to get back to the actual app I was trying to develop. One way to do that is to turn off digest authentication on the home automation server for the time being – so that’s what I’ve done – I’ll have to come back to getting the digest working later on!

another wrinkle!

As a result of another conversation with Andy Chitty I’m trying to use PhoneGap with an external URL to the index.html file so that I can maintain the software without having to push an install to 13 tablets fixed into the walls around the house.  It’s not actually that tough as I have a script to do it but it would be nicer if the web-app was actually served rather than being resident on the tablet.

The problem is that when I change to an external URL the whitelisting doesn’t seem to work any more so I’m back into the cross-domain problems again! Aaargh.

the situation now

So now I have some code which is essentially the same code I started with but using dataType: “text” in the JQuery ajax call.  I’ve added the whitelisting to the PhoneGap app and that makes it work for PhoneGap with a local URL (on the local file system) but not using an external one (on a server).

And it works fine on the desktop browser by using Fiddler to insert the required Access-Control-Allow-Origin headers.

The next steps are:

  1. Do something to resolve the digest authentication problem
  2. Possibly move to a situation where all communication with tablet goes through the same server that serves the web-app to PhoneGap – and hope this gets around the cross-domain problem completely OR
  3. Perhaps go back to a PhoneGap app with a local URL

I’ve learned a lot but still feel rather let down by Jquery’s limited error handling capabilities and the obscurity of the error messages.

*I’m using the Better CoffeeScript and Less2Css plugins for Sublime