Using MongoDb and Sleepy Mongoose and jQuery to save data for a D3 js dataviz

It took me a while to put all these pieces together, so in case anyone else is in the same position, here is what I did.  The code is a bit rough with very little validation...


I wanted to be able to save save different datasets for a d3 dataviz I was working on.  I could have done this in MySQL but as the data was in json already, I thought it a good opportunity to experiment with MongoDB and an HTTP interface.

Having installed from instructions here: http://www.kchodorow.com/blog/2010/02/22/sleepy-mongoose-a-mongodb-rest-interface/

Saving Data

I first tried saving data I already had in javascript:

var request = $.ajax({

url: DB + "_insert",

type: "POST",

data: 'docs=[{"dataset" : "' + dataset + '", "links": '+JSON.stringify(dataitems)+'}]',

dataType: "json"

});

request.done(function(msg) {

alert("success");

});

request.fail(function(jqXHR, textStatus) {

alert( "Request failed: " + textStatus );

});

I had a couple of issues here.  The first was trying to use jsonp to avoid cross domain problems.  I was getting this error:

            jQuery191001886391662992537_1364976848482({"ok" : 0, "errmsg" : "_insert must be a POST request"})

Jsonp converts the post to a get and Mongoose does not support jsonp, so you have to use dataType json.  There are a number of ways to avoid the problem and the comments on the installation page are helpful.  I chose  to start Mongoose using --xorigin parameter:

                         python httpd.py --xorigin 

The second problem was formatting the data.  There needs to be a docs= or you get

            {"ok" : 0, "errmsg" : "missing docs"}

And it needs to be stringified or encodeURI to avoid a parse error.  In the above example, dataset is a string variable and links is an array.


So now my data was saving and I could check by doing:

         http://test.com:27080/sanky/dataset/_find

Listing Data

So now I wanted to pull back a list of the datasets I had saved so the user could choose which dataset to use.  Ideally I want to only pull back the list of dataset fields, it's currently pulling back all the data for each record.

The html has:

<div id="datasets"></div> <button id="load">Load</button><



The javascript to populate:

var request = $.ajax({

url: DB + "_find",

type: "GET",

dataType: "json"

});

request.done(function(data) {

var results = data.results;

// build a select list from the data

var list = d3.select("#datasets").append("select")

.attr("id", "dataset");

list.selectAll("option")

.data(results)

.enter()

.append("option")

.attr("value", function(d) {return d.dataset;})

.text(function(d) {

return d.dataset; });

});

request.fail(function(jqXHR, textStatus) {

alert( "Request failed: " + textStatus );

});

Getting a specific Dataset

And finally, when the user has selected a dataset, retrieving the values:

$("#load").click(function() {

// load selected dataset

var request = $.ajax({

url: DB + "_find",

type: "GET",

data: encodeURI('criteria={"dataset" : "' + $("#dataset").val() + '"}'),

dataType: "json"

});

request.done(function(data) {

// needs some error checking here

links = data.results[0].links;

// redraw diagram here

});

request.fail(function(jqXHR, textStatus) {

alert( "Request failed: " + textStatus );

});

});

The page where this was used: http://beautifuldata.eu/sankey/experiment.html