Using Lovefield as an AJAX proxy maybe

30 September 2015   1 comment   Web development, Javascript

https://www.peterbe.com/ajaxornot/view7a

Lovefield, by Arthur Hsu at Google, is a cool little Javascript browser abstraction on top of IndexedDB . IndexedDB is this amazingly powerful asynchronous framework for storing data in the browser tied to the domain you're visiting. Unlike it's much "simpler" sibling localStorage, with IndexedDB you can store individual keys in a schema, use indexes for faster retrieval, asynchronous querying and much larger memory capacity than general DOM storage (e.g. localStorage and sessionStorage).

What Lovefield brings is best described by watching this video. But to save you time let me try...:

Anyway, it sounds really cool and I'm looking forward to using it for something. But before I do I thought I'd try using it as a "AJAX proxy".

So what's an AJAX proxy, you ask. Well, it can mean many things but what I have in mind is a pattern where a web app's MVC is tied to a local storage and the local storage is tied to AJAX. That means two immediate benefits:

Another subtle benefit is a "corner case" of that offline capability; when you load up the app you can read from local storage much much faster than from the network meaning you can display user data on the screen sooner. (With the obvious caveat that it might be stale and that the data will change once the network read is completed later)

So, to take this idea for a spin (the use of a local storage to remember the loaded data last time first) I extended my AJAX or Not playground with a hybrid that uses React to render the data, but render the data from Lovefield (and from localStorage too). Remember, it's an experiment so it's a bit clunky and perhaps contrieved. The objective is to notice how soon after loading the page, that data because available for your eyes to consume.

Here is the playground test page

You have to load it at least once to fill your IndexedDB with some data from an AJAX request. Then, reload the page and it'll display what it has locally (in IndexedDB extracted with the Lovefield API). Then, after it's loaded, try refreshing the browser repeatedly. With or without a Wifi connection.

Basically, it works. However, perhaps I've chosen the worst possible test bed for playing with Lovefield. Because it is super slow. If you open the web console, you'll see it reports how long it takes to extract the data out of Lovefield. The code looks like this:

...
getPostsFromDB: function() {
  return schemaBuilder.connect().then(function(db) {
    var table = db.getSchema().table('post');
    return db.select().from(table).exec();
  });
},
...
var t0 = performance.now();
this.getPostsFromDB()
.then(function(results) {
  var t1 = performance.now();
  console.log(results.length, 'records extracted');
  console.log((t1 - t0).toFixed(2) + 'ms to extract');
  ...

You can see the source here in full.

So out of curiousity, I forked this experiment. Kept almost all the React code but replaced the Lovefield stuff with good old JSON.parse(localStorage.getItem('posts') || '[]'). See code here.
This only takes 1-2 milliseconds. Lovefield repeatedly takes about 400-550 milliseconds on my Firefox version 43.

By the way, load up the localStorage fork and after a first load, try refreshing it over and over and notice how amazingly fast it is. Yay localStorage!

Comments

dpapad@chromium.org

Thanks for using Lovefield. Just a few comments about the observed performance.

1) Calling connect() more than once is unnecessary and hurts performance. Connecting to the DB invokes reading from IDB and re-creating all in-memory indices. The intended usage is to connect once and then use the same connection to issue queries to the database. If you are only interesting querying the DB once at startup then I understand that keeping a reference to the DB connection around does not help.

The times you are mentioning (400-550ms) correspond to about 20k rows in Lovefield's performance benchmarks (Chrome only) at https://google.github.io/lovefield/dashboard/src/dashboard.html (see "Initialize populated db"). If you are observing these times and have significantly less records in the DB, something else is going wrong and needs investigation.

2) If you are using IDB as Lovefield's backing store, you can greatly reduce the connect() time by using Lovefield's "bundled mode", more details at https://github.com/google/lovefield/blob/master/docs/spec/99_postfix.md#bundled-mode. This trick basically stores multiple records in a single IDB row, which greatly improves read performance (and therefore connect() performance), at the expense of a decrease in write performance.

Your email will never ever be published


Related posts

Previous:
Panasonic Lumix from 2008 or a iPhone 5S from 2014 26 September 2015
Next:
django-pipeline + django-jinja 04 October 2015
Related by Keyword:
AJAX or not 22 December 2014
localForage vs. XHR 22 October 2014
Lazy loading below the fold 26 October 2013
QUnit testing my jQuery Mobile site in full swing 17 March 2011
XML header and childNodes 26 July 2007
Related by Text:
jQuery and Highslide JS 08 January 2008
I'm back! Peterbe.com has been renewed 05 June 2005
Anti-McCain propaganda videos 12 August 2008
Ever wondered how much $87 Billion is? 04 November 2003
Guake, not Yakuake or Yeahconsole 23 January 2010