Yeah, 400-500ms is really bad, although I should note I just tried http://www.peterbe.com/ajaxornot/view7a several times in a row and my times on e10s nightly the times reported were largely in the 100ms-150ms time range. (I did get one outlier at 207ms.) Since you're loading all the rows, it's possible the improvement came from https://bugzilla.mozilla.org/show_bug.cgi?id=1168606 landing which implements pre-fetching.
For FxOS we in general have been using the non-standard mozGetAll method to fetch things in batches, which avoids wasteful roundtrips, etc. (They really could get out of hand... IDB was erring on the side of not letting people footgun too much. And people did. Many a tale of woe can be told about overuse of mozGetAll by FxOS apps/custom web APIs!)
There is definitely a non-trivial overhead in the act of opening an IDB database. It looks like your call to connect() is part of your timed logic, and per https://github.com/google/lovefield/blob/master/docs/spec/03_life_of_db.md this would seem to involve the database opening, so I expect that could have a lot to do with it. It would be interesting to split the numbers out to separate the connect(), although obviously that number is very important/significant to user responsiveness at startup!
And yes, absolutely there is a non-trivial cost to spinning up/and opening an IndexedDB database in terms of disk I/O etc. IDB is pretty good about spinning things back down, but it wouldn't surprise me if at 500K of data local storage was better on balance, especially if you're storing the data in only one or a few keys.
I love this "debate"! These are really interesting questions to solve.
We're not disagreeing on anything but one argument I'd like to make is that if you look at http://www.peterbe.com/plog/lovefield-ajax-proxy where I used a wrapper over IDB called Lovefield. It's a similar pattern, load from Lovefield first, then wait for AJAX to get more fresh data. What I found was that loading stuff from Lovefield takes 400-500ms EVERY time. And it only takes 1-2ms with localStorage. (granted the point of something like Lovefield is not to select EVERY record every time but we'll let that slide). So my question is, what's it doing to my CPU/RAM during that half second? Whatever it is, even if it's off the main thread, is bound to cause slowdown on the main thread and strain on the resources.
I.e. localStorage is like a buffalo but it gets the job done. Perhaps that ultimately wins despite it's shortcomings.
For what it's worth, the 400-500ms I can't get now any more either. Hovering around 190-230ms in my laptop.
It's perhaps silly to compare pure localStorage with Lovefield + connect() + IDB all in one lump but the pragmatist in me is only interested in "getting the data out" :)
Comment
Yeah, 400-500ms is really bad, although I should note I just tried http://www.peterbe.com/ajaxornot/view7a several times in a row and my times on e10s nightly the times reported were largely in the 100ms-150ms time range. (I did get one outlier at 207ms.) Since you're loading all the rows, it's possible the improvement came from https://bugzilla.mozilla.org/show_bug.cgi?id=1168606 landing which implements pre-fetching.
For FxOS we in general have been using the non-standard mozGetAll method to fetch things in batches, which avoids wasteful roundtrips, etc. (They really could get out of hand... IDB was erring on the side of not letting people footgun too much. And people did. Many a tale of woe can be told about overuse of mozGetAll by FxOS apps/custom web APIs!)
There is definitely a non-trivial overhead in the act of opening an IDB database. It looks like your call to connect() is part of your timed logic, and per https://github.com/google/lovefield/blob/master/docs/spec/03_life_of_db.md this would seem to involve the database opening, so I expect that could have a lot to do with it. It would be interesting to split the numbers out to separate the connect(), although obviously that number is very important/significant to user responsiveness at startup!
And yes, absolutely there is a non-trivial cost to spinning up/and opening an IndexedDB database in terms of disk I/O etc. IDB is pretty good about spinning things back down, but it wouldn't surprise me if at 500K of data local storage was better on balance, especially if you're storing the data in only one or a few keys.
Parent comment
I love this "debate"! These are really interesting questions to solve. We're not disagreeing on anything but one argument I'd like to make is that if you look at http://www.peterbe.com/plog/lovefield-ajax-proxy where I used a wrapper over IDB called Lovefield. It's a similar pattern, load from Lovefield first, then wait for AJAX to get more fresh data. What I found was that loading stuff from Lovefield takes 400-500ms EVERY time. And it only takes 1-2ms with localStorage. (granted the point of something like Lovefield is not to select EVERY record every time but we'll let that slide). So my question is, what's it doing to my CPU/RAM during that half second? Whatever it is, even if it's off the main thread, is bound to cause slowdown on the main thread and strain on the resources. I.e. localStorage is like a buffalo but it gets the job done. Perhaps that ultimately wins despite it's shortcomings.
Replies
For what it's worth, the 400-500ms I can't get now any more either. Hovering around 190-230ms in my laptop.
It's perhaps silly to compare pure localStorage with Lovefield + connect() + IDB all in one lump but the pragmatist in me is only interested in "getting the data out" :)