24 November 2020 0 comments That's Groce!
An old friend of mine, from The Netherlands, contacted me about That's Groce! because the default food word suggestions simple don't make sense to him since they're all in English. American English too, I suspect.
Let's back up a bit. Here's a picture of some of the ~100 default food words that are meant to help you when you start out:
The idea is that until you've started using That's Groce! it'll take a while to get your patterns settled. The assumption is that for most people it makes sense to have some sensible default suggestions. This way, as you start typing ch it can suggest: "Cherries 🍒", "Chilis 🌶", "Chicken 🍗", etc.
Now, you can turn this functionality off. The option looks like this:
What's cool about this new feature is that the feature was borne from feedback. My friend used the "Feedback" form and is actually the first one to ever do so. Thanks Ivo!
21 November 2020 0 comments Web development, Mobile, That's Groce!
tl;dr; Up until recently, when you started to type a new entry in your That's Groce shopping list, the suggestions that would appear weren't sorted intelligently. Now they are. They're sorted by popularity.
The whole point with the suggestions that appear is to make it easier for you to not have to type the rest. The first factor that decides which should appear is simply based on what you've typed so far. If you started typing ch we can suggest:
Cherry tomatoesChocolate chipsMini chocolate chipsRainbow chardGoat cheeseChickpeas- etc.
They all contain ch in some form (starting of words only). But space is limited and you can't show every suggestion. So, if you're going to cap it to only show, say, 4 suggestions; which ones should you show first?
I think the solution is to do it by frequency. I.e. items you often put onto the list.
How to calculate the frequency
The way That's Groce
now does it is that it knows the exact times a certain item was added to the list. It then takes that list and applies the following algorithm:
For each item...
- Discard the dates older than 3 months
- Discard any duplicates from clusters (e.g. you accidentally added it and removed it and added it again within minutes)
- Calculate the distance (in seconds) between each add
- From the last 4 times it was added, take the median value of the distance between
So the frequency becomes a number of seconds. It should feel somewhat realistic. In my family, it actually checks out. We buy bananas every week but sometimes slightly more often than that and in our case, the number comes to ~6 days.
The results
Before sorting by popularity
After sorting by popularity
Great! The chances of appreciating and picking one of the suggestions is greater if it's more likely to be what you were looking for. And things that have been added frequently in the past are more likely to be added again.
How to debug this
There's now a new page called the "Popularity contest"
. You get to it from the "List options" button in the upper right-hand corner. On its own, it's fairly "useless" because it just lists them. But it's nice to get a feeling for what your family most frequently add to the list. A lot more can probably be done to this page but for now, it really helps to back up the understanding of how the suggestions are sorted when you're adding new entries.
If you look carefully at my screenshot here you'll notice two little bugs. There are actually two different entries for "Lemon 🍋" and that was from the early days when that could happen.
Also, another bug is that there's one entry called "Bananas" and one called "Bananas 🍌" which is also something that's being fixed in the latest release. My own family's list predates those fixes.
Hope it helps!
28 October 2020 0 comments Web development, Django, Python
tl;dr; <img src="/avatar.random.png" alt="Random avataaar"> generates this image:
(try reloading to get a random new one. funny aren't they?)
When you use
Gravatar you can convert people's email addresses to their mugshot.
It works like this:
<img src="https://www.gravatar.com/avatar/$(md5(user.email))">
But most people don't have their mugshot on Gravatar.com unfortunately. But you still want to display an avatar that is distinct per user. Your best option is to generate one and just use the user's name or email as a seed (so it's always random but always deterministic for the same user). And you can also supply a fallback image to Gravatar that they use if the email doesn't match any email they have. That's where this blog post comes in.
I needed that so I shopped around and found avataaars generator which is available as a React component. But I need it to be server-side and in Python. And thankfully there's a great port called: py-avataaars.
It depends on CairoSVG to convert an SVG to a PNG but it's easy to install. Anyway, here's my hack to generate random "avataaars" from Django:
import io
import random
import py_avataaars
from django import http
from django.utils.cache import add_never_cache_headers, patch_cache_control
def avatar_image(request, seed=None):
if not seed:
seed = request.GET.get("seed") or "random"
if seed != "random":
random.seed(seed)
bytes = io.BytesIO()
def r(enum_):
return random.choice(list(enum_))
avatar = py_avataaars.PyAvataaar(
style=py_avataaars.AvatarStyle.CIRCLE,
# style=py_avataaars.AvatarStyle.TRANSPARENT,
skin_color=r(py_avataaars.SkinColor),
hair_color=r(py_avataaars.HairColor),
facial_hair_type=r(py_avataaars.FacialHairType),
facial_hair_color=r(py_avataaars.FacialHairColor),
top_type=r(py_avataaars.TopType),
hat_color=r(py_avataaars.ClotheColor),
mouth_type=r(py_avataaars.MouthType),
eye_type=r(py_avataaars.EyesType),
eyebrow_type=r(py_avataaars.EyebrowType),
nose_type=r(py_avataaars.NoseType),
accessories_type=r(py_avataaars.AccessoriesType),
clothe_type=r(py_avataaars.ClotheType),
clothe_color=r(py_avataaars.ClotheColor),
clothe_graphic_type=r(py_avataaars.ClotheGraphicType),
)
avatar.render_png_file(bytes)
response = http.HttpResponse(bytes.getvalue())
response["content-type"] = "image/png"
if seed == "random":
add_never_cache_headers(response)
else:
patch_cache_control(response, max_age=60, public=True)
return response
It's not perfect but it works. The URL to this endpoint is /avatar.<seed>.png and if you make the seed parameter random the response is always different.
To make the image not random, you replace the <seed> with any string. For example (use your imagination):
{% for comment in comments %}
<img src="/avatar.{{ comment.user.id }}.png" alt="{{ comment.user.name }}">
<blockquote>{{ comment.text }}</blockquote>
<i>{{ comment.date }}</i>
{% endfor %}
I've put together this test page if you want to see more funny avatar combinations instead of doing work :)
22 October 2020 0 comments Web development, Family, Mobile, Preact, That's Groce!
tl;dr That's Groce! is:
A mobile web app to help families do grocery shopping and meal planning. Developed out of necessity by a family (Peter and Ashley) and used daily in their home.
Hopefully, the About page explains what it does.
Screenshot of a sample list
The backstory
We used to use Wunderlist, but that stopped working. Next, we tried Cozi and that worked for a while but it was buggy and annoying in so many ways. Finally, we gave up and decided to build our own. Exactly how we need it to be, as efficient as possible.
We also tried a couple of regular to-do list apps where you can have shared accounts but we wanted something perfectly tailored towards the specific needs of family grocery shopping (and meal planning). That's how That's Groce! was born.
The killer features
The about page does a good job of listing the killer features but let's emphasize it one more time.
- Free and simple.
- Is really smart about suggestions and auto-complete for speedy entry.
- Works offline (our usual grocery store has no reception unless you brave onto their WiFi).
- Is real-time, so every other device updates immediately as you update or add something on your device.
- Shared list; you can have your own list but you can invite co-owners.
- You can sort your grocery list in the same order you usually walk through your grocery store.
It's not an app store app
You won't find it on the Apple App store. It's a web app that's been tailored to work well in mobile web browsers (iOS Safari) and you can use the "Add to Home screen" so it looks and acts like a regular app.
It would be nice to try to make it a regular native mobile app but that takes significant time which is hard to find but certainly something to aspire to if it can be done in a nice way.
"Really smart about suggestions"
What does that killer feature mean? (At the time of writing (Oct 2020), it isn't launched yet but the pieces are coming together.) Are there certain stables you buy recurringly?
Like milk or bananas or Cheerios. If the app can start to see a pattern of commonly added items, it can suggest it immediately so when you're making your list on Monday morning, you just need to tap to add those.
Another important thing is that as you type, it can suggest many things based on the first or couple of characters you type in, but you can't suggest every single possible word so which one should you suggest first?
The way That's Groce! works is that it learns based on the number of times and how recently you add something to your list. As of today, look what happens when I type a on my list:
When I type a
it suggests things that start with "A" but based on frequency.
The more you use it, the better the suggestions get.
Also, to get you started, over 100 items are preloaded as good suggestions but that's just to get you up and running. Once your family starts to use it, your own suggestions get better and better over time.
"Same order you usually walk through your grocery store"
This was important to us because we found we walk through the aisles in pretty much the same way. Every time. When you walk in you have your produce (veggies first, then fruit, then salad stuff) on the right. Then baked goods and deli. Then meats and alcohol. Etc. So if you can group your items based on these descriptions you can be really efficient with your list and it becomes a lot easier to cross off sections of the store and not have to scroll up and down or having to walk back to pick up that pizza dough all the way back at the deli section.
For this to work, you need to type in groups for your items. But you can call them whatever you like. If you want to type "Aisle 1", "Aisle 2", "Dairy stuff" you can. It's all up to you. Keep in mind that it might feel like a bit of up-front work at first, and it is, but your list is learning so you essentially only have to do it once.
Don't be a slave to your list!
If you do decide to try it, keep one thing in mind: You're in control. You don't need to type in perfect descriptions, amounts, groups, and quantities. If you don't know how to spell "bee-ar-naise sauce", don't worry about it. It's your list. You can type whatever you want or need. A lot of to-do lists invite you with complex options to organize the hell out of your list items. Don't do that. Think of That's Groce! as a fridge post-it note that you and your partner keep in their pocket that automatically synchronizes.
You can help
We built this for ourselves but it's built in a way that any family can use it and hopefully also be better organized. But once you sign in you can submit feedback for suggestions. And if you're into coding, the whole app is Open Source so it's fairly easy to modify the code or even host it yourself if you wanted to: https://github.com/peterbe/groce/
Also, if you do try it and like it, please consider going to the Share the ❤️ page and, you know, share it with friends. Much appreciated!
26 September 2020 0 comments Web development, Web Performance, JavaScript
You can write your CSS so that it depends on images. Like this:
li.one {
background-image: url("skull.png");
}
That means that the browser will do its best to style the li.one with what little it has from the CSS. Then, it'll ask the browser to go ahead and network download that skull.png URL.
But, another option is to embed the image as a data URL like this:
li.one{background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAIAAAACACAYAAADDPmHL...rkJggg==)
As a block of CSS, it's much larger but it's one less network call. What if you know that skull.png will be needed? Is it faster to inline it or to leave it as a URL? Let's see!
First of all, I wanted to get a feeling for how much larger an image is in bytes if you transform them to data URLs. Check out this script's output:
▶ ./bin/b64datauri.js src/*.png src/*.svg
src/lizard.png 43,551 58,090 1.3x
src/skull.png 7,870 10,518 1.3x
src/clippy.svg 483 670 1.4x
src/curve.svg 387 542 1.4x
src/dino.svg 909 1,238 1.4x
src/sprite.svg 10,330 13,802 1.3x
src/survey.svg 2,069 2,786 1.3x
Basically, as a blob of data URL, the images become about 1.3x larger. Hopefully, with HTTP2, the headers are cheap for each URL downloaded over the network, but it's not 0. (No idea what the CPU-work multiplier is)
Experiment assumptions and notes
- When you first calculate the critical CSS, you know that there's no
url(data:mime/type;base64,....) that goes to waste. I.e. you didn't put that in the CSS file or HTML file, bloating it, for nothing. - The images aren't too large. Mostly icons and fluff.
-
If it's SVG images should probably inline them in the HTML natively so you can control their style.
- The HTML is compressed for best results.
- The server is HTTP2
It's a fairly commonly known fact that data URLs have a CPU cost. That base64 needs to be decoded before the image can be decoded by the renderer. So let's stick to fairly small images.
The experiment
I made a page that looks like this:
li {
background-repeat: no-repeat;
width: 150px;
height: 150px;
margin: 20px;
background-size: contain;
}
li.one {
background-image: url("skull.png");
}
li.two {
background-image: url("dino.svg");
}
li.three {
background-image: url("clippy.svg");
}
li.four {
background-image: url("sprite.svg");
}
li.five {
background-image: url("survey.svg");
}
li.six {
background-image: url("curve.svg");
}
and
<ol>
<li class="one">One</li>
<li class="two">Two</li>
<li class="three">Three</li>
<li class="four">Four</li>
<li class="five">Five</li>
<li class="six">Six</li>
</ol>
See the whole page here
The page also uses Bootstrap to make it somewhat realistic. Then, using minimalcss combine the external CSS with the CSS inline and produce a page that is just HTML + 1 <style> tag.
Now, based on that page, the variant is that each url($URL) in the CSS gets converted to url(data:mime/type;base64,blablabla...). The HTML is gzipped (and brotli compressed) and put behind a CDN. The URLs are:
Also, there's this page which is without the critical CSS inlined.
To appreciate what this means in terms of size on the HTML, let's compare:
inlined.html with external URLs: 2,801 bytes (1,282 gzipped)inlined-datauris.html with data URLs: 32,289 bytes (17,177 gzipped)
Considering that gzip (accept-encoding: gzip,deflate) is almost always used by browsers, that means the page is 15KB more before it can be fully downloaded. (But, it's streamed so maybe the comparison is a bit flawed)
Analysis
WebPagetest.org results here. I love WebPagetest, but the results are usually a bit erratic to be a good enough for comparing. Maybe if you could do the visual comparison repeated times, but I don't think you can.
WebPagetest visual comparison
And the waterfalls...
With regular URLs
With data URLs
Fairly expected.
With external image URLs, the browser will start to display the CSSOM before the images have downloaded. Meaning, the CSS is render-blocking, but the external images are not.
The final result comes in sooner with data URLs.
With data URLs you have to stare at a white screen longer.
Next up, using Google Chrome's Performance dev tools panel. Set to 6x CPU slowdown and online with Fast 3G.
I don't know how to demonstrate this other than screenshots:
Performance with external images
Performance with data URLs
Those screenshots are rough attempts at showing the area when it starts to display the images.
Whole Performance tab with external images
Whole Performance tab with data URLs
I ran these things 2 times and the results were pretty steady.
- With external images, fully loaded at about 2.5 seconds
- With data URLs, fully loaded at 1.9 seconds
I tried Lighthouse but the difference was indistinguishable.
Summary
Yes, inlining your CSS images is faster. But it's with a slim margin and the disadvantages aren't negligible.
This technique costs more CPU because there's a lot more base64 decoding to be done, and what if you have a big fat JavaScript bundle in there that wants a piece of the CPU? So ask yourself, how valuable is to not
hog the CPU. Perhaps someone who understands the browser engines better can tell if the base64 decoding cost is spread nicely onto multiple CPUs or if it would stand in the way of the main thread.
What about anti-progressive rendering
When Facebook redesigned www.facebook.com in mid-2020 one of their conscious decisions was to inline the SVG glyphs into the JavaScript itself.
"To prevent flickering as icons come in after the rest of the content, we inline SVGs into the HTML using React rather than passing SVG files to <img> tags."
Although that comment was about SVGs in the DOM, from a JavaScript perspective, the point is nevertheless relevant to my experiment. If you look closely, at the screenshots above (or you open the URL yourself and hit reload with HTTP caching disabled) the net effect is that the late-loading images do cause a bit of "flicker". It's not flickering as in "now it's here", "now it's gone", "now it's back again". But it's flickering in that things are happening with progressive rendering. Your eyes might get tired and they say to your brain "Wake me up when the whole thing is finished. I can wait."
This topic quickly escalates into perceived performance which is a stratosphere of its own. And personally, I can only estimate and try to speak about my gut reactions.
In conclusion, there are advantages to using data URIs over external images in CSS. But please, first make sure you don't convert the image URLs in a big bloated .css
file to data URLs if you're not sure they'll all be needed in the DOM.
Bonus!
If you're not convinced of the power of inlining the critical CSS, check out this WebPagetest run that includes the image where it references the whole bootstrap.min.css as before doing any other optimizations.
With baseline that isn't just the critical CSS
10 September 2020 0 comments Node, JavaScript
To transpile .scss (or .sass) in Node you have the choice between sass and node-sass. sass is a JavaScript compilation of Dart Sass which is supposedly "the primary implementation of Sass" which is a pretty powerful statement. node-sass on the other hand is a wrapper on LibSass which is written in C++. Let's break it down a little bit more.
Speed
node-sass is faster. About 7 times faster
. I took all the SCSS files behind the current MDN Web Docs which is fairly large. Transformed into CSS it becomes a ~180KB blob of CSS (92KB when optimized with csso).
Here's my ugly benchmark test which I run about 10 times like this:
node-sass took 101ms result 180kb 92kb
node-sass took 99ms result 180kb 92kb
node-sass took 99ms result 180kb 92kb
node-sass took 100ms result 180kb 92kb
node-sass took 100ms result 180kb 92kb
node-sass took 103ms result 180kb 92kb
node-sass took 102ms result 180kb 92kb
node-sass took 113ms result 180kb 92kb
node-sass took 100ms result 180kb 92kb
node-sass took 101ms result 180kb 92kb
And here's the same thing for sass:
sass took 751ms result 173kb 92kb
sass took 728ms result 173kb 92kb
sass took 728ms result 173kb 92kb
sass took 798ms result 173kb 92kb
sass took 854ms result 173kb 92kb
sass took 726ms result 173kb 92kb
sass took 727ms result 173kb 92kb
sass took 782ms result 173kb 92kb
sass took 834ms result 173kb 92kb
In another example, I ran sass and node-sass on ./node_modules/bootstrap/scss/bootstrap.scss (version 5.0.0-alpha1) and the results are after 5 runs:
node-sass took 269ms result 176kb 139kb
node-sass took 260ms result 176kb 139kb
node-sass took 288ms result 176kb 139kb
node-sass took 261ms result 176kb 139kb
node-sass took 260ms result 176kb 139kb
versus
sass took 1423ms result 176kb 139kb
sass took 1350ms result 176kb 139kb
sass took 1338ms result 176kb 139kb
sass took 1368ms result 176kb 139kb
sass took 1467ms result 176kb 139kb
Output
The unminified CSS difference primarily in the indentation. But you minify both outputs and the pretty print them (with prettier) you get the following difference:
▶ diff /tmp/sass.min.css.pretty /tmp/node-sass.min.css.pretty
152c152
< letter-spacing: -0.0027777778rem;
---
> letter-spacing: -0.00278rem;
231c231
< content: "▼︎";
---
> content: "\25BC\FE0E";
...snip...
2804c2812
< .external-icon:not([href^="https://mdn.mozillademos.org"]):not(.ignore-external) {
---
> .external-icon:not([href^='https://mdn.mozillademos.org']):not(.ignore-external) {
Basically, sass will use produce things like letter-spacing: -0.0027777778rem; and content: "▼︎";. And node-sass will produce letter-spacing: -0.00278rem; and
content: "\25BC\FE0E";.
I also noticed some minor difference just in the order of some selectors but when I look more carefully, they're immaterial order differences meaning they're not cascading each other in any way.
Note! I don't know why the use of ' and " is different or if it matters. I don't know know why prettier (version 2.1.1) didn't pick one over the other consistently.
node_modules
Here's how I created two projects to compare
cd /tmp
mkdir just-sass && cd just-sass && yarn init -y && time yarn add sass && cd ..
mkdir just-node-sass && cd just-node-sass && yarn init -y && time yarn add node-sass && cd ..
Considering that sass is just a JavaScript compilation of a Dart program, all you get is basically a 3.6MB node_modules/sass/sass.dart.js file.
The /tmp/just-sass/node_modules directory is only 113 files and folders weighing a total of 4.1MB.
Whereas /tmp/just-node-sass/node_modules
directory is 3,658 files and folders weighing a total of 15.2MB.
I don't know about you but I'm very skeptical that node-gyp ever works. Who even has Python 2.7 installed anymore? Being able to avoid node-gyp seems like a win for sass.
Conclusion
The speed difference may or may not matter. If you're only doing it once, who cares about a couple of hundred milliseconds. But if you're forced to have to wait 1.4 seconds on every Ctrl-S when Webpack or whatever tooling you have starts up sass it might become very painful.
I don't know much about the sass-loader Webpack plugin but it apparently works with either but they do recommend sass in their documentation. And it's the default implementation too.
It's definitely a feather in sass
's hat that Dart Sass is the "primary implementation" of Sass. That just has a nice feelin in sass's favor.
Bonus
NPMCompare has a nice comparison of them as projects but you have to study each row of numbers because it's rarely as simple as more (or less) number is better. For example, the number of open issues isn't a measure of bugs.
The new module system launched in October 2019 supposedly only comes to Dart Sass which means sass is definitely going to get it first. If that stuff matters to you. For example, true, the Sass unit-testing tool, now requires Dart Sass and drops support for node-sass.
