Peterbe.com

To transpile .scss (or .sass) in Node you have the choice between sass and node-sass. sass is a JavaScript compilation of Dart Sass which is supposedly "the primary implementation of Sass" which is a pretty powerful statement. node-sass on the other hand is a wrapper on LibSass which is written in C++. Let's break it down a little bit more.

Speed

node-sass is faster. About 7 times faster. I took all the SCSS files behind the current MDN Web Docs which is fairly large. Transformed into CSS it becomes a ~180KB blob of CSS (92KB when optimized with csso).

Here's my ugly benchmark test which I run about 10 times like this:

node-sass took 101ms result 180kb 92kb
node-sass took 99ms result 180kb 92kb
node-sass took 99ms result 180kb 92kb
node-sass took 100ms result 180kb 92kb
node-sass took 100ms result 180kb 92kb
node-sass took 103ms result 180kb 92kb
node-sass took 102ms result 180kb 92kb
node-sass took 113ms result 180kb 92kb
node-sass took 100ms result 180kb 92kb
node-sass took 101ms result 180kb 92kb

And here's the same thing for sass:

sass took 751ms result 173kb 92kb
sass took 728ms result 173kb 92kb
sass took 728ms result 173kb 92kb
sass took 798ms result 173kb 92kb
sass took 854ms result 173kb 92kb
sass took 726ms result 173kb 92kb
sass took 727ms result 173kb 92kb
sass took 782ms result 173kb 92kb
sass took 834ms result 173kb 92kb

In another example, I ran sass and node-sass on ./node_modules/bootstrap/scss/bootstrap.scss (version 5.0.0-alpha1) and the results are after 5 runs:

node-sass took 269ms result 176kb 139kb
node-sass took 260ms result 176kb 139kb
node-sass took 288ms result 176kb 139kb
node-sass took 261ms result 176kb 139kb
node-sass took 260ms result 176kb 139kb

versus

sass took 1423ms result 176kb 139kb
sass took 1350ms result 176kb 139kb
sass took 1338ms result 176kb 139kb
sass took 1368ms result 176kb 139kb
sass took 1467ms result 176kb 139kb

Output

The unminified CSS difference primarily in the indentation. But you minify both outputs and the pretty print them (with prettier) you get the following difference:

▶ diff /tmp/sass.min.css.pretty /tmp/node-sass.min.css.pretty
152c152
<   letter-spacing: -0.0027777778rem;
---
>   letter-spacing: -0.00278rem;
231c231
<   content: "▼︎";
---
>   content: "\25BC\FE0E";

...snip...


2804c2812
< .external-icon:not([href^="https://mdn.mozillademos.org"]):not(.ignore-external) {
---
> .external-icon:not([href^='https://mdn.mozillademos.org']):not(.ignore-external) {

Basically, sass will use produce things like letter-spacing: -0.0027777778rem; and content: "▼︎";. And node-sass will produce letter-spacing: -0.00278rem; and content: "\25BC\FE0E";.
I also noticed some minor difference just in the order of some selectors but when I look more carefully, they're immaterial order differences meaning they're not cascading each other in any way.

Note! I don't know why the use of ' and " is different or if it matters. I don't know know why prettier (version 2.1.1) didn't pick one over the other consistently.

node_modules

Here's how I created two projects to compare

cd /tmp
mkdir just-sass && cd just-sass && yarn init -y && time yarn add sass && cd ..
mkdir just-node-sass && cd just-node-sass && yarn init -y && time yarn add node-sass && cd ..

Considering that sass is just a JavaScript compilation of a Dart program, all you get is basically a 3.6MB node_modules/sass/sass.dart.js file.

The /tmp/just-sass/node_modules directory is only 113 files and folders weighing a total of 4.1MB.
Whereas /tmp/just-node-sass/node_modules directory is 3,658 files and folders weighing a total of 15.2MB.

I don't know about you but I'm very skeptical that node-gyp ever works. Who even has Python 2.7 installed anymore? Being able to avoid node-gyp seems like a win for sass.

Conclusion

The speed difference may or may not matter. If you're only doing it once, who cares about a couple of hundred milliseconds. But if you're forced to have to wait 1.4 seconds on every Ctrl-S when Webpack or whatever tooling you have starts up sass it might become very painful.

I don't know much about the sass-loader Webpack plugin but it apparently works with either but they do recommend sass in their documentation. And it's the default implementation too.

It's definitely a feather in sass's hat that Dart Sass is the "primary implementation" of Sass. That just has a nice feelin in sass's favor.

Bonus

NPMCompare has a nice comparison of them as projects but you have to study each row of numbers because it's rarely as simple as more (or less) number is better. For example, the number of open issues isn't a measure of bugs.

The new module system launched in October 2019 supposedly only comes to Dart Sass which means sass is definitely going to get it first. If that stuff matters to you. For example, true, the Sass unit-testing tool, now requires Dart Sass and drops support for node-sass.

I'm working on a Firebase app called That's Groce! based on preact-cli, with TypeScript, and I wanted to see how it appears with or without Firestore and Authenticated lazy-loaded.

In the root, there's an app.tsx that used look like this:

import { FunctionalComponent, h } from "preact";
import { useState, useEffect } from "preact/hooks";

import firebase from "firebase/app";
import "firebase/auth";
import "firebase/firestore";

import { firebaseConfig } from "./firebaseconfig";

const app = firebase.initializeApp(firebaseConfig);

const App: FunctionalComponent = () => {
  const [auth, setAuth] = useState<firebase.auth.Auth | null>(null);
  const [db, setDB] = useState<firebase.firestore.Firestore | null>(null);

  useEffect(() => {
    const appAuth = app.auth();
    setAuth(appAuth);
    appAuth.onAuthStateChanged(authStateChanged);

    const db = firebase.firestore();
    setDB(db);
  }, []);

...

While this works, it does make a really large bundle when both firebase/firestore and firebase/auth imported in the main bundle. In fact, it looks like this:

▶ ls -lh build/*.esm.js
-rw-r--r--  1 peterbe  staff   510K Sep  1 14:13 build/bundle.0438b.esm.js
-rw-r--r--  1 peterbe  staff   5.0K Sep  1 14:13 build/polyfills.532e0.esm.js

510K is pretty hefty to have to ask the client to download immediately. It's loaded like this (in build/index.html):

<script crossorigin="anonymous" src="/bundle.0438b.esm.js" type="module"></script>
<script nomodule src="/polyfills.694cb.js"></script>
<script nomodule defer="defer" src="/bundle.a4a8b.js"></script>

To lazy-load this

To lazy-load the firebase/firestore and firebase/auth you do this instead:

...

const App: FunctionalComponent = () => {
  const [auth, setAuth] = useState<firebase.auth.Auth | null>(null);
  const [db, setDB] = useState<firebase.firestore.Firestore | null>(null);

  useEffect(() => {
    import("firebase/auth")
      .then(() => {
        const appAuth = app.auth();
        setAuth(appAuth);
        appAuth.onAuthStateChanged(authStateChanged);
      })
      .catch((error) => {
        console.error("Unable to lazy-load firebase/auth:", error);
      });

    import("firebase/firestore")
      .then(() => {
        const db = firebase.firestore();
        setDB(db);
      })
      .catch((error) => {
        console.error("Unable to lazy-load firebase/firestore:", error);
      });
  }, []);

...

Now it looks like this instead:

▶ ls -lh build/*.esm.js
-rw-r--r--  1 peterbe  staff   173K Sep  1 14:24 build/11.chunk.b8684.esm.js
-rw-r--r--  1 peterbe  staff   282K Sep  1 14:24 build/12.chunk.3c1c4.esm.js
-rw-r--r--  1 peterbe  staff    56K Sep  1 14:24 build/bundle.7225c.esm.js
-rw-r--r--  1 peterbe  staff   5.0K Sep  1 14:24 build/polyfills.532e0.esm.js

The total sum of all (relevant) .esm.js files is the same (minus a difference of 430 bytes).

But what does it really look like? The app is already based around that

const [db, setDB] = useState<firebase.firestore.Firestore | null>(null);

so it knows to wait until db is truthy and it displays a <Loading/> component until it's ready.

To test how it loads I used the Chrome Performance devtools with or without the lazy-loading and it's fairly self-explanatory:

Before, no lazy-loading

After, with lazy-loading

Clearly, the lazy-loaded has a nicer pattern in that it breaks up the work by the main thread.

Conclusion

It's fairly simple to do and it works. The main bundle becomes lighter and allows the browser to start rendering the Preact component sooner. But it's not entirely obvious if it's that much better. The same amount of JavaScript needs to downloaded and parsed no matter what. It's clearly working as a pattern but it's still pretty hard to judge if it's worth it. Now there's more "swapping".

And the whole page is server-side rendered anyway so in terms of immediate first-render it's probably the same. Hopefully, HTTP2 loading does the right thing but it's not yet entirely clear if the complete benefit is there. I certainly hope that this can improve the "Total Blocking Time" and "Time to Interactive".

The other important thing is that not all imports from firebase/* work in Node because they depend on window. It works for firebase/firestore and firestore/auth but not for firestore/analytics and firestore/performance. Now, I can add those lazy-loaded in the client and have the page rendered in Node for that initial build/index.html.

<datalist> is an underrated HTML API. It's basically a native autocomplete widget that requires 0 JavaScript. What I didn't know is how great it is on mobile devices. Especially the iOS Safari platform which is, to be honest, the only mobile device I have.

What's cool about it is that it's easy to implement, from a developer's point-of-view. But most importantly, it works great for users. The problem usually on mobile devices and autocomplete is that it's hard to find a good spot to display the suggestions. Most autocomplete widgets are a styled form of <div class="results"><ul><li>Suggestion 1</li><li>Suggestion 2</li></ul></div> that usually follows the <input> element. Usually, this simply boils down to screen height real estate. Oftentimes you want to display so much more rich stuff in the autocomplete results but it's hard to fit in a nice list of results between the <input> and the native keyboard display. For example, on this page ...

Note how the search results get hidden underneath the keyboard.

Demo

The cool thing about <datalist> is that gets embedded in the native mobile keyboard in a sense. But what's extra cool is that the browser will do an OK job of filtering all the options for you, so that you, as a developer, just need to supply all options and the browser will take care of the rest.

I put together a dead-simple app here: https://cnfyl.csb.app/ (source here) which looks like this on iOS:

Caveats

The space that the keyboard now populates with suggestions is usually reserved for helping you autocomplete regular words. It still does if you start typing a word that isn't an option. So arguably, the <datalist> options are primarily helping you when it's very likely that the user will type one of the suggestions.

The matching isn't great in my opinion. If you type "ea" it will match "Peaches" but I find it extremely unlikely that that's helping users. (What do you think?) If you've started typing "ea" if there's no match called "Each" or "North East" then it's probably better with no match at all.

Mind you, check out this hack (source here) which takes control of the <option> tags inside the <datalist> by having an event listener on the input. So if the input is "ea" it only matches expressions that are left-word-delimited and discard the rest.

Default/Native filtering

Custom filtering

Conclusion

It is without a doubt the simplest autocomplete functionality you can buy. I would buy it again.

Perhaps it's not right for every application. Perhaps it's important to be able to include images in your autocomplete suggestions. Either way, the best thing to do is to park this in the back of your mind till next time you're up against the need for some sort of assisted search or choice. Especially if you predict you'll have a lot of users on mobile devices.

This saved my bacon today and I quite like it so I hope that others might benefit from this little tip.

So you have two "URLs" and you want to know if they are "equal". I write those words, in the last sentence, in quotation marks because they might not be fully formed URLs and what you consider equal might depend on the current business logic.

In my case, I wanted http://www.peterbe.com/path/to?a=b to be considered equal to/path/to#anchor. Because, in this case the both share the exact same pathname (/path/to). So how to do it:

function equalUrls(url1, url2) {
  return (
    new URL(url1, "http://example.com").pathname ===
    new URL(url2, "http://example.com").pathname
  );
}

I need this function to relate to open-editor which is a Node program that can open your $EDITOR from Node and jump to a specific file, to a specific line, to a specific column.

Here's the code:

function* findMatchesInText(needle, haystack, { inQuotes = false } = {}) {
  const escaped = needle.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
  let rex;
  if (inQuotes) {
    rex = new RegExp(`['"](${escaped})['"]`, "g");
  } else {
    rex = new RegExp(`(${escaped})`, "g");
  }
  for (const match of haystack.matchAll(rex)) {
    const left = haystack.slice(0, match.index);
    const line = (left.match(/\n/g) || []).length + 1;
    const lastIndexOf = left.lastIndexOf("\n") + 1;
    const column = match.index - lastIndexOf + 1;
    yield { line, column };
  }
}

And you use it like this:

const text = ` bravo
Abra
cadabra

bravo
`;

console.log(Array.from(findMatchesInText("bra", text)));

Which prints:

[
  { line: 1, column: 2 },
  { line: 2, column: 2 },
  { line: 3, column: 5 },
  { line: 5, column: 1 }
]

The inQuotes option is because a lot of times this function is going to be used for finding the href value in unstructured documents that contain HTML <a> tags.

tl;dr hashin 0.15.0 makes package comparison agnostic to underscore or hyphens

See issue #116 for a fuller story. Basically, now it doesn't matter if you write...

hashin python_memcached

...or...

hashin python-memcached

And the same can be said about the contents of your requirements.txt file. Suppose it already had something like this:

python_memcached==1.59 \
    --hash=sha256:4dac64916871bd35502 \
    --hash=sha256:a2e28637be13ee0bf1a8

and you type hashin python-memcached it will do the version comparison on these independent of the underscore or hyphen.

Thank @caphrim007 who implemented this for the benefit of Renovate.

tl;dr; Here's a useful bash script to avoid starting something when its already running as a ghost process.

Huey is a great little Python library for doing background tasks. It's like Celery but much lighter, faster, and easier to understand.

What cost me almost an hour of hair-tearing debugging today was that I didn't realize that a huey daemon process had gotten stuck in the background with code that wasn't updating as I made changes to the tasks.py file in my project. I just couldn't understand what was going on.

The way I start my project is with honcho which is a Python Foreman clone. The Procfile looks something like this:

elasticsearch: cd /Users/peterbe/dev/PETERBECOM/elasticsearch-7.7.0 && ./bin/elasticsearch -q
web: ./bin/run.sh web
minimalcss: cd minimalcss && PORT=5000 yarn run start
huey: ./manage.py run_huey --flush-locks --huey-verbose
adminui: cd adminui && yarn start
pulse: cd pulse && yarn run dev

And you start that with simply typing:

honcho start

When you Ctrl-C, it kills all those processes but somehow somewhere it doesn't always kill everything. Restarting the computer isn't a fun alternative.

So, to prevent my sanity from draining I wrote this script:

#!/usr/bin/env bash
set -eo pipefail

# This is used to make sure that before you start huey, 
# there isn't already one running the background.
# It has happened that huey gets lingering stuck as a 
# ghost and it's hard to notice it sitting there 
# lurking and being weird.

bad() {
    echo "Huey is already running!"
    exit 1
}

good() {
    echo "Huey is NOT already running"
    exit 0
}

ps aux | rg huey | rg -v 'rg huey' | rg -v 'huey-isnt-running.sh' && bad || good

(If you're wondering what rg is; it's short for ripgrep)

And I change my Procfile accordingly:

-huey: ./manage.py run_huey --flush-locks --huey-verbose
+huey: ./bin/huey-isnt-running.sh && ./manage.py run_huey --flush-locks --huey-verbose

There really isn't much rocket science or brain surgery about this blog post but I hope it inspires someone who's been in similar trenches that a simple bash script can make all the difference.

So recently, in MDN, we changed the setting WELCOME_EMAIL_FROM. Seems harmless right? Wrong, it failed horribly in runtime and we didn't notice until it was in production. Here's the traceback:

SMTPSenderRefused: (552, b"5.1.7 The sender's address was syntactically invalid.\n5.1.7 see : http://support.socketlabs.com/kb/84 for more information.", '=?utf-8?q?Janet?=')
(8 additional frame(s) were not displayed)
...
  File "newrelic/api/function_trace.py", line 151, in literal_wrapper
    return wrapped(*args, **kwargs)
  File "django/core/mail/message.py", line 291, in send
    return self.get_connection(fail_silently).send_messages([self])
  File "django/core/mail/backends/smtp.py", line 110, in send_messages
    sent = self._send(message)
  File "django/core/mail/backends/smtp.py", line 126, in _send
    self.connection.sendmail(from_email, recipients, message.as_bytes(linesep='\r\n'))
  File "python3.8/smtplib.py", line 871, in sendmail
    raise SMTPSenderRefused(code, resp, from_addr)

SMTPSenderRefused: (552, b"5.1.7 The sender's address was syntactically invalid.\n5.1.7 see : http://support.socketlabs.com/kb/84 for more information.", '=?utf-8?q?Janet?=')

Yikes!

So, to prevent this from happening every again we're putting this check in:

from email.utils import parseaddr

WELCOME_EMAIL_FROM = config("WELCOME_EMAIL_FROM", ...)

# If this fails, SMTP will probably also fail.
assert parseaddr(WELCOME_EMAIL_FROM)[1].count('@') == 1, parseaddr(WELCOME_EMAIL_FROM)

You could go to town even more on this. Perhaps use the email validator within django but for now I'd call that overkill. This is just a decent check before anything gets a chance to go wrong.

tl;dr; I wanted to see which is fastest, in Node, Highlight.js or Prism. The result is; they're both plenty fast but Prism is 9% faster.

The context is all the thousands of little snippets of CSS, HTML, and JavaScript code on MDN.
I first wrote a script that stored almost 9,000 snippets of code. 60% is Javascript and 22% is CSS and rest is HTML.
The mean snippet size was 400 bytes and the median 300 bytes. All ASCII.

Then I wrote three functions:

1. f1 - opens the snippet, extracts the payload, and saves it in a different place. This measures the baseline for how long the disk I/O read and the disk I/O write takes.
2. f2 - same as f1 but uses const html = Prism.highlight(payload, Prism.languages[name], name); before saving.
3. f3 - same as f1 but uses const html = hljs.highlight(name, payload).value; before saving.

The experiment

You can see the hacky benchmark code here: https://github.com/peterbe/syntax-highlight-node-benchmark/blob/master/index.js

Results

The results are (after running each 12 times each):

f1 0.947s   fastest
f2 1.361s   43.6% slower
f3 1.494s   57.7% slower

Memory

In terms of memory usage, Prism maxes heap memory at 60MB (the f1 baseline was 18MB), and Highlight.js maxes heap memory at 60MB too.

Disk space in HTML

Each library produces different HTML. Examples:

Prism

<span class="token selector">.item::after</span> <span class="token punctuation">{</span>
    <span class="token property">content</span><span class="token punctuation">:</span> <span class="token string">"This is my content."</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>

Highlight.js

<span class="hljs-selector-class">.item</span><span class="hljs-selector-pseudo">::after</span> {
    <span class="hljs-attribute">content</span>: <span class="hljs-string">"This is my content."</span>;
}

Yes, not only does it mean they look different, they use up a different amount of disk space when saved. That matters for web performance and also has an impact on build resources.

• f1 - baseline "HTML" files amounts to 11.9MB (across 3,025 files)
• f2 - Prism: 17.6MB
• f3 - Highlight.js: 13.6MB

Conclusion

Prism is plenty fast for Node. If you're already using Prism, don't worry about having to switch to Highlight.js for added performance.

RAM memory consumption is about the same.

Final HTML from Prism is 30% larger than Highlight.js but when the rendered HTML is included in a full HTML page, the HTML compresses very well because of all the repetition so this is not a good comparison. Or rather, not a lot to worry about.

Well, speed is just one dimension. The features differ too. MDN already uses Prism but does so in the browser. The ultimate context for this blog post is; the speed if we were to do all the syntax highlighting in the server as a build step.

Did you know, if you can create your own new Error instance and attach your own custom properties on that? This can come in very handy when you, from the caller, want to get more structured information from the error without relying on the error message.

// WRONG ⛔️

try {
  for (const i of [...Array(10000).keys()]) {
    if (Math.random() > 0.999) {
      throw new Error(`Failed at ${i}`);
    }
  }
} catch (err) {
  const iteration = parseInt(err.toString().match(/Failed at (\d+)/)[1]);
  console.warn(`Made it to ${iteration}`);
}

// RIGHT ✅

try {
  for (const i of [...Array(10000).keys()]) {
    if (Math.random() > 0.999) {
      const failure = new Error(`Failed at ${i}`);
      failure.iteration = i;
      throw failure;
    }
  }
} catch (err) {
  const iteration = err.iteration;
  console.warn(`Made it to ${iteration}`);
}

The above examples are obviously a bit contrived but you have to imagine that whatever code can throw an error might be "far away" from where you deal with errors thrown. For example, imagine you start off a build and you want to get extra information about what the context was. In Python, you use exception classes as a form of natural filtering but JavaScript doesn't have that. Using custom error properties can be a great tool to separate unexpected errors from expected errors.

Bonus - Checking for the custom property

Imagine this refactoring:

try {
  for (const i of [...Array(10000).keys()]) {
    if (Math.random() > 0.999) {
      const failure = new Error(`Failed at ${i}`);
      failure.iteration = i;
      throw failure;
    }
    if (Math.random() < 0.001) {
      throw new Error("something else is wrong");
    }
  }
} catch (err) {
  const iteration = err.iteration;
  console.warn(`Made it to ${iteration}`);
}

With that code it's very possible you'd get Made it to undefined. So here's how you'd make the distinction:

try {
  for (const i of [...Array(10000).keys()]) {
    if (Math.random() > 0.999) {
      const failure = new Error(`Failed at ${i}`);
      failure.iteration = i;
      throw failure;
    }
    if (Math.random() < 0.001) {
      throw new Error("something else is wrong");
    }
  }
} catch (err) {
  if (err.hasOwnProperty("iteration")) {
    const iteration = err.iteration;
    console.warn(`Made it to ${iteration}`);
  } else {
    throw err;
  }
}

```