Peterbe.com

Remember jQuery? Yeah, it was great. But it was also horrible in its own ways but only when compared to the more powerful tools that we have now as of 2021. I still (almost) use it here on my site. Atually, I use "fork" of jQuery called Cash which calls itself: "An absurdly small jQuery alternative for modern browsers."
Cash is written in TypeScript, which gives me peace of mind, and as a JS bundle, it's only 19KB minified (5.3KB Brotli compressed) whereas jQuery is 87KB minified (27KB Brotli compressed).

But something that jQuery has, that Cash doesn't, is animations. E.g. $('myselector').fadeIn(). If you need to do this with Cash you can use the following pure JavaScript solution:

// Example implementation

const msg = $('<div class="message">')
  .text(`Random message: ${Math.random()}`)
  .css("opacity", 0)
  .css("transition", "opacity 600ms")
  .prependTo($("#root"));
setTimeout(() => msg.css("opacity", 1), 0);

setTimeout(() => {
  msg.css("transition", "opacity 1000ms").css("opacity", 0);
  setTimeout(() => msg.remove(), 1000);
}, 3000);

What this application demonstrates is the creation of a <div> that's immediately injected into the DOM but slowly fades into view. And 3 seconds later it fades out and is removed. Full demo/sample application here.

Sample application using cash like jQuery's $.fadeIn().

The point of the demo is how you can cause the fade-in effect with just Cash but still relies on CSS for the actual animation.
The trick is to, ultimately, create it first like this:

<div class="message" style="opacity:0; transition: opacity 600ms">
  Random message: 0.6517198324628395
</div>

and then, right after it's been added to the DOM, change the style=... to:

-<div class="message" style="opacity:0; transition: opacity 600ms">
+<div class="message" style="opacity:1; transition: opacity 600ms">

What's neat about this is that you use the transition shortcut so it's done entirely with CSS instead of a requestAnimationFrame and/or while-loop like jQuery's effects.js does it.

Note! This is not a polyfill since jQuery's fadeIn() (etc.) can do a lot more such as callbacks. The example might not be great but I hope this little solution becomes useful for someone else who needs this.

Like any responsible parent, I get heart palpitations when my kids (and especially their friends) rush through the door, in or out, and just leave the door wide open letting all that sweet sweet air-conditioned coldness rush out like a prison break. My wife gave me these Gibcloser "Safety Door Closer" as a present and we installed them today on all 3 doors (front door, to-basement door, back porch door).

They're ~$18 on Amazon.com and it appears they come in many different colors. Our doors were white so that that's what we used. It took less than 30 minutes to install on all 3 doors. All you need is a screwdriver and, I would suggest, some thin foam command strips.

Door held at 90° and released.

I know it might sound silly but I've wanted this for so long and it never occurred to me that such a simple solution might exist. Clearly, we've all seen door closers like this before, but almost always bigger. Especially on commercial buildings. But they all seem so complicated and expensive-looking and definitely strike fear in you of: "that'd never work on my home doors". What's neat about these is that they are gentle. It's just a gentle (albeit accelerating) push but if someone/something was to get stuck as it closes, it wouldn't chop off a finger or a foot.

Truth be told, they don't close the door all the way. In the above video, it closes nicely because I held the door open at 90° which is a bit unrealistic angle that someone would forget to close the door. But at least it'll close it enough to stop the flow of cold/warm air going through.

Because it was driving me insane, and because I don't want to ever forget...

Playwright is a wonderful alternative to jest-puppeteer for doing automated headless browser end-to-end testing. But one I couldn't find in the documentation, Google search, or Stackoverflow was: How do you submit a form without clicking a button?. I.e. you have focus in an input field and hit Enter. Here's how you do it:

await page.$eval('form[role="search"]', (form) => form.submit());

The first part is any CSS selector that gets you to the <form> element. In this case, imagine it was:

<form action="/search" role="search">
  <input type="search" name="q">
</form>

You, or my future self, might be laughing at me for missing something obvious but this one took me forever to solve so I thought I'd better blog about it in case someone else gets into the same jam.

UPDATE (Sep 2021)

I found a much easier way:

await page.keyboard.press("Enter");

This obviously only works when you've typed something into an input so the focus is on that <input> element. E.g.:

await page.fill('input[aria-label="New shopping list item"]', "Carrots");
await page.keyboard.press("Enter");

We use Poetry in a GitHub project. There's a pyproject.toml file (and a poetry.lock file) which with the help of the executable poetry gets you a very reliable Python environment. The only problem is that adding the poetry executable is slow. Like 10+ seconds slow. It might seem silly but in the project I'm working on, that 10+s delay is the slowest part of a GitHub Action workflow which needs to be fast because it's trying to post a comment on a pull request as soon as it possibly can.

First I tried caching $(pip cache dir) so that the underlying python -v pip install virtualenv -t $tmp_dir that install-poetry.py does would get a boost from avoiding network. The difference was negligible. I also didn't want to get too weird by overriding how the install-poetry.py works or even make my own hacky copy. I like being able to just rely on the snok/install-poetry GitHub Action to do its thing (and its future thing).

The solution was to cache the whole $HOME/.local directory. It's as simple as this:

- name: Load cached $HOME/.local
  uses: actions/cache@v2.1.6
  with:
    path: ~/.local
    key: dotlocal-${{ runner.os }}-${{ hashFiles('.github/workflows/pr-deployer.yml') }}

The key is important. If you do copy-n-paste this block of YAML to speed up your GitHub Action, please remember to replace .github/workflows/pr-deployer.yml with the name of your .yml file that uses this. It's important because otherwise, the cache might be overzealously hot when you make a change like:

       - name: Install Python poetry
-        uses: snok/install-poetry@v1.1.6
+        uses: snok/install-poetry@v1.1.7
         with:

...for example.

Now, thankfully install-poetry.py (which is the recommended way to install poetry by the way) can notice that it's already been created and so it can omit a bunch of work. The result of this is as follows:

From 10+ seconds to 2 seconds. And what's neat is that the optimization is very "unintrusive" because it doesn't mess with how the snok/install-poetry workflow works.

But wait, there's more!

If you dig up our code where we use poetry you might find that it does a bunch of other caching too. In particular, it caches .venv it creates too. That's relevant but ultimately unrelated. It basically caches the generated virtualenv from the poetry install command. It works like this:

- name: Load cached venv
  id: cached-poetry-dependencies
  uses: actions/cache@v2.1.6
  with:
    path: deployer/.venv
    key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}-${{ hashFiles('.github/workflows/pr-deployer.yml') }}

...

- name: Install deployer
  run: |
    cd deployer
    poetry install
  if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'

In this example, deployer is just the name of the directory, in the repository root, where we have all the Python code and the pyproject.toml etc. If you have yours at the root of the project you can just do: run: poetry install and in the caching step change it to: path: .venv.

Now, you get a really powerful complete caching strategy. When the caches are hot (i.e. no changes to the .yml, poetry.lock, or pyproject.toml files) you get the executable (so you can do poetry run ...) and all its dependencies in roughly 2 seconds. That'll be hard to beat!

tl;dr; To make 2 lists into 1 without mutating them use list1 + list2.

I'm blogging about this because today I accidentally complicated my own code. From now on, let's just focus on the right way.

Suppose you have something like this:

winners = [123, 503, 1001]
losers = [45, 812, 332]

combined = winners + losers

that will create a brand new list. To prove that it's immutable:

>>> combined.insert(0, 100)
>>> combined
[100, 123, 503, 1001, 45, 812, 332]
>>> winners
[123, 503, 1001]
>>> losers
[45, 812, 332]

What I originally did was:

winners = [123, 503, 1001]
losers = [45, 812, 332]

combined = [*winners, *losers]

This works the same and that syntax feels very JavaScript'y. E.g.

> var winners = [123, 503, 1001]
[ 123, 503, 1001 ]
> var losers = [45, 812, 332]
[ 45, 812, 332 ]
> var combined = [...winners, ...losers]
[ 123, 503, 1001, 45, 812, 332 ]
> combined.pop()
332
> losers
[ 45, 812, 332 ]

By the way, if you want to filter out duplicates, do this:

>>> a = [1, 2, 3]
>>> b = [2, 3, 4]
>>> list(dict.fromkeys(a + b))
[1, 2, 3, 4]

It's the most performant way to do it if the order is important.

And if you don't care about the order you can use this:

>>> a = [1, 2, 3]
>>> b = [2, 3, 4]
>>> list(set(a + b))
[1, 2, 3, 4]
>>> list(set(b + a))
[1, 2, 3, 4]

tl;dr; git clone https://github.com/mdn/content.git && cd content && yarn install && yarn start && open http://localhost:5000/ will get you all of MDN Web Docs running on your laptop.

The MDN Web Docs is built from a git repository: github.com/mdn/content. It contains all you need to get all the content running locally. Including search. Embedded inside that repository is a package.json which helps you start a Yari server. Aka. the preview server. It's a static build of the github.com/mdn/yari project which handles client-side rendering, search, an just-in-time server-side rendering server.

Basics

All you need is the following:

▶ git clone https://github.com/mdn/content.git
▶ cd content
▶ yarn install
▶ yarn start

And now open http://localhost:5000 in your browser.

This will now run in "preview server" mode. It's meant for contributors (and core writers) to use when they're working on a git branch. Because of that, you'll see a "Writer's homepage" at the root URL. And when viewing each document, you get buttons about "flaws" and stuff. Looks like this:

Alternative ways to download

If you don't want to use git clone you can download the ZIP file. For example:

▶ wget https://github.com/mdn/content/archive/refs/heads/main.zip
▶ unzip main.zip
▶ cd content-main
▶ yarn install
▶ yarn start

At the time of writing, the downloaded Zip file is 86MB and unzipped the directory is 278MB on disk.

When you use git clone, by default it will download all the git history. That can actually be useful. This way, when rendering each document, it can figure out from the git logs when each individual document was last modified. For example:

If you don't care about the "Last modified" date, you can do a "shallow git clone" instead. Replace the above-mentioned first command with:

▶ git clone --depth 1 https://github.com/mdn/content.git

At the time of writing the shallow cloned content folder becomes 234MB instead of (the deep clone) 302MB.

Just the raw rendered data

Every MDN Web Docs page has an index.json equivalent. Take any MDN page and add /index.json to the URL. For example /en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/slice/index.json

Essentially, this is the intermediate state that's used for server-side rendering the page. A glorified way of sandwiching the content in a header, a footer, and a sidebar to the side. These URLs work on localhost:5000 too. Try http://localhost:5000/en-US/docs/Web/API/Fetch_API/Using_Fetch/index.json for example.

The content for that index.json is built just in time. It also contains a bunch of extra metadata about "flaws"; a system used to highlight things that should be fixed that is somewhat easy to automate. So, it doesn't contain things like spelling mistakes or code snippets that are actually invalid.

But suppose you want all that raw (rendered) data, without any of the flaw detections, you can run this command:

▶ BUILD_FLAW_LEVELS="*:ignore" yarn build

It'll take a while (because it produces an index.html file too). But now you have all the index.json files for everything in the newly created ./build/ directory. It should have created a lot of files:

▶ find build -name index.json | wc -l
   11649

If you just want a subtree of files you could have run it like this instead:

▶ BUILD_FOLDERSEARCH=web/javascript BUILD_FLAW_LEVELS="*:ignore" yarn build

Programmatic API access

The programmatic APIs are all about finding the source files. But you can use the sources to turn that into the built files you might need. Or just to get a list of URLs. To get started, create a file called find-files.js in the root:

const { Document } = require("@mdn/yari/content");

console.log(Document.findAll().count);

Now, run it like this:

▶ export CONTENT_ROOT=files

▶ node find-files.js
11649

Other things you can do with that findAll function:

const { Document } = require("@mdn/yari/content");

const found = Document.findAll({
  folderSearch: "web/javascript/reference/statements/f",
});
for (const document of found.iter()) {
  console.log(document.url);
}

Or, suppose you want to actually build each of these that you find:

const { Document } = require("@mdn/yari/content");
const { buildDocument } = require("@mdn/yari/build");

const found = Document.findAll({
  folderSearch: "web/javascript/reference/statements/f",
});

Promise.all([...found.iter()].map((document) => buildDocument(document))).then(
  (built) => {
    for (const { doc } of built) {
      console.log(doc.title.padEnd(20), doc.popularity);
    }
  }
);

That'll output something like this:

▶ node find-files.js
for                  0.0143
for await...of       0.0129
for...in             0.0748
for...of             0.0531
function declaration 0.0088
function*            0.0122

All the HTML content in production-grade mode

In the most basic form, it will start the "preview server" which is tailored towards building just in time and has all those buttons at the top for writers/contributors. If you want the more "production-grade" version, you can't use the copy of @mdn/yari that is "included" in the mdn/content repo. To do this, you need to git clone mdn/yari and install that. Hang on, this is about to get a bit more advanced:

▶ git clone https://github.com/mdn/yari.git
▶ cd yari
▶ yarn install
▶ yarn build:client
▶ yarn build:ssr
▶ CONTENT_ROOT=../files REACT_APP_DISABLE_AUTH=true BUILD_FLAW_LEVELS="*:ignore" yarn build
▶ CONTENT_ROOT=../files node server/static.js

Now, if you go to something like http://localhost:5000/en-US/docs/Web/Guide/ you'll get the same thing as you get on https://developer.mozilla.org but all on your laptop. Should be pretty snappy.

Is it really entirely offline?

No, it leaks a little. For example, there are interactive examples that uses an iframe that's hardcoded to https://interactive-examples.mdn.mozilla.net/.

There are also external images for example. You might get a live sample that refers to sample images on https://mdn.mozillademos.org/files/.... So that'll fail if you're without WiFi in a spaceship.

Conclusion

Making all of MDN Web Docs available offline is, honestly, not a priority. The focus is on A) a secure production build, and B) a good environment for previewing content changes. But all the pieces are there. Search is a little bit tricky, as an example. When you're running it as a preview server you can't do a full-text search on all the content, but you get a useful autocomplete search widget for navigating between different titles. And the full-text search engine is a remote centralized server that you can't take with you offline.

But all the pieces are there. Somehow. It all depends on your use case and what you're willing to "compromise" on.