12 May 2020 0 comments Node, JavaScript
Did you know, if you can create your own new Error
instance and attach your own custom properties on that? This can come in very handy when you, from the caller, want to get more structured information from the error without relying on the error message.
// WRONG ⛔️
try {
for (const i of [...Array(10000).keys()]) {
if (Math.random() > 0.999) {
throw new Error(`Failed at ${i}`);
}
}
} catch (err) {
const iteration = parseInt(err.toString().match(/Failed at (\d+)/)[1]);
console.warn(`Made it to ${iteration}`);
}
// RIGHT ✅
try {
for (const i of [...Array(10000).keys()]) {
if (Math.random() > 0.999) {
const failure = new Error(`Failed at ${i}`);
failure.iteration = i;
throw failure;
}
}
} catch (err) {
const iteration = err.iteration;
console.warn(`Made it to ${iteration}`);
}
The above examples are obviously a bit contrived but you have to imagine that whatever code can throw an error might be "far away" from where you deal with errors thrown. For example, imagine you start off a build and you want to get extra information about what the context was. In Python, you use exception classes as a form of natural filtering but JavaScript doesn't have that. Using custom error properties can be a great tool to separate unexpected errors from expected errors.
Bonus - Checking for the custom property
Imagine this refactoring:
try {
for (const i of [...Array(10000).keys()]) {
if (Math.random() > 0.999) {
const failure = new Error(`Failed at ${i}`);
failure.iteration = i;
throw failure;
}
if (Math.random() < 0.001) {
throw new Error("something else is wrong");
}
}
} catch (err) {
const iteration = err.iteration;
console.warn(`Made it to ${iteration}`);
}
With that code it's very possible you'd get Made it to undefined. So here's how you'd make the distinction:
try {
for (const i of [...Array(10000).keys()]) {
if (Math.random() > 0.999) {
const failure = new Error(`Failed at ${i}`);
failure.iteration = i;
throw failure;
}
if (Math.random() < 0.001) {
throw new Error("something else is wrong");
}
}
} catch (err) {
if (err.hasOwnProperty("iteration")) {
const iteration = err.iteration;
console.warn(`Made it to ${iteration}`);
} else {
throw err;
}
}
```
24 April 2020 0 comments Web development, Node, JavaScript
minimalcss requires that you have your HTML in a serving HTTP web page so that puppeteer
can open it to find out the CSS within. Suppose, in your build system, you don't yet really have a server. Well, what you can do is start one on-the-fly and shut it down as soon as you're done.
Suppose you have .html file
First install all the stuff:
yarn add minimalcss http-server
Then run it:
const path = require("path");
const minimalcss = require("minimalcss");
const httpServer = require("http-server");
const HTML_FILE = "index.html"; // THIS IS YOURS
(async () => {
const server = httpServer.createServer({
root: path.dirname(path.resolve(HTML_FILE)),
});
server.listen(8080);
let result;
try {
result = await minimalcss.minimize({
urls: ["http://0.0.0.0:8080/" + HTML_FILE],
});
} catch (err) {
console.error(err);
throw err;
} finally {
server.close();
}
console.log(result.finalCss);
})();
And the index.html file:
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" href="styles.css">
</head>
<body>
<p>Hi @peterbe</p>
</body>
</html>
And the styles.css file:
h1 {
color: red;
}
p,
li {
font-weight: bold;
}
And the output from running that Node script:
p{font-weight:700}It works!
Suppose all you have is the HTML string and the CSS blob(s)
Suppose all you have is a string of HTML and a list of strings of CSS:
const fs = require("fs");
const path = require("path");
const minimalcss = require("minimalcss");
const httpServer = require("http-server");
const HTML_BODY = `
<p>Hi Peterbe</p>
`;
const CSSes = [
`
h1 {
color: red;
}
p,
li {
font-weight: bold;
}
`,
];
(async () => {
const csses = CSSes.map((css, i) => {
fs.writeFileSync(`${i}.css`, css);
return `<link rel="stylesheet" href="${i}.css">`;
});
const html = `<!doctype html><html>
<head>${csses}</head>
<body>${HTML_BODY}</body>
</html>`;
const fp = path.resolve("./index.html");
fs.writeFileSync(fp, html);
const server = httpServer.createServer({
root: path.dirname(fp),
});
server.listen(8080);
let result;
try {
result = await minimalcss.minimize({
urls: ["http://0.0.0.0:8080/" + path.basename(fp)],
});
} catch (err) {
console.error(err);
throw err;
} finally {
server.close();
fs.unlinkSync(fp);
CSSes.forEach((_, i) => fs.unlinkSync(`${i}.css`));
}
console.log(result.finalCss);
})();
Truth be told, you'll need a good pinch of salt to appreciate that example code. It works but most likely, if you're into web performance so much that you're even doing this, your parameters are likely to be more complex.
Suppose you have your own puppeteer instance
In the first example above, minimalcss will create an instance of puppeteer (e.g. const browser = await puppeteer.launch()) but that means you have less control over which version of puppeteer or which parameters you need. Also, if you have to run minimalcss on a bunch of pages it's costly to have to create and destroy puppeteer browser instances repeatedly.
To modify the original example, here's how you use your own instance of puppeteer:
const path = require("path");
+ const puppeteer = require("puppeteer");
const minimalcss = require("minimalcss");
const httpServer = require("http-server");
const HTML_FILE = "index.html"; // THIS IS YOURS
(async () => {
const server = httpServer.createServer({
root: path.dirname(path.resolve(HTML_FILE)),
});
server.listen(8080);
+ const browser = await puppeteer.launch(/* your special options */);
+
let result;
try {
result = await minimalcss.minimize({
urls: ["http://0.0.0.0:8080/" + HTML_FILE],
+ browser,
});
} catch (err) {
console.error(err);
throw err;
} finally {
+ await browser.close();
server.close();
}
console.log(result.finalCss);
})();
Note that this doesn't buy us anything in this particular example. But that's where your imagination comes in!
Conclusion
You can see the code here as a git repo if that helps.
The point is that this might solve some of the chicken-and-egg problem you might have is that you're building your perfect HTML + CSS and you want to perfect it before you ship it.
Note also that there are other ways to run minimalcss other than programmatically. For example, minimalcss-server is minimalcss wrapped in a express server.
Another thing that you might have is that you have multiple .html files that you want to process. The same technique applies but you just need to turn it into a loop and make sure you call server.close()
(and optionally await browser.close()) when you know you've processed the last file. Exercise left to the reader?
15 April 2020 0 comments Node, JavaScript
tl;dr; No need install or require body-parser and it's important to send the right content-type header.
I know Express has great documentation but I'm still confused about how to receive JSON and/or how to test it from curl. A great deal of confusion comes from the fact that, I think, body-parser
used to be a third-party library you had to install yourself to add it to your Express app. You don't. It now gets installed by installing express. E.g.
▶ yarn init -y
▶ yarn add express
▶ ls node_modules/body-parser
HISTORY.md LICENSE README.md index.js lib package.json
Let's work backward. This is how you set up the Express handler:
const express = require("express"); // v4.17.x as of Apr 2020
const app = express();
app.use(express.json());
app.post("/echo", (req, res) => {
res.json(req.body);
});
app.listen(5000);
And, this is how you test it:
▶ curl -XPOST -d '{"foo": "bar"}' -H 'content-type: application/json' localhost:5000/echo
{"foo":"bar"}%That's it. No need to require("body-parser") or anything like that. And make sure you're sending the content-type: application/json in the curl command.
Things that can go wrong
I kept fumbling around on StackOverflow questions and rummaging the Express documentation until I figured out what mistake I kept doing. So, here's a variant of the handler above, but much more verbose:
app.post("/echo", (req, res) => {
if (req.body === undefined) {
throw new Error("express.json middleware not installed");
}
if (!Object.keys(req.body).length) {
// E.g curl -v -XPOST http://localhost:5000/echo
if (!req.get("content-Type")) {
return res.status(400).send("no content-type header\n");
}
// E.g. curl -v -XPOST -d '{"foo": "bar"}' http://localhost:5000/echo
if (!req.get("content-Type").includes("application/json")) {
return res.status(400).send("content-type not application/json\n");
}
// E.g. curl -XPOST -H 'content-type:application/json' http://localhost:5000/echo
return res.status(400).send("no data payload included\n");
}
// At this point 'req.body' is *something*.
// For example, you might want to `console.log(req.body.foo)`
res.json(req.body);
});
How you treat these things is up to you. For example, an empty JSON data might be OK in your application.
I.e. perhaps curl -XPOST -d '{}' -H 'content-type:application/json' http://localhost:5000/echo might be fine.
An important option
express.json() is a piece of middleware. By default, it has a simple mechanism for bothering to do put .body into the request object. The default configuration is as if you'd typed:
app.use(express.json({
type: 'application/json',
}));
(it's actually a bit more complicated than that)
If you're confident that you'll always be sending JSON to this handler, and you don't want to have to force clients to have to specify the application/json Content-Type you can change this to:
app.use(express.json({
type: '*/*',
}));Now you'll find that curl -XPOST -d '{"foo": "bar3"}' localhost:5000/ will work fine.
Instead of curl, let's fetch
This code works the same with node-fetch or browser Fetch API.
fetch("http://localhost:5000/echo", {
method: "post",
body: JSON.stringify({ foo: "bar" }),
headers: { "Content-Type": "application/json" },
})
.then((res) => res.json())
.then((json) => console.log(json));
03 February 2020 0 comments Node, JavaScript
I'm working on a Node project that involves large transformations of large sets of data here and there. For example:
if (!Object.keys(this.allTitles).length) {
...
In my case, that this.allTitles is a plain object with about 30,000 key/value pairs. That particular line of code actually only runs 1 single time so if it's hundreds of milliseconds, it's really doesn't matter that much. However, that's not a guarantee! What if you had something like this:
for (const thing of things) {
if (!Object.keys(someObj).length) {
// mutate someObj
}
}
then, you'd potentially have a performance degradation once someObj becomes considerably large. And it gets particularly degraded if the length of things is considerably large as it would do the operation many times.
Actually, consider this:
const obj = {};
[...Array(30000)].forEach((_, i) => {
obj[i] = i;
});
console.time("Truthcheck obj");
[...Array(100)].forEach((_, i) => {
return !!Object.keys(obj).length;
});
console.timeEnd("Truthcheck obj");
On my macBook with Node 13.5, this outputs:
Truthcheck obj: 260.564ms
Maps
The MDN page on Map has a nice comparison, in terms of performance, between Map and regular object. Consider this super simple benchmark:
const obj = {};
const map = new Map();
[...Array(30000)].forEach((_, i) => {
obj[i] = i;
map.set(i, i);
});
console.time("Truthcheck obj");
[...Array(100)].forEach((_, i) => {
return !!Object.keys(obj).length;
});
console.timeEnd("Truthcheck obj");
console.time("Truthcheck map");
[...Array(100)].forEach((_, i) => {
return !!map.size;
});
console.timeEnd("Truthcheck map");
So, fill a Map instance and a plain object with 30,000 keys and values. Then, for each in turn, check if the thing is truthy 100 times. The output I get:
Truthcheck obj: 235.017ms
Truthcheck map: 0.029ms
That's not unexpected. The map instance maintains a size counter, which increments on .set (if the key is new), so doing that "truthy" check just takes O(1) seconds.
Conclusion
Don't run to rewrite everything to Maps!
In fact, I took the above mentioned little benchmark and changed the times to be a 3,000 item map and obj (instead of 30,000) and only did 10 iterations (instead of 100) and then the numbers are:
Truthcheck obj: 0.991ms
Truthcheck map: 0.044ms
These kinds of small numbers are very unlikely to matter in the scope of other things going on.
Anyway, consider using Map if you fear that you might be working with really reeeeally large mappings.
18 January 2020 0 comments Python, JavaScript
In Python
I'm sure it's been blogged about a buncha times before but, I couldn't find it, and I had to search too hard to find an example of this. Basically, what I'm trying to do is what Python does in this case, but in JavaScript:
def do_something(arg="notset", **kwargs):
print(f"arg='{arg.upper()}'")
do_something(arg="peter")
do_something(something="else")
do_something()
In Python, the output of all this is:
arg='PETER'
arg='NOTSET'
arg='NOTSET'
It could also have been implemented in a more verbose way:
def do_something(**kwargs):
arg = kwargs.get("arg", "notset")
print(f"arg='{arg.upper()}'")
This more verbose format has the disadvantage that you can't quickly skim it and see and what the default is. That thing (arg = kwargs.get("arg", "notset")) might happen far away deeper in the function, making it hard work to spot the default.
In JavaScript
Here's the equivalent in JavaScript (ES6?):
function doSomething({ arg = "notset", ...kwargs } = {}) {
return `arg='${arg.toUpperCase()}'`;
}
console.log(doSomething({ arg: "peter" }));
console.log(doSomething({ something: "else" }));
console.log(doSomething());
Same output as in Python:
arg='PETER'
arg='NOTSET'
arg='NOTSET'
Notes
I'm still not convinced I like this syntax. It feels a bit too "hip" and too one-liner'y. But it's also pretty useful.
Mind you, the examples here are contrived because they're so short in terms of the number of arguments used in the function.
A more realistic thing like be a function that lists, upfront, all the possible parameters and for some of them, it wants to point out some defaults. E.g.
function processFolder({
source,
destination = "/tmp",
quiet = false,
verbose = false
} = {}) {
console.log({ source, destination, quiet, verbose });
// outputs
// { source: '/user', destination: '/tmp', quiet: true, verbose: false }
}
console.log(processFolder({ source: "/user", quiet: true }));
One could maybe argue that arguments that don't have a default are expected to always be supplied so they can be regular arguments like:
function processFolder(source, {
destination = "/tmp",
quiet = false,
verbose = false
} = {}) {
console.log({ source, destination, quiet, verbose });
// outputs
// { source: '/user', destination: '/tmp', quiet: true, verbose: false }
}
console.log(processFolder("/user", { quiet: true }));
But, I quite like keeping all arguments in an object. It makes it easier to write wrapper functions and I find this:
setProfile(
"My biography here",
false,
193.5,
230,
["anders", "bengt"],
"South Carolina"
);
...harder to read than...
setProfile({
bio: "My biography here",
dead: false,
height: 193.5,
weight: 230,
middlenames: ["anders", "bengt"],
state: "South Carolina"
});
15 January 2020 0 comments JavaScript
Suppose that you're working on ~/dev/my-cool-project and inside ~/dev/my-cool-project/package.json you might have something like this:
"dependencies": {
"that-cool-lib": "1.2.3",
...But that that-cool-lib is one of your own projects. You're also working on that project and it's over at ~/dev/that-cool-lib. Within that-cool-lib you might be in a git branch or perhaps you're preparing a 2.0.0 release.
Now you're interested if that-cool-lib@2.0.0
is going to work here inside my-cool-project.
What you could do
First, you release this fancy that-cool-lib@2.0.0 to npmjs.com with that project's npm publish procedure. Then as soon as that's done and you can see that the release made it onto https://www.npmjs.com/package/that-cool-lib/v/2.0.0.
Then you go over to my-cool-project and start a new git branch to try the upgrade, npm install that-cool-project@2.0.0 --save so you have this:
"dependencies": {
- "that-cool-lib": "1.2.3",
+ "that-cool-lib": "2.0.0",
...Now you can try it that new version of my-cool-project and if that-cool-lib had any of its own entry point executables or post/pre install steps, they'd be fully resolved.
What you should do
Instead, use install-local. Don't use npm link because it might not install entry point executables and I also don't like the fact that I need to go into that-cool-lib and install it (globally?) first (when you do cd that-cool-lib && npm link). Also, see "What's wrong with npm-link?".
Here's how you do it:
npx install-local ~/dev/that-cool-lib
and it acts pretty much exactly as if you had gotten it from npmjs.com the normal way.
Notes
I almost never use npm these days. Go yarn! So, perhaps I've misinterpreted something.
Also, I try my very hardest to never use npm install -g ... (or yarn global ... for that matter) now that we have npx
. Perhaps if you'd install it locally it'd speed up the use of local-install by 1-3 seconds each time you run this. Again, my skillset of modern npm is fading so I don't think I understand why it takes me 14 seconds the first time I run npx install that-cool-lib and then it takes 14 seconds again when I run the exact same command again. Does it not benefit from any caching? How much of that time is spent on npmjs.com resolving other sub-dependencies that that-cool-lib requires?
Hopefully, this helps other people stuck in a similar boat.
