10 September 2020 0 comments Node, JavaScript
To transpile .scss (or .sass) in Node you have the choice between sass and node-sass. sass
is a JavaScript compilation of Dart Sass which is supposedly "the primary implementation of Sass" which is a pretty powerful statement. node-sass on the other hand is a wrapper on LibSass which is written in C++. Let's break it down a little bit more.
Speed
node-sass is faster. About 7 times faster. I took all the SCSS files behind the current MDN Web Docs which is fairly large. Transformed into CSS it becomes a ~180KB blob of CSS (92KB when optimized with csso).
Here's my ugly benchmark test which I run about 10 times like this:
node-sass took 101ms result 180kb 92kb
node-sass took 99ms result 180kb 92kb
node-sass took 99ms result 180kb 92kb
node-sass took 100ms result 180kb 92kb
node-sass took 100ms result 180kb 92kb
node-sass took 103ms result 180kb 92kb
node-sass took 102ms result 180kb 92kb
node-sass took 113ms result 180kb 92kb
node-sass took 100ms result 180kb 92kb
node-sass took 101ms result 180kb 92kb
And here's the same thing for sass:
sass took 751ms result 173kb 92kb
sass took 728ms result 173kb 92kb
sass took 728ms result 173kb 92kb
sass took 798ms result 173kb 92kb
sass took 854ms result 173kb 92kb
sass took 726ms result 173kb 92kb
sass took 727ms result 173kb 92kb
sass took 782ms result 173kb 92kb
sass took 834ms result 173kb 92kb
In another example, I ran sass and node-sass on ./node_modules/bootstrap/scss/bootstrap.scss (version 5.0.0-alpha1) and the results are after 5 runs:
node-sass took 269ms result 176kb 139kb
node-sass took 260ms result 176kb 139kb
node-sass took 288ms result 176kb 139kb
node-sass took 261ms result 176kb 139kb
node-sass took 260ms result 176kb 139kb
versus
sass took 1423ms result 176kb 139kb
sass took 1350ms result 176kb 139kb
sass took 1338ms result 176kb 139kb
sass took 1368ms result 176kb 139kb
sass took 1467ms result 176kb 139kb
Output
The unminified CSS difference primarily in the indentation. But you minify both outputs and the pretty print them (with prettier) you get the following difference:
▶ diff /tmp/sass.min.css.pretty /tmp/node-sass.min.css.pretty
152c152
< letter-spacing: -0.0027777778rem;
---
> letter-spacing: -0.00278rem;
231c231
< content: "▼︎";
---
> content: "\25BC\FE0E";
...snip...
2804c2812
< .external-icon:not([href^="https://mdn.mozillademos.org"]):not(.ignore-external) {
---
> .external-icon:not([href^='https://mdn.mozillademos.org']):not(.ignore-external) {
Basically, sass will use produce things like letter-spacing: -0.0027777778rem; and content: "▼︎";. And node-sass will produce letter-spacing: -0.00278rem; and content: "\25BC\FE0E";.
I also noticed some minor difference just in the order of some selectors but when I look more carefully, they're immaterial order differences meaning they're not cascading each other in any way.
Note! I don't know why the use of ' and " is different or if it matters. I don't know know why prettier (version 2.1.1) didn't pick one over the other consistently.
node_modules
Here's how I created two projects to compare
cd /tmp
mkdir just-sass && cd just-sass && yarn init -y && time yarn add sass && cd ..
mkdir just-node-sass && cd just-node-sass && yarn init -y && time yarn add node-sass && cd ..
Considering that sass is just a JavaScript compilation of a Dart program, all you get is basically a 3.6MB node_modules/sass/sass.dart.js file.
The /tmp/just-sass/node_modules directory is only 113 files and folders weighing a total of 4.1MB.
Whereas /tmp/just-node-sass/node_modules
directory is 3,658 files and folders weighing a total of 15.2MB.
I don't know about you but I'm very skeptical that node-gyp ever works. Who even has Python 2.7 installed anymore? Being able to avoid node-gyp seems like a win for sass.
Conclusion
The speed difference may or may not matter. If you're only doing it once, who cares about a couple of hundred milliseconds. But if you're forced to have to wait 1.4 seconds on every Ctrl-S when Webpack or whatever tooling you have starts up sass it might become very painful.
I don't know much about the sass-loader Webpack plugin but it apparently works with either but they do recommend sass in their documentation. And it's the default implementation too.
It's definitely a feather in sass
's hat that Dart Sass is the "primary implementation" of Sass. That just has a nice feelin in sass's favor.
Bonus
NPMCompare has a nice comparison of them as projects but you have to study each row of numbers because it's rarely as simple as more (or less) number is better. For example, the number of open issues isn't a measure of bugs.
The new module system launched in October 2019 supposedly only comes to Dart Sass which means sass is definitely going to get it first. If that stuff matters to you. For example, true, the Sass unit-testing tool, now requires Dart Sass and drops support for node-sass.
22 June 2020 0 comments Node, JavaScript
I need this function to relate to open-editor which is a Node program that can open your $EDITOR from Node and jump to a specific file, to a specific line, to a specific column.
Here's the code:
function* findMatchesInText(needle, haystack, { inQuotes = false } = {}) {
const escaped = needle.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
let rex;
if (inQuotes) {
rex = new RegExp(`['"](${escaped})['"]`, "g");
} else {
rex = new RegExp(`(${escaped})`, "g");
}
for (const match of haystack.matchAll(rex)) {
const left = haystack.slice(0, match.index);
const line = (left.match(/\n/g) || []).length + 1;
const lastIndexOf = left.lastIndexOf("\n") + 1;
const column = match.index - lastIndexOf + 1;
yield { line, column };
}
}
And you use it like this:
const text = ` bravo
Abra
cadabra
bravo
`;
console.log(Array.from(findMatchesInText("bra", text)));
Which prints:
[
{ line: 1, column: 2 },
{ line: 2, column: 2 },
{ line: 3, column: 5 },
{ line: 5, column: 1 }
]
The inQuotes option is because a lot of times this function is going to be used for finding the href value in unstructured documents that contain HTML <a> tags.
19 May 2020 0 comments Node, JavaScript
tl;dr; I wanted to see which is fastest, in Node, Highlight.js or Prism. The result is; they're both plenty fast but
Prism is 9% faster.
The context is all the thousands of little snippets of CSS, HTML, and JavaScript code on MDN.
I first wrote a script that stored almost 9,000 snippets of code. 60% is Javascript and 22% is CSS and rest is HTML.
The mean snippet size was 400 bytes and the median 300 bytes. All ASCII.
Then I wrote three functions:
f1 - opens the snippet, extracts the payload, and saves it in a different place. This measures the baseline for how long the disk I/O read and the disk I/O write takes.f2 - same as f1 but uses const html = Prism.highlight(payload, Prism.languages[name], name); before saving.f3 - same as f1 but uses const html = hljs.highlight(name, payload).value; before saving.
The experiment
You can see the hacky benchmark code here: https://github.com/peterbe/syntax-highlight-node-benchmark/blob/master/index.js
Results
The results are (after running each 12 times each):
f1 0.947s fastest
f2 1.361s 43.6% slower
f3 1.494s 57.7% slower
Memory
In terms of memory usage, Prism maxes heap memory at 60MB (the f1 baseline was 18MB), and Highlight.js maxes heap memory at 60MB too.
Disk space in HTML
Each library produces different HTML. Examples:
Prism
<span class="token selector">.item::after</span> <span class="token punctuation">{</span>
<span class="token property">content</span><span class="token punctuation">:</span> <span class="token string">"This is my content."</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
Highlight.js
<span class="hljs-selector-class">.item</span><span class="hljs-selector-pseudo">::after</span> {
<span class="hljs-attribute">content</span>: <span class="hljs-string">"This is my content."</span>;
}
Yes, not only does it mean they look different, they use up a different amount of disk space when saved. That matters for web performance and also has an impact on build resources.
f1 - baseline "HTML" files amounts to 11.9MB (across 3,025 files)f2 - Prism: 17.6MBf3 - Highlight.js: 13.6MB
Conclusion
Prism is plenty fast for Node. If you're already using Prism, don't worry about having to switch to Highlight.js for added performance.
RAM memory consumption is about the same.
Final HTML from Prism is 30% larger than Highlight.js but when the rendered HTML is included in a full HTML page, the HTML compresses very well because of all the repetition so this is not a good comparison. Or rather, not a lot to worry about.
Well, speed is just one dimension. The features differ too. MDN
already uses Prism but does so in the browser. The ultimate context for this blog post is; the speed if we were to do all the syntax highlighting in the server as a build step.
12 May 2020 0 comments Node, JavaScript
Did you know, if you can create your own new Error instance and attach your own custom properties on that? This can come in very handy when you, from the caller, want to get more structured information from the error without relying on the error message.
// WRONG ⛔️
try {
for (const i of [...Array(10000).keys()]) {
if (Math.random() > 0.999) {
throw new Error(`Failed at ${i}`);
}
}
} catch (err) {
const iteration = parseInt(err.toString().match(/Failed at (\d+)/)[1]);
console.warn(`Made it to ${iteration}`);
}
// RIGHT ✅
try {
for (const i of [...Array(10000).keys()]) {
if (Math.random() > 0.999) {
const failure = new Error(`Failed at ${i}`);
failure.iteration = i;
throw failure;
}
}
} catch (err) {
const iteration = err.iteration;
console.warn(`Made it to ${iteration}`);
}
The above examples are obviously a bit contrived but you have to imagine that whatever code can throw an error might be "far away" from where you deal with errors thrown. For example, imagine you start off a build and you want to get extra information about what the context was. In Python, you use exception classes as a form of natural filtering but JavaScript doesn't have that. Using custom error properties can be a great tool to separate unexpected errors from expected errors.
Bonus - Checking for the custom property
Imagine this refactoring:
try {
for (const i of [...Array(10000).keys()]) {
if (Math.random() > 0.999) {
const failure = new Error(`Failed at ${i}`);
failure.iteration = i;
throw failure;
}
if (Math.random() < 0.001) {
throw new Error("something else is wrong");
}
}
} catch (err) {
const iteration = err.iteration;
console.warn(`Made it to ${iteration}`);
}
With that code it's very possible you'd get Made it to undefined. So here's how you'd make the distinction:
try {
for (const i of [...Array(10000).keys()]) {
if (Math.random() > 0.999) {
const failure = new Error(`Failed at ${i}`);
failure.iteration = i;
throw failure;
}
if (Math.random() < 0.001) {
throw new Error("something else is wrong");
}
}
} catch (err) {
if (err.hasOwnProperty("iteration")) {
const iteration = err.iteration;
console.warn(`Made it to ${iteration}`);
} else {
throw err;
}
}
```
24 April 2020 0 comments Web development, Node, JavaScript
minimalcss requires that you have your HTML in a serving HTTP web page so that puppeteer
can open it to find out the CSS within. Suppose, in your build system, you don't yet really have a server. Well, what you can do is start one on-the-fly and shut it down as soon as you're done.
Suppose you have .html file
First install all the stuff:
yarn add minimalcss http-server
Then run it:
const path = require("path");
const minimalcss = require("minimalcss");
const httpServer = require("http-server");
const HTML_FILE = "index.html"; // THIS IS YOURS
(async () => {
const server = httpServer.createServer({
root: path.dirname(path.resolve(HTML_FILE)),
});
server.listen(8080);
let result;
try {
result = await minimalcss.minimize({
urls: ["http://0.0.0.0:8080/" + HTML_FILE],
});
} catch (err) {
console.error(err);
throw err;
} finally {
server.close();
}
console.log(result.finalCss);
})();
And the index.html file:
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" href="styles.css">
</head>
<body>
<p>Hi @peterbe</p>
</body>
</html>
And the styles.css file:
h1 {
color: red;
}
p,
li {
font-weight: bold;
}
And the output from running that Node script:
p{font-weight:700}It works!
Suppose all you have is the HTML string and the CSS blob(s)
Suppose all you have is a string of HTML and a list of strings of CSS:
const fs = require("fs");
const path = require("path");
const minimalcss = require("minimalcss");
const httpServer = require("http-server");
const HTML_BODY = `
<p>Hi Peterbe</p>
`;
const CSSes = [
`
h1 {
color: red;
}
p,
li {
font-weight: bold;
}
`,
];
(async () => {
const csses = CSSes.map((css, i) => {
fs.writeFileSync(`${i}.css`, css);
return `<link rel="stylesheet" href="${i}.css">`;
});
const html = `<!doctype html><html>
<head>${csses}</head>
<body>${HTML_BODY}</body>
</html>`;
const fp = path.resolve("./index.html");
fs.writeFileSync(fp, html);
const server = httpServer.createServer({
root: path.dirname(fp),
});
server.listen(8080);
let result;
try {
result = await minimalcss.minimize({
urls: ["http://0.0.0.0:8080/" + path.basename(fp)],
});
} catch (err) {
console.error(err);
throw err;
} finally {
server.close();
fs.unlinkSync(fp);
CSSes.forEach((_, i) => fs.unlinkSync(`${i}.css`));
}
console.log(result.finalCss);
})();
Truth be told, you'll need a good pinch of salt to appreciate that example code. It works but most likely, if you're into web performance so much that you're even doing this, your parameters are likely to be more complex.
Suppose you have your own puppeteer instance
In the first example above, minimalcss will create an instance of puppeteer (e.g. const browser = await puppeteer.launch()) but that means you have less control over which version of puppeteer or which parameters you need. Also, if you have to run minimalcss on a bunch of pages it's costly to have to create and destroy puppeteer browser instances repeatedly.
To modify the original example, here's how you use your own instance of puppeteer:
const path = require("path");
+ const puppeteer = require("puppeteer");
const minimalcss = require("minimalcss");
const httpServer = require("http-server");
const HTML_FILE = "index.html"; // THIS IS YOURS
(async () => {
const server = httpServer.createServer({
root: path.dirname(path.resolve(HTML_FILE)),
});
server.listen(8080);
+ const browser = await puppeteer.launch(/* your special options */);
+
let result;
try {
result = await minimalcss.minimize({
urls: ["http://0.0.0.0:8080/" + HTML_FILE],
+ browser,
});
} catch (err) {
console.error(err);
throw err;
} finally {
+ await browser.close();
server.close();
}
console.log(result.finalCss);
})();
Note that this doesn't buy us anything in this particular example. But that's where your imagination comes in!
Conclusion
You can see the code here as a git repo if that helps.
The point is that this might solve some of the chicken-and-egg problem you might have is that you're building your perfect HTML + CSS and you want to perfect it before you ship it.
Note also that there are other ways to run minimalcss other than programmatically. For example, minimalcss-server is minimalcss wrapped in a express server.
Another thing that you might have is that you have multiple .html files that you want to process. The same technique applies but you just need to turn it into a loop and make sure you call server.close()
(and optionally await browser.close()) when you know you've processed the last file. Exercise left to the reader?
15 April 2020 0 comments Node, JavaScript
tl;dr; No need install or require body-parser and it's important to send the right content-type header.
I know Express has great documentation but I'm still confused about how to receive JSON and/or how to test it from curl. A great deal of confusion comes from the fact that, I think, body-parser
used to be a third-party library you had to install yourself to add it to your Express app. You don't. It now gets installed by installing express. E.g.
▶ yarn init -y
▶ yarn add express
▶ ls node_modules/body-parser
HISTORY.md LICENSE README.md index.js lib package.json
Let's work backward. This is how you set up the Express handler:
const express = require("express"); // v4.17.x as of Apr 2020
const app = express();
app.use(express.json());
app.post("/echo", (req, res) => {
res.json(req.body);
});
app.listen(5000);
And, this is how you test it:
▶ curl -XPOST -d '{"foo": "bar"}' -H 'content-type: application/json' localhost:5000/echo
{"foo":"bar"}%That's it. No need to require("body-parser") or anything like that. And make sure you're sending the content-type: application/json in the curl command.
Things that can go wrong
I kept fumbling around on StackOverflow questions and rummaging the Express documentation until I figured out what mistake I kept doing. So, here's a variant of the handler above, but much more verbose:
app.post("/echo", (req, res) => {
if (req.body === undefined) {
throw new Error("express.json middleware not installed");
}
if (!Object.keys(req.body).length) {
// E.g curl -v -XPOST http://localhost:5000/echo
if (!req.get("content-Type")) {
return res.status(400).send("no content-type header\n");
}
// E.g. curl -v -XPOST -d '{"foo": "bar"}' http://localhost:5000/echo
if (!req.get("content-Type").includes("application/json")) {
return res.status(400).send("content-type not application/json\n");
}
// E.g. curl -XPOST -H 'content-type:application/json' http://localhost:5000/echo
return res.status(400).send("no data payload included\n");
}
// At this point 'req.body' is *something*.
// For example, you might want to `console.log(req.body.foo)`
res.json(req.body);
});
How you treat these things is up to you. For example, an empty JSON data might be OK in your application.
I.e. perhaps curl -XPOST -d '{}' -H 'content-type:application/json' http://localhost:5000/echo might be fine.
An important option
express.json() is a piece of middleware. By default, it has a simple mechanism for bothering to do put .body into the request object. The default configuration is as if you'd typed:
app.use(express.json({
type: 'application/json',
}));
(it's actually a bit more complicated than that)
If you're confident that you'll always be sending JSON to this handler, and you don't want to have to force clients to have to specify the application/json Content-Type you can change this to:
app.use(express.json({
type: '*/*',
}));Now you'll find that curl -XPOST -d '{"foo": "bar3"}' localhost:5000/ will work fine.
Instead of curl, let's fetch
This code works the same with node-fetch or browser Fetch API.
fetch("http://localhost:5000/echo", {
method: "post",
body: JSON.stringify({ foo: "bar" }),
headers: { "Content-Type": "application/json" },
})
.then((res) => res.json())
.then((json) => console.log(json));
