How to change the current query string URL in NextJS v13 with next/navigation
December 9, 2022
4 comments React, JavaScript
At the time of writing, I don't know if this is the optimal way, but after some trial and error, I got it working.
This example demonstrates a hook that gives you the current value of the ?view=...
(or a default) and a function you can call to change it so that ?view=before
becomes ?view=after
.
In NextJS v13 with the pages
directory:
import { useRouter } from "next/router";
export function useNamesView() {
const KEY = "view";
const DEFAULT_NAMES_VIEW = "buttons";
const router = useRouter();
let namesView: Options = DEFAULT_NAMES_VIEW;
const raw = router.query[KEY];
const value = Array.isArray(raw) ? raw[0] : raw;
if (value === "buttons" || value === "table") {
namesView = value;
}
function setNamesView(value: Options) {
const [asPathRoot, asPathQuery = ""] = router.asPath.split("?");
const params = new URLSearchParams(asPathQuery);
params.set(KEY, value);
const asPath = `${asPathRoot}?${params.toString()}`;
router.replace(asPath, asPath, { shallow: true });
}
return { namesView, setNamesView };
}
In NextJS v13 with the app
directory.
import { useRouter, useSearchParams, usePathname } from "next/navigation";
type Options = "buttons" | "table";
export function useNamesView() {
const KEY = "view";
const DEFAULT_NAMES_VIEW = "buttons";
const router = useRouter();
const searchParams = useSearchParams();
const pathname = usePathname();
let namesView: Options = DEFAULT_NAMES_VIEW;
const value = searchParams.get(KEY);
if (value === "buttons" || value === "table") {
namesView = value;
}
function setNamesView(value: Options) {
const params = new URLSearchParams(searchParams);
params.set(KEY, value);
router.replace(`${pathname}?${params}`);
}
return { namesView, setNamesView };
}
The trick is that you only want to change 1 query string value and respect whatever was there before. So if the existing URL was /page?foo=bar
and you want that to become /page?foo=bar&and=also
you have to consume the existing query string and you do that with:
const searchParams = useSearchParams();
...
const params = new URLSearchParams(searchParams);
params.set('and', 'also')
How much faster is Cheerio at parsing depending on xmlMode?
December 5, 2022
0 comments Node, JavaScript
Cheerio is a fantastic Node library for parsing HTML and then being able to manipulate and serialize it. But you can also just use it for parsing HTML and plucking out what you need. We use that to prepare the text that goes into our search index for our site. It basically works like this:
const body = await getBody('http://localhost:4002' + eachPage.path)
const $ = cheerio.load(body)
const title = $('h1').text()
const intro = $('p.intro').text()
...
But it hit me, can we speed that up? cheerio
actually ships with two different parsers:
One is faster and one is more strict.
But I wanted to see this in a real-world example.
So I made two runs where I used:
const $ = cheerio.load(body)
in one run, and:
const $ = cheerio.load(body, { xmlMode: true })
in another.
After having parsed 1,635 pages of HTML of various sizes the results are:
FILE: load.txt MEAN: 13.19457640586797 MEDIAN: 10.5975 FILE: load-xmlmode.txt MEAN: 3.9020372860635697 MEDIAN: 3.1020000000000003
So, using {xmlMode:true}
leads to roughly a 3x speedup.
I think it pretty much confirms the original benchmark, but now I know based on a real application.
First impressions trying out Rome to format/lint my TypeScript and JavaScript
November 14, 2022
1 comment Node, JavaScript
Rome is a new contender to compete with Prettier and eslint, combined. It's fast and its suggestions are much easier to understand.
I have a project that uses .js
, .ts
, and .tsx
files. At first, I thought, I'd just use rome
to do formatting but the linter part was feeling nice as I was experimenting so I thought I'd kill two birds with one stone.
Things that worked well
It is fast
My little project only has 28 files, but time rome check lib scripts components *.ts
consistently takes 0.08 seconds.
The CLI looks great
You get this nice prompt after running npx rome init
the first time:
Suggestions just look great
Easy to understand and needs no explanation because the suggested fix tells a story that means it's immediately easy to understand what the warning is trying to say.
It is smaller
If I run npx create-next-app@latest
, say yes to Eslint, and then run npm I -D prettier
, the node_modules
becomes 275.3 MiB.
Whereas if I run npx create-next-app@latest
, say no to Eslint, and then run npm I -D rome
, the node_modules
becomes 200.4 MiB.
Editing the rome.json
's JSON schema works in VS Code
I don't know how this magically worked, but I'm guessing it just does when you install the Rome VS Code extension. Neat with autocomplete!
Things that didn't work so well
Almost all things that I'm going to "complain" about is down to usability. I might look back at this in a year (or tomorrow!) and laugh at myself for being dim, but it nevertheless was part of my experience so it's worth pointing out.
Lint, check, or format?
It's confusing what is what. If lint
means checking without modifying, what is check
then? I'm guessing rome format
means run the lint
but with permission to edit my files.
What is rome format
compared to rome check --apply
then??
I guess rome check --apply
doesn't just complain but actually applies the things it spots. So what is rome check --apply-suggested
?? (if you're reading this and feel eager to educate me with a comment, please do, but I'm trying to point out that it's not user-friendly)
How do I specify wildcards?
Unfortunately, in this project, not all files are in one single directory (e.g. rome check src/
is not an option). How do I specify a wildcard expression?
▶ rome check *.ts
Checked 3 files in 942µs
Cool, but how do I do all .ts
files throughout the project?
▶ rome check "**/*.ts"
**/*.ts internalError/io ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✖ No such file or directory (os error 2)
Checked 0 files in 66µs
Clearly, it's not this:
▶ rome check **/*.ts
...
The number of diagnostics exceeds the number allowed by Rome.
Diagnostics not shown: 1018.
Checked 2534 files in 1387ms
Skipped 1 files
Error: errors where emitted while running checks
...because bash will include all the files from node_modules/**/*.ts
.
In the end, I ended up with this (in my package.json
):
"scripts": { "code:lint": "rome check lib scripts components *.ts", ...
There's no documentation about how to ignore certain rules
Yes, I can contribute this back to the documentation, but today's not the day to do that.
It took me a long time to find out how to disable certain rules (in the rome.json
file) and finally I landed on this:
{ "linter": { "enabled": true, "rules": { "recommended": true, "style": { "recommended": true, "noImplicitBoolean": "off" }, "a11y": { "useKeyWithClickEvents": "off", "useValidAnchor": "warn" } } } }
Much better than having to write inline code comments with the source files themselves.
However, it's still not clear to me what "recommended": true
means. Is it shorthand for listing all the default rules all set to true
? If I remove that, are no rules activated?
The rome.json
file is JSON
JSON is cool for many things, but writing comments is not one of them.
For example, I don't know what would be better, Yaml or Toml, but it would be nice to write something like:
"a11y": { # Disabled because of issue #1234 # Consider putting this back in December after the refactor launch "useKeyWithClickEvents": "off",
Nextjs and rome needs to talk
When create-react-app
first came onto the scene, the coolest thing was the zero-config webpack
. But, if you remember, it also came with a really nice zero-config eslint
configuration for React apps. It would even print warnings when the dev server was running. Now it's many years later and good linting config is something you depend/rely on in a framework. Like it or not, there are specific things in Nextjs that is exclusive to that framework. It's obviously not an easy people-problem to solve but it would be nice if Nextjs and rome
could be best friends so you get all the good linting ideas from the code Nextjs framework but all done using rome
instead.
Spot the JavaScript bug with recursion and incrementing
September 28, 2022
0 comments JavaScript
What will this print?
function doSomething(iterations = 0) {
if (iterations < 10) {
console.log("Let's do this again!")
doSomething(iterations++)
}
}
doSomething()
The answer is it will print
Let's do this again! Let's do this again! Let's do this again! Let's do this again! Let's do this again! Let's do this again! Let's do this again! Let's do this again! ...forever...
The bug is the use of a "postfix increment" which is a bug I had in some production code (almost, it never shipped).
The solution is simple:
console.log("Let's do this again!")
- doSomething(iterations++)
+ doSomething(++iterations)
That's called "prefix increment" which means it not only changes the variable but returns what the value became rather than what it was before increment.
The beautiful solution is actually the simplest solution:
console.log("Let's do this again!")
- doSomething(iterations++)
+ doSomething(iterations + 1)
Now, you don't even mutate the value of the iterations
variable but create a new one for the recursion call.
All in all, pretty simple mistake but it can easily happen. Particular if you feel inclined to look cool by using the spiffy ++
shorthand because it looks neater or something.
Programmatically render a NextJS page without a server in Node
September 6, 2022
0 comments Web development, Node, JavaScript
If you use getServerSideProps()
in Next you can render a page by visiting it. E.g. GET http://localhost:3000/mypages/page1
Or if you use getStaticProps()
with getStaticPaths()
, you can use npm run build
to generate the HTML file (e.g. .next/server/pages
directory).
But what if you don't want to start a server. What if you have a particular page/URL in mind that you want to generate but without starting a server and sending an HTTP GET
request to it? This blog post shows a way to do this with a plain Node script.
Here's a solution to programmatically render a page:
#!/usr/bin/env node
import http from "http";
import next from "next";
async function main(uris) {
const nextApp = next({});
const nextHandleRequest = nextApp.getRequestHandler();
await nextApp.prepare();
const htmls = Object.fromEntries(
await Promise.all(
uris.map((uri) => {
try {
// If it's a fully qualified URL, make it its pathname
uri = new URL(uri).pathname;
} catch {}
return renderPage(nextHandleRequest, uri);
})
)
);
console.log(htmls);
}
async function renderPage(handler, url) {
const req = new http.IncomingMessage(null);
const res = new http.ServerResponse(req);
req.method = "GET";
req.url = url;
req.path = url;
req.cookies = {};
req.headers = {};
await handler(req, res);
if (res.statusCode !== 200) {
throw new Error(`${res.statusCode} on rendering ${req.url}`);
}
for (const { data } of res.outputData) {
const [, body] = data.split("\r\n\r\n");
if (body) return [url, body];
}
throw new Error("No output data has a body");
}
main(process.argv.slice(2)).catch((err) => {
console.error(err);
process.exit(1);
});
To demonstrate I created this sample repo: https://github.com/peterbe/programmatically-render-next-page
Note, that you need to run npm run build
first so Next can have all the static assets ready.
In conclusion
The alternative, in automation, would be run something like this:
▶ npm run build && npm run start &
▶ sleep 5 # give the server a chance to start
▶ xh http://localhost:3000/aboutus
HTTP/1.1 200 OK
Connection: keep-alive
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Date: Tue, 06 Sep 2022 12:23:42 GMT
Etag: "m8ff9sdduo1hk"
Keep-Alive: timeout=5
Transfer-Encoding: chunked
Vary: Accept-Encoding
X-Powered-By: Next.js
<!DOCTYPE html><html><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><title>About Us page</title><meta name="description" content="We do things. I hope."/><link rel="icon" href="/favicon.ico"/><meta name="next-head-count" content="5"/><link rel="preload" href="/_next/static/css/ab44ce7add5c3d11.css" as="style"/><link rel="stylesheet" href="/_next/static/css/ab44ce7add5c3d11.css" data-n-g=""/><link rel="preload" href="/_next/static/css/ae0e3e027412e072.css" as="style"/><link rel="stylesheet" href="/_next/static/css/ae0e3e027412e072.css" data-n-p=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/_next/static/chunks/polyfills-c67a75d1b6f99dc8.js"></script><script src="/_next/static/chunks/webpack-7ee66019f7f6d30f.js" defer=""></script><script src="/_next/static/chunks/framework-db825bd0b4ae01ef.js" defer=""></script><script src="/_next/static/chunks/main-3123a443c688934f.js" defer=""></script><script src="/_next/static/chunks/pages/_app-deb173bd80cbaa92.js" defer=""></script><script src="/_next/static/chunks/996-f1475101e84cf548.js" defer=""></script><script src="/_next/static/chunks/pages/aboutus-41b1f037d974ef60.js" defer=""></script><script src="/_next/static/REJUWXI26y-lp9JVmzJB5/_buildManifest.js" defer=""></script><script src="/_next/static/REJUWXI26y-lp9JVmzJB5/_ssgManifest.js" defer=""></script></head><body><div id="__next"><div class="Home_container__bCOhY"><main class="Home_main__nLjiQ"><h1 class="Home_title__T09hD">About Use page</h1><p class="Home_description__41Owk"><a href="/">Go to the <b>Home</b> page</a></p></main><footer class="Home_footer____T7K"><a href="/">Home page</a></footer></div></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{}},"page":"/aboutus","query":{},"buildId":"REJUWXI26y-lp9JVmzJB5","nextExport":true,"autoExport":true,"isFallback":false,"scriptLoader":[]}</script></body></html>
There are probably many great ideas that this can be used for. At work we use getServerSideProps()
and we have too many pages to build them all statically. We need a solution like this to do custom analysis of the rendered HTML to check for broken links by analyzing every generated <a href>
tag.
Join a list with a bitwise or operator in Python
August 22, 2022
0 comments Python
The bitwise OR operator in Python is often convenient when you want to combine multiple things into one thing. For example, with the Django ORM you might do this:
from django.db.models import Q
filter_ = Q(first_name__icontains="peter") | Q(first_name__icontains="ashley")
for contact in Contact.objects.filter(filter_):
print((contact.first_name, contact.last_name))
See how it hardcodes the filtering on strings peter
and ashley
.
But what if that was a bit more complicated:
from django.db.models import Q
filter_ = Q(first_name__icontains="peter")
if include("ashley"):
filter_ | = Q(first_name__icontains="ashley")
for contact in Contact.objects.filter(filter_):
print((contact.first_name, contact.last_name))
So far, same functionality.
But what if the business logic is more complicated? You can't do this:
filter_ = None
if include("peter"):
filter_ | = Q(first_name__icontains="peter") # WILL NOT WORK
if include("ashley"):
filter_ | = Q(first_name__icontains="ashley")
for contact in Contact.objects.filter(filter_):
print((contact.first_name, contact.last_name))
What if the list of things you want to filter on depends on a list? You'd need to do the |=
stuff "dynamically". One way to solve that is with functools.reduce
. Suppose the list of things you want to bitwise-OR together is a list:
from django.db.models import Q
from operator import or_
from functools import reduce
def include(_):
import random
return random.random() > 0.5
filters = []
if include("peter"):
filters.append(Q(first_name__icontains="peter"))
if include("ashley"):
filters.append(Q(first_name__icontains="ashley"))
assert len(filters), "must have at least one filter"
filter_ = reduce(or_, filters) # THE MAGIC!
for contact in Contact.objects.filter(filter_):
print((contact.first_name, contact.last_name))
And finally, if it's a list already:
from django.db.models import Q
from operator import or_
from functools import reduce
names = ["peter", "ashley"]
qs = [Q(first_name__icontains=x) for x in names]
filter_ = reduce(or_, qs)
for contact in Contact.objects.filter(filter_):
print((contact.first_name, contact.last_name))
Side note
Django's django.db.models.Q
is actually quite flexible with used with MyModel.objects.filter(...)
because this actually works:
from django.db.models import Q
def include(_):
import random
return random.random() > 0.5
filter_ = Q() # MAGIC SAUCE
if include("peter"):
filter_ |= Q(first_name__icontains="peter")
if include("ashley"):
filter_ |= Q(first_name__icontains="ashley")
for contact in Contact.objects.filter(filter_):
print((contact.first_name, contact.last_name))