{"next_page": 2, "previous_page": null, "max_next_page": 10, "posts": [{"oid": "pytest-import-file-mismatch", "title": "pytest \"import file mismatch\"", "pub_date": "2026-04-01T13:20:53.199Z", "comments": 0, "categories": ["Python"], "html": "<p>I stumbled into a strange error when running <code>pytest</code> in a side-project:</p>\n<pre>import file mismatch:\nimported module &#x27;test_comments&#x27; has this __file__ attribute:\n  /&lt;root&gt;/django-peterbecom/peterbecom/api/tests/test_comments.py\nwhich is not the same as the test file we want to collect:\n  /&lt;root&gt;/django-peterbecom/peterbecom/publicapi/tests/test_comments.py\nHINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules</pre>\n\n<p>I desperately made sure I deleted every <code>__pycache__</code> directory and every <code>*.pyc</code> file. I also ran <code>rm -fr .pytest_cache</code>.</p>\n<p>It was strange because I knew I had, in the past, had two files called exactly the same. E.g. <code>test_utils.py</code>.</p>\n<p>Turns out, you <em>can</em> have two <code>test_something.py</code> called exactly the same but they <strong>must both sit in a directory that has a <code>__init__.py</code></strong>.</p>\n<p>So there was nothing wrong with <code>/&lt;root&gt;/django-peterbecom/peterbecom/api/tests/test_comments.py</code>. I just had to do:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-built_in\">touch</span> /&lt;root&gt;/django-peterbecom/peterbecom/api/tests/__init__.py\n</code></pre>\n\n<p>...and the problem got solved.</p>", "url": null, "disallow_comments": false, "split": null}, {"oid": "copy-the-current-line-in-vs-code", "title": "Copy the current line in VS Code", "pub_date": "2026-03-12T15:07:52.002Z", "comments": 0, "categories": ["macOS"], "html": "<p><strong>tl;dr; In VS Code you can copy the current line without first selecting it. Cmd-C without selection defaults to whole line.</strong></p>\n<p>One of those \"How did I not know this until now?!?!\"<br />\nVS Code is not emacs even though its emulation gets many things right. But one thing that I only just learned is that to copy a whole current line, you don't have to select it first. If nothing is selected, whichever line the cursor is on gets the entire line copied.</p>\n<p>The regular way of doing it:</p>\n<video controls poster=\"/cache/0f/21/0f21ee81bf7fcf4fd4555b9628219961.jpg\" style=\"max-width:100%\">  \n  <source src=\"/cache/plog/copy-the-current-line-in-vs-code/453259ccea.1773321273.mov\" type=\"video/mov\"/>\n  <source src=\"/cache/plog/copy-the-current-line-in-vs-code/453259ccea.1773321273.mp4\" type=\"video/mp4\"/>\n  <source src=\"/cache/plog/copy-the-current-line-in-vs-code/453259ccea.1773321273.webm\" type=\"video/webm\"/>\n  Download the <a href=\"/cache/plog/copy-the-current-line-in-vs-code/453259ccea.1773321273.mov\">video/mov</a> or <a href=\"/cache/plog/copy-the-current-line-in-vs-code/453259ccea.1773321273.mp4\">video/mp4</a> or <a href=\"/cache/plog/copy-the-current-line-in-vs-code/453259ccea.1773321273.webm\">video/webm</a>\n</video>\n\n<p><em>the keyboard use here is down, Ctrl-Shift-left, Ctrl-Shift-left, Cmd-C. Then moving down to a new line and Cmd-V</em></p>\n<p>Here, note that the cursor is just somewhere on line 6:</p>\n<video controls poster=\"/cache/ab/c0/abc0d70308a40cdc6e3121f272de23fe.jpg\" style=\"max-width:100%\">  \n  <source src=\"/cache/plog/copy-the-current-line-in-vs-code/0bac73b362.1773321406.mov\" type=\"video/mov\"/>\n  <source src=\"/cache/plog/copy-the-current-line-in-vs-code/0bac73b362.1773321406.mp4\" type=\"video/mp4\"/>\n  <source src=\"/cache/plog/copy-the-current-line-in-vs-code/0bac73b362.1773321406.webm\" type=\"video/webm\"/>\n  Download the <a href=\"/cache/plog/copy-the-current-line-in-vs-code/0bac73b362.1773321406.mov\">video/mov</a> or <a href=\"/cache/plog/copy-the-current-line-in-vs-code/0bac73b362.1773321406.mp4\">video/mp4</a> or <a href=\"/cache/plog/copy-the-current-line-in-vs-code/0bac73b362.1773321406.webm\">video/webm</a>\n</video>\n\n<p><em>the keyboard use here is down, down, Cmd-C, down, Cmd-V</em></p>\n<h4 id=\"bonus\"><a class=\"toclink\" href=\"#bonus\">Bonus</a></h4>\n<p>In this particular demo video, I'm just replicating line 6 and putting a copy of that on (new) line 8. There's actually another keyboard shortcut sequence for that: Option+Shift+down then Option+down</p>\n<p>Option+Shift+down replicates the current line and puts a copy right below. Then Option+down moves the current line.</p>", "url": null, "disallow_comments": false, "split": null}, {"oid": "logger.error-or-logger.exception", "title": "logger.error or logger.exception in Python", "pub_date": "2026-03-06T15:27:44.219Z", "comments": 0, "categories": ["Python"], "html": "<p>Consider this Python code:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">try</span>:\n    <span class=\"hljs-number\">1</span> / <span class=\"hljs-number\">0</span>\n<span class=\"hljs-keyword\">except</span> Exception <span class=\"hljs-keyword\">as</span> e:\n    logger.error(<span class=\"hljs-string\">&quot;An error occurred while dividing by zero.: %s&quot;</span>, e)\n</code></pre>\n\n<p>The output of this is:</p>\n<pre><code class=\"hljs\">\nAn error occurred while dividing by zero.: division by zero\n</code></pre>\n\n<p>No traceback. Perhaps you don't care because you don't need it.<br />\nI see code like this quite often and it's curious that you even use <code>logger.error</code> if it's not a problem. And it's curious that you include the stringified exception into the logger message.</p>\n<p>Another common pattern I see is use of <code>exc_info=True</code> like this:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">try</span>:\n    <span class=\"hljs-number\">1</span> / <span class=\"hljs-number\">0</span>\n<span class=\"hljs-keyword\">except</span> Exception:\n    logger.error(<span class=\"hljs-string\">&quot;An error occurred while dividing by zero.&quot;</span>, exc_info=<span class=\"hljs-literal\">True</span>)\n</code></pre>\n\n<p>Its output is:</p>\n<pre><code class=\"hljs\">\nAn error occurred while dividing by zero.\nTraceback (most recent call last):\n  File &quot;/Users/peterbengtsson/dummy.py&quot;, line 23, in &lt;module&gt;\n    1 / 0\n    ~~^~~\nZeroDivisionError: division by zero\n</code></pre>\n\n<p>Ok, now you get the traceback and the error value (<code>division by zero</code> in this case).</p>\n<p>But a more convenient function is <code>logger.exception</code> which looks like this:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">try</span>:\n    <span class=\"hljs-number\">1</span> / <span class=\"hljs-number\">0</span>\n<span class=\"hljs-keyword\">except</span> Exception:\n    logger.exception(<span class=\"hljs-string\">&quot;An error occurred while dividing by zero.&quot;</span>)\n</code></pre>\n\n<p>Its output is:</p>\n<pre><code class=\"hljs\">\nAn error occurred while dividing by zero.\nTraceback (most recent call last):\n  File &quot;/Users/peterbengtsson/dummy.py&quot;, line 9, in &lt;module&gt;\n    1 / 0\n    ~~^~~\nZeroDivisionError: division by zero\n</code></pre>\n\n<p>So it's sugar for <code>logger.error</code>.</p>\n<p>Also, a common logging config is something like this:</p>\n<pre>import logging\n\nlogger = logging.getLogger(__name__)\nlogging.basicConfig(\n    format=&quot;%(asctime)s - %(levelname)s - %(name)s - %(message)s&quot;, level=logging.ERROR\n)</pre>\n\n<p>So if you use <code>logger.exception</code> what will it print? In short, the same as if you used <code>logger.error</code>. For example, with the <code>logger.exception(\"An error occurred while dividing by zero.\")</code> line above:</p>\n<pre><code class=\"hljs\">\n2026-03-06 10:45:23,570 - ERROR - __main__ - An error occurred while dividing by zero.\nTraceback (most recent call last):\n  File &quot;/Users/peterbengtsson/dummy.py&quot;, line 12, in &lt;module&gt;\n    1 / 0\n    ~~^~~\nZeroDivisionError: division by zero\n</code></pre>\n\n<h3 id=\"bonus-add_note\"><a class=\"toclink\" href=\"#bonus-add_note\">Bonus - <code>add_note</code></a></h3>\n<p>You can, if it's applicable, inject some more information about the exception. Consider:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">try</span>:\n    n / <span class=\"hljs-number\">0</span>\n<span class=\"hljs-keyword\">except</span> Exception <span class=\"hljs-keyword\">as</span> exception:\n    exception.add_note(<span class=\"hljs-string\">f&quot;The numerator was <span class=\"hljs-subst\">{n}</span>.&quot;</span>)\n    logger.exception(<span class=\"hljs-string\">&quot;An error occurred while dividing by zero.&quot;</span>)\n</code></pre>\n\n<p>The net output of this is:</p>\n<pre><code class=\"hljs\">\n2026-03-06 10:48:34,279 - ERROR - __main__ - An error occurred while dividing by zero.\nTraceback (most recent call last):\n  File &quot;/Users/peterbengtsson/dummy.py&quot;, line 13, in &lt;module&gt;\n    1 / 0\n    ~~^~~\nZeroDivisionError: division by zero\nThe numerator was 123.\n</code></pre>", "url": null, "disallow_comments": false, "split": null}, {"oid": "how-to-find-which-git-sha-it-was-when-you-merged-in-the-default-branch", "title": "How to find which git SHA it was when you merged in the default branch", "pub_date": "2026-02-26T17:23:04.239Z", "comments": 1, "categories": ["Linux", "Git"], "html": "<p>Please tell me there's a better way.</p>\n<p>I'm in a git feature branch. It was branched off of <code>master</code> some time ago. Then I made some commits. Later, I merged in latest <code>master</code>, and after that some more commits. The question is what was the SHA when I merged in latest <code>master</code>? How to find that out?</p>\n<p>Here's what I did:</p>\n<p>I'm in my feature branch and I run:</p>\n<pre><code class=\"hljs\">\ngit <span class=\"hljs-built_in\">log</span> --merges --first-parent\n</code></pre>\n\n<p>It prints this out:</p>\n<pre><code class=\"hljs\">\ncommit 2c010ef6e94d792f25f7309ed83a2d35e41e5051\nMerge: 24b2a2513d6f e2817859ed02\nAuthor: Peter Bengtsson &lt;email&gt;\nDate:   Thu Feb 26 14:06:24 2026 +0000\n\n  Merge remote-tracking branch &#x27;origin&#x27; into name-of-my-feature-branch\n</code></pre>\n\n<p>The SHA that was, on <code>master</code>, at the time of merging, was <code>e2817859ed02</code> which is the second thing the <code>Merge: 24b2a2513d6f e2817859ed02</code> line.</p>\n<p>Now, equipped with this I can go back in time <code>git checkout e2817859ed02</code> and on that detached HEAD, I can do things like run (unit) tests to see if the <code>master</code> branch was busted at the time I merged it into my feature branch.</p>", "url": null, "disallow_comments": false, "split": null}, {"oid": "gg2-branches-got-50percent-faster-by-a-promise.all", "title": "gg2 branches got 50% faster by a Promise.all", "pub_date": "2026-02-16T13:17:09.034Z", "comments": 3, "categories": ["TypeScript", "Bun", "JavaScript"], "html": "<p><a href=\"https://gg2.peterbe.com/\">gg2</a> is a command line tool for doing things with Git branches, faster. One of the convenient commands is <code>gg2 branches</code>. \nIt lists your recent branches, sorted by committer-date and for each branch it displays a human-friendly date distance and whether it's already been merged. Looks like this:</p>\n<p><a href=\"/cache/d9/44/d94443c94b1f1d57abc02f7d4dcb717c.png\"><img src=\"/cache/d9/44/d94443c94b1f1d57abc02f7d4dcb717c.png\" class=\"fullsize\"></a></p>\n<p>Turns out, for large repos figuring out which branches have already been merged is quite slow. For example, in this repo it takes half a second to list all merged branches:</p>\n<pre><code class=\"hljs\">\n$ git branch --all --merged | wc -l\n669\n\n$ time git branch --all --merged 1&gt;/dev/null\n\nreal    0m0.490s\nuser    0m0.460s\nsys     0m0.030s\n</code></pre>\n\n<p>The problem was that <code>gg2 branches</code> also does multiple other <code>git branch</code> listing commands to get information. In particular: get the current branch, figure out the default branch, a list of branch committer dates, and a list of merged branches.<br />\nAll of these async operations were done in serial. With a bit of rearranging, and refactoring, all of these can be turned into this:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">const</span> [defaultBranch, currentBranch, dates, isMerged] = <span class=\"hljs-keyword\">await</span> <span class=\"hljs-title class_\">Promise</span>.<span class=\"hljs-title function_\">all</span>([\n  <span class=\"hljs-title function_\">getDefaultBranch</span>(git),\n  <span class=\"hljs-title function_\">getCurrentBranch</span>(git),\n  <span class=\"hljs-title function_\">getAllBranchDates</span>(git),\n  <span class=\"hljs-title function_\">getAllMergedBranches</span>(git),\n])\n</code></pre>\n\n<p>The <code>getAllMergedBranches(git)</code> function still takes a very long time and I think it lists all remote branches that have been merged. Anyway, now it's only the slowest part of the <code>Promise.all</code> rather than one of multiple.</p>\n<p><strong>UPDATE</strong></p>\n<p>The act of blogging about something can sometimes bring out how wrong you are.<br />\nWhat I didn't realize is that I was using <code>git branch --all --merged</code> to know which local branches have already been merged into the default branch. But you can just use <code>git branch --merged</code> to get the same result but much much faster.</p>\n<p><a href=\"https://github.com/peterbe/gg2/pull/101\">Correcting for this</a>, on a large repo, the average time to execute <code>gg2 branches</code> went from median 541ms to median 188ms. So that's 3x speed improvement.</p>", "url": null, "disallow_comments": false, "split": null}, {"oid": "xbar-my-prs", "title": "xbar-my-prs Info about your GitHub PRs in the macOS menu bar", "pub_date": "2026-01-29T18:37:54.287Z", "comments": 0, "categories": ["TypeScript", "macOS", "Bun"], "html": "<p><a href=\"https://github.com/peterbe/xbar-my-prs\">xbar-my-prs</a> is a plugin I wrote for something called <a href=\"https://xbarapp.com/\">xbar</a> which is a macOS app that let's you write scripts that then show up in the macOS menu bar. Mine is about fetching information about my recent GitHub pull requests.</p>\n<p>A picture of it in action:</p>\n<p><a href=\"/cache/b8/f6/b8f6c9b52314c6a7674ac86d44fc8a46.png\"><img src=\"/cache/b8/f6/b8f6c9b52314c6a7674ac86d44fc8a46.png\" class=\"fullsize\" style=\"width:70%\"></a></p>\n<p>Every minute, it does this GitHub search: <a href=\"https://github.com/search?q=is%3Apr+is%3Aopen+author%3A%40me+sort%3Aupdated+&amp;type=pullrequests\"><code>is:pr author:@me sort:updated</code></a></p>\n<p>Then it parses the results and attempts to display a neat summary.</p>\n<p>And each time it runs, it stores a copy of the result in the <code>/tmp</code> directory so that it can compare what the result became compared to before. So if something has changed in the last minute, it displays a summary of the change(s). Example:</p>\n<p><a href=\"/cache/d5/d5/d5d5ee05f145495eb912840129a5b20b.png\"><img src=\"/cache/d5/d5/d5d5ee05f145495eb912840129a5b20b.png\" class=\"fullsize\" style=\"width:60%\"></a></p>\n<h3 id=\"how-you-can-get-it-too\"><a class=\"toclink\" href=\"#how-you-can-get-it-too\">How you can get it too</a></h3>\n<p>First you have to install <a href=\"https://xbarapp.com/\">xbar</a> onto your macOS.</p>\n<p>At the time of writing you can't install it with Homebrew. Working on that. But all you need is <code>just</code> and Bun and then you can install it like this:</p>\n<pre><code class=\"hljs\">\ngit <span class=\"hljs-built_in\">clone</span> https://github.com/peterbe/xbar-my-prs.git &amp;&amp; <span class=\"hljs-built_in\">cd</span> xbar-my-prs\njust install\njust ship\n</code></pre>\n\n<p>If all goes well, it should have crated an executable file at:</p>\n<pre><code class=\"hljs\">\n~/Library/Application\\ Support/xbar/plugins/my-prs.1m.bin\n</code></pre>\n\n<p>on your system. And that should be all xbar needs to know there's a new plugin to run.</p>", "url": "https://github.com/peterbe/xbar-my-prs", "disallow_comments": false, "split": null}, {"oid": "optimizing-bun-compiled-binary-for-gg2", "title": "Optimizing Bun compiled binary for gg2", "pub_date": "2026-01-13T16:06:09.784Z", "comments": 0, "categories": ["TypeScript", "Bun"], "html": "<p><strong>tl;dr; adding <code>--bytecode</code> and <code>--production</code> to <code>bun build --compile</code> gives a marginal boost.</strong></p>\n<p>I have a CLI tool called <a href=\"https://gg2.peterbe.com/\">gg2</a> which is written in TypeScript and using Bun <strong>compiled to a single executable binary</strong> that can be <a href=\"https://www.peterbe.com/plog/gg2-is-now-installable-with-homebrew\">installed with Homebrew</a>. It's fast. <a href=\"https://www.peterbe.com/plog/gg2-is-now-installable-with-homebrew\">Only 17% slower than GitHub's Go-compile <code>gh</code> CLI</a>. But can it get even faster?</p>\n<p>I read about <a href=\"https://bun.com/docs/bundler/bytecode\">Bytecode Caching</a> which is supposed to make the executable \"dramatically improved\". Hmm. Let's see. There's also the <code>--production</code> option, which according to <code>Set NODE_ENV=production and enable minification</code>. Not convinced it's doing much but let's try.</p>\n<p>I compiled the binary 4 different ways:</p>\n<ul>\n<li><code>bun build src/index.ts --target=bun --compile --outfile out/gg</code></li>\n<li><code>bun build src/index.ts --target=bun --compile --outfile out/gg-bytecode --bytecode</code></li>\n<li><code>bun build src/index.ts --target=bun --compile --outfile out/gg-production --production</code></li>\n<li><code>bun build src/index.ts --target=bun --compile --outfile out/gg-bytecode-production --production --bytecode</code></li>\n</ul>\n<p>Then I ran <code>hyperfine</code> like this:</p>\n<pre><code class=\"hljs\">\nhyperfine &quot;./out/gg --version&quot; &quot;./out/gg-bytecode --version&quot; &quot;./out/gg-production --version&quot; &quot;./out/gg-bytecode-production --version&quot; --warmup 6\n</code></pre>\n\n<p>The results are as follows:</p>\n<table>\n<thead>\n<tr>\n<th>BINARY</th>\n<th>TIME (ms, smaller is better)</th>\n<th>SIZE (MB)</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>gg</td>\n<td>87</td>\n<td>58</td>\n</tr>\n<tr>\n<td>gg-bytecode</td>\n<td>81</td>\n<td>60</td>\n</tr>\n<tr>\n<td>gg-production</td>\n<td>82.4</td>\n<td>55</td>\n</tr>\n<tr>\n<td>gg-bytecode-production</td>\n<td>79.6</td>\n<td>60</td>\n</tr>\n</tbody>\n</table>\n<p><a href=\"/cache/c7/1f/c71f48b8b4ab4acb5c455a140d0301da.png\"><img src=\"/cache/c7/1f/c71f48b8b4ab4acb5c455a140d0301da.png\" class=\"fullsize\"></a></p>\n<p>(<em>sorry for being terrible at using Excel to draw charts</em>)</p>\n<h3 id=\"conclusion\"><a class=\"toclink\" href=\"#conclusion\">Conclusion</a></h3>\n<p>I'm skeptical. The <code>hyperfine</code> often complains about statistical outliers so you have to run it with warmup and re-run it till it doesn't complain.</p>\n<p>One caveat with using <code>--bytecode</code> is that the compiled code is not re-usable across <strong>different versions of bun</strong> but I don't understand how that's applicable if it's a executable binary that is shipped outside of Bun.</p>\n<p>I'm going to add <code>--bytecode</code> and <code>--production</code> to the next release of <a href=\"https://github.com/peterbe/gg2\"><code>gg2</code></a>.</p>", "url": null, "disallow_comments": false, "split": null}, {"oid": "gg2-has-a-web-page-now", "title": "gg2 has a web page now", "pub_date": "2026-01-05T15:59:19.418Z", "comments": 0, "categories": ["TypeScript", "JavaScript"], "html": "<p><code>gg2</code> is a CLI tool to become more efficient on the command line with <code>git</code>. It's open source and <a href=\"https://www.peterbe.com/plog/gg2-is-now-installable-with-homebrew\">installable with Homebrew</a>. And now it has its own web page:</p>\n<p><a href=\"https://gg2.peterbe.com/features.html\">https://gg2.peterbe.com/</a> <br />\n\ud83c\udf89 \ud83e\udd73 \ud83c\udf7e</p>\n<p>The site is built on <a href=\"https://vuepress.vuejs.org/\">Vuepress 2.0</a>. It's a couple of <code>.md</code> files that look pretty. The static site is hosted on GitHub Pages. All the documentation content and configuration is <a href=\"https://github.com/peterbe/gg2/tree/main/docs\">in the <code>docs/</code> directory</a>.</p>", "url": "https://gg2.peterbe.com/", "disallow_comments": false, "split": null}, {"oid": "one-tanstack-query-in-scattered-components", "title": "You don't need a context or state manager for TanStack Query in scattered React components", "pub_date": "2026-01-02T12:48:56.476Z", "comments": 0, "categories": ["React", "TypeScript", "JavaScript"], "html": "<p>Because TanStack Query uses a global cache, you can use the same <code>useQuery</code> hook in multiple places without making excess XHR requests.</p>\n<p>Traditionally, if you have data on a server, that you want to use in multiple places throughout your component tree, the advice is to lift it to the top and either prop-drill its data or put it in a context. In this contrived example, suppose the <code>&lt;Header&gt;</code> component and the <code>&lt;Dashboard&gt;</code> component both need the data from the primary server fetch. And the <code>&lt;Footer&gt;</code> component needs to know if there was an error fetching that server data:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">function</span> <span class=\"hljs-title function_\">App</span>(<span class=\"hljs-params\"></span>) {\n  <span class=\"hljs-keyword\">const</span> query = <span class=\"hljs-title function_\">useQuery</span>({\n    <span class=\"hljs-attr\">queryKey</span>: [<span class=\"hljs-string\">&quot;data&quot;</span>],\n    <span class=\"hljs-attr\">queryFn</span>: <span class=\"hljs-function\">() =&gt;</span> <span class=\"hljs-title function_\">fetchData</span>()\n  })\n\n  <span class=\"hljs-keyword\">return</span> <span class=\"language-xml\"><span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">div</span>&gt;</span>\n    <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">Header</span> <span class=\"hljs-attr\">data</span>=<span class=\"hljs-string\">{query.data}/</span>&gt;</span>\n    <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">Dashboard</span> <span class=\"hljs-attr\">data</span>=<span class=\"hljs-string\">{query.data}</span> <span class=\"hljs-attr\">isPending</span>=<span class=\"hljs-string\">{query.isPending}/</span>&gt;</span>\n    <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">Footer</span> <span class=\"hljs-attr\">error</span>=<span class=\"hljs-string\">{query.error}/</span>&gt;</span>\n  <span class=\"hljs-tag\">&lt;/<span class=\"hljs-name\">div</span>&gt;</span></span>\n}\n</code></pre>\n\n<p>But prop-drilling can be annoying. For example, that <code>&lt;Footer&gt;</code> component might look like this:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">function</span> <span class=\"hljs-title function_\">Footer</span>(<span class=\"hljs-params\">{error}: {error: <span class=\"hljs-built_in\">Error</span> | <span class=\"hljs-literal\">null</span>}</span>) {\n  <span class=\"hljs-keyword\">return</span> <span class=\"language-xml\"><span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">footer</span>&gt;</span>\n    <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">Alert</span> <span class=\"hljs-attr\">error</span>=<span class=\"hljs-string\">{error}/</span>&gt;</span>\n    <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">p</span>&gt;</span>Copyright 2026<span class=\"hljs-tag\">&lt;/<span class=\"hljs-name\">p</span>&gt;</span>\n  <span class=\"hljs-tag\">&lt;/<span class=\"hljs-name\">footer</span>&gt;</span></span>\n}\n\n<span class=\"hljs-keyword\">function</span> <span class=\"hljs-title function_\">Alert</span>(<span class=\"hljs-params\">{error}: {error: <span class=\"hljs-built_in\">Error</span> | <span class=\"hljs-literal\">null</span>}</span>) {\n  <span class=\"hljs-keyword\">if</span> (error) {\n    <span class=\"hljs-keyword\">return</span> <span class=\"language-xml\"><span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">div</span> <span class=\"hljs-attr\">className</span>=<span class=\"hljs-string\">&quot;footer-error&quot;</span>&gt;</span>There was an error fetching the data<span class=\"hljs-tag\">&lt;/<span class=\"hljs-name\">div</span>&gt;</span></span>\n  }\n  <span class=\"hljs-keyword\">return</span> <span class=\"hljs-literal\">null</span>\n}\n</code></pre>\n\n<h4 id=\"what-you-can-do-instead\"><a class=\"toclink\" href=\"#what-you-can-do-instead\">What you can do instead</a></h4>\n<p>First, refactor the <code>useQuery</code> call so it becomes a hook that encapsulates any of its configuration. For example:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">function</span> <span class=\"hljs-title function_\">useData</span>(<span class=\"hljs-params\"></span>) {\n  <span class=\"hljs-keyword\">return</span> <span class=\"hljs-title function_\">useQuery</span>({\n    <span class=\"hljs-attr\">queryKey</span>: [<span class=\"hljs-string\">&quot;data&quot;</span>],\n    <span class=\"hljs-attr\">queryFn</span>: <span class=\"hljs-function\">() =&gt;</span> <span class=\"hljs-title function_\">fetchData</span>(),\n    <span class=\"hljs-attr\">retry</span>: <span class=\"hljs-number\">0</span>,\n  });\n}\n</code></pre>\n\n<p>Now, you can use that hook in all the places where it's needed. For example:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">function</span> <span class=\"hljs-title function_\">Header</span>(<span class=\"hljs-params\"></span>) {\n  <span class=\"hljs-keyword\">const</span> {data} = <span class=\"hljs-title function_\">useData</span>();\n  <span class=\"hljs-keyword\">return</span> <span class=\"language-xml\"><span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">h1</span>&gt;</span>\n    Page Title {data ? ` (${data.count} items)` : null}\n  <span class=\"hljs-tag\">&lt;/<span class=\"hljs-name\">h1</span>&gt;</span></span>\n}\n\n<span class=\"hljs-keyword\">function</span> <span class=\"hljs-title function_\">Footer</span>(<span class=\"hljs-params\"></span>) {\n  <span class=\"hljs-keyword\">return</span> <span class=\"language-xml\"><span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">footer</span>&gt;</span>\n    <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">Alert</span>/&gt;</span>\n    <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">p</span>&gt;</span>Copyright 2026<span class=\"hljs-tag\">&lt;/<span class=\"hljs-name\">p</span>&gt;</span>\n  <span class=\"hljs-tag\">&lt;/<span class=\"hljs-name\">footer</span>&gt;</span></span>\n}\n\n<span class=\"hljs-keyword\">function</span> <span class=\"hljs-title function_\">Alert</span>(<span class=\"hljs-params\"></span>) {\n  <span class=\"hljs-keyword\">const</span> {error} = <span class=\"hljs-title function_\">useData</span>()\n  <span class=\"hljs-keyword\">if</span> (error) {\n    <span class=\"hljs-keyword\">return</span> <span class=\"language-xml\"><span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">div</span> <span class=\"hljs-attr\">className</span>=<span class=\"hljs-string\">&quot;footer-error&quot;</span>&gt;</span>There was an error fetching the data<span class=\"hljs-tag\">&lt;/<span class=\"hljs-name\">div</span>&gt;</span></span>\n  }\n  <span class=\"hljs-keyword\">return</span> <span class=\"hljs-literal\">null</span>\n}\n</code></pre>\n\n<p>And in the end, you can remove any drop-drilling</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">function</span> <span class=\"hljs-title function_\">App</span>(<span class=\"hljs-params\"></span>) {\n  <span class=\"hljs-keyword\">return</span> <span class=\"language-xml\"><span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">div</span>&gt;</span>\n    <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">Header</span>/&gt;</span>\n    <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">Dashboard</span>/&gt;</span>\n    <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">Footer</span>/&gt;</span>\n  <span class=\"hljs-tag\">&lt;/<span class=\"hljs-name\">div</span>&gt;</span></span>\n}\n</code></pre>\n\n<h3 id=\"conclusion\"><a class=\"toclink\" href=\"#conclusion\">Conclusion</a></h3>\n<p>The <code>useQuery</code> hook will use the global cache, that probably looks something like this:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">import</span> { <span class=\"hljs-title class_\">App</span> } <span class=\"hljs-keyword\">from</span> <span class=\"hljs-string\">&quot;./App&quot;</span>;\n<span class=\"hljs-keyword\">import</span> { <span class=\"hljs-title class_\">QueryClient</span>, <span class=\"hljs-title class_\">QueryClientProvider</span> } <span class=\"hljs-keyword\">from</span> <span class=\"hljs-string\">&quot;@tanstack/react-query&quot;</span>;\n\n<span class=\"hljs-keyword\">const</span> queryClient = <span class=\"hljs-keyword\">new</span> <span class=\"hljs-title class_\">QueryClient</span>();\n\n<span class=\"hljs-keyword\">const</span> app = (\n  <span class=\"language-xml\"><span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">StrictMode</span>&gt;</span>\n    <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">QueryClientProvider</span> <span class=\"hljs-attr\">client</span>=<span class=\"hljs-string\">{queryClient}</span>&gt;</span>\n      <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">App</span> /&gt;</span>\n    <span class=\"hljs-tag\">&lt;/<span class=\"hljs-name\">QueryClientProvider</span>&gt;</span>\n  <span class=\"hljs-tag\">&lt;/<span class=\"hljs-name\">StrictMode</span>&gt;</span></span>\n);\n</code></pre>\n\n<p>And that global cache will recognize if a query has <em>started</em> by the <code>queryKey</code> value. If one such Promise already exists, it won't start another. So you're guaranteed to not make multiple XHR queries to the back end.</p>", "url": null, "disallow_comments": false, "split": null}, {"oid": "autocomplete-using-postgresql-instead-of-elasticsearch", "title": "Autocomplete using PostgreSQL instead of Elasticsearch", "pub_date": "2025-12-18T09:46:00.462Z", "comments": 0, "categories": ["Elasticsearch", "Python", "PostgreSQL"], "html": "<p>Here on my blog I have a site search. Before you search, there's autocomplete. The autocomplete is solved by using <a href=\"https://github.com/downshift-js/downshift\">downshift</a> in React and on the backend, there's an API <code>/api/v1/typeahead?q=bla</code>. Up until today, that backend was powered by Elasticsearch. Now it's powered by PostgreSQL. Here's how I implemented it.</p>\n<h4 id=\"indexing\"><a class=\"toclink\" href=\"#indexing\">Indexing</a></h4>\n<p>A cron job loops over all titles in all blog posts and finds portions of the words in the titles as singles, doubles, and triples. For each one, the popularity of the blog post is accumulated to the extracted keywords and combos.</p>\n<p>These are then inserted into a Django ORM model that looks like this:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">class</span> <span class=\"hljs-title class_\">SearchTerm</span>(models.Model):\n    term = models.CharField(max_length=<span class=\"hljs-number\">100</span>, db_index=<span class=\"hljs-literal\">True</span>)\n    popularity = models.FloatField(default=<span class=\"hljs-number\">0.0</span>)\n    add_date = models.DateTimeField(auto_now=<span class=\"hljs-literal\">True</span>)\n    index_version = models.IntegerField(default=<span class=\"hljs-number\">0</span>)\n\n    <span class=\"hljs-keyword\">class</span> <span class=\"hljs-title class_\">Meta</span>:\n        unique_together = (<span class=\"hljs-string\">&quot;term&quot;</span>, <span class=\"hljs-string\">&quot;index_version&quot;</span>)\n        indexes = [\n            GinIndex(\n                name=<span class=\"hljs-string\">&quot;plog_searchterm_term_gin_idx&quot;</span>,\n                fields=[<span class=\"hljs-string\">&quot;term&quot;</span>],\n                opclasses=[<span class=\"hljs-string\">&quot;gin_trgm_ops&quot;</span>],\n            ),\n        ]\n</code></pre>\n\n<p>The <code>index_version</code> is used like this, in the indexing code:</p>\n<pre><code class=\"hljs\">\ncurrent_index_version = (\n    SearchTerm.objects.aggregate(Max(<span class=\"hljs-string\">&quot;index_version&quot;</span>))[<span class=\"hljs-string\">&quot;index_version__max&quot;</span>]\n    <span class=\"hljs-keyword\">or</span> <span class=\"hljs-number\">0</span>\n)\nindex_version = current_index_version + <span class=\"hljs-number\">1</span>\n\n...\n\nSearchTerm.objects.bulk_create(bulk)\n\nSearchTerm.objects.<span class=\"hljs-built_in\">filter</span>(index_version__lt=index_version).delete()\n</code></pre>\n\n<p>That means that I don't have to delete previous entries until new ones have been created. So if something goes wrong during the indexing, it doesn't break the API.<br />\nEssentially, there are about 13k entries in that model. For a very brief moment there are 2x13k entries and then back to 13k entries when the whole task is done.</p>\n<h4 id=\"search\"><a class=\"toclink\" href=\"#search\">Search</a></h4>\n<p>The search is done with the <code>LIKE</code> operator.</p>\n<pre><code class=\"hljs\">\npeterbecom=# select term from plog_searchterm where term like &#x27;za%&#x27;;\n            term\n-----------------------------\n zahid\n zappa\n zappa biography\n zappa biography barry\n zappa biography barry miles\n zappa blog\n(6 rows)\n</code></pre>\n\n<p>In Python, it's as simple as:</p>\n<pre><code class=\"hljs\">\nbase_qs = SearchTerm.objects.<span class=\"hljs-built_in\">all</span>()\nqs = base_qa.<span class=\"hljs-built_in\">filter</span>(term__startswith=term.lower())\n</code></pre>\n\n<p>But suppose someone searches for <code>bio</code>  we want it to match things like <code>frank zappa biography</code> so what it actually does is:</p>\n<pre><code class=\"hljs\">\n<span class=\"hljs-keyword\">from</span> django.db.models <span class=\"hljs-keyword\">import</span> Q \n\nqs = base_qs.<span class=\"hljs-built_in\">filter</span>(\n    Q(term__startswith=term.lower()) | Q(term__contains=<span class=\"hljs-string\">f&quot; <span class=\"hljs-subst\">{term.lower()}</span>&quot;</span>)\n)\n</code></pre>\n\n<h4 id=\"typo-tolerance\"><a class=\"toclink\" href=\"#typo-tolerance\">Typo tolerance</a></h4>\n<p>This is done with the <code>%</code> operator.</p>\n<pre><code class=\"hljs\">\npeterbecom=# select term from plog_searchterm where term % &#x27;frenk&#x27;;\n  term\n--------\n free\n frank\n freeze\n french\n(4 rows)\n</code></pre>\n\n<p>In the Django ORM it looks like this:</p>\n<pre><code class=\"hljs\">\nbase_qs = SearchTerm.objects.<span class=\"hljs-built_in\">all</span>()\nqs = base_qs.<span class=\"hljs-built_in\">filter</span>(term__trigram_similar=term.lower())\n</code></pre>\n\n<p>And if that doesn't work, it gets even more desperate. It does this using the <code>similarity()</code> function. Looks like this in SQL:</p>\n<pre><code class=\"hljs\">\npeterbecom=# select term from plog_searchterm where similarity(term, &#x27;zuppa&#x27;) &gt; 0.14;\n       term\n-------------------\n frank zappa\n zappa\n zappa biography\n radio frank zappa\n frank zappa blog\n zappa blog\n zurich\n(7 rows)\n</code></pre>\n\n<h4 id=\"note-on-typo-tolerance\"><a class=\"toclink\" href=\"#note-on-typo-tolerance\">Note on typo tolerance</a></h4>\n<p>Most of the time, the most basic query works and yields results. I.e. the <code>.filter(term__startswith=term.lower())</code> query.<br />\nIt's only if it yields fewer results than the pagination size. That's why the fault tolerance query is only-if-needed. This means, it might send 2 SQL select queries from Python to PostgreSQL. In Elasticsearch, you usually don't do this. You send multiple queries and boost the differently.</p>\n<p>It <em>can</em> be done with PostgreSQL too using an <code>UNION</code> operator so that you send <em>one</em> but <em>more complex</em> query.</p>\n<h4 id=\"speed\"><a class=\"toclink\" href=\"#speed\">Speed</a></h4>\n<p>It's hard to measure the true performance of these things because they're so fast that it's more about the network speed.</p>\n<p>On my fast MacBook Pro M4, I ran about 50 realistic queries and measured the time it took each with this new PostgreSQL-based solution versus the previous Elasticsearch solution. They both take about 4ms per query. I suspect, that 90% of that 4ms is serialization &amp; transmission, and not much time inside the database itself.</p>\n<p>The number of rows it searches is only, at the time of writing, 13,000+ so it's hard to get a feel for how much faster Elasticsearch would be than PostgreSQL. But with a GIN index in PostgreSQL, it would have to scale much much larger to feel too slow.</p>\n<h4 id=\"about-elasticsearch\"><a class=\"toclink\" href=\"#about-elasticsearch\">About Elasticsearch</a></h4>\n<p>Elasticsearch is better than PostgreSQL at full-text search, including n-grams. Elasticsearch is highly optimized for these kinds of things and has powerful ways that you can make a query be a <em>product</em> of how well it matched with each entry's popularity. With PostgreSQL that gets difficult.</p>\n<p>But PostgreSQL is simple. It's solid and it doesn't take up nearly <a href=\"/plog/elasticsearch-memory-usage\">as much memory as Elasticsearch</a>.</p>", "url": null, "disallow_comments": false, "split": null}]}