Peterbe.com

A blog and website by Peter Bengtsson

Filtered home page!
Currently only showing blog entries under thecategory: Linux. Clear filter

The best grep tool in the world; ripgrep

Linux, Web development, MacOSX

https://github.com/BurntSushi/ripgrep

tl;dr; ripgrep (aka. rg) is the best tool to grep today.

ripgrep is a tool for searching files. Its killer feature is that it's fast. Like, really really fast. Faster than sift, git grep, ack, regular grep etc.

If you don't believe me, either read this detailed blog post from its author or just jump straight to the conclusion:

  • For both searching single files and huge directories of files, no other tool obviously stands above ripgrep in either performance or correctness.

  • ripgrep is the only tool with proper Unicode support that doesn’t make you pay dearly for it.

  • Tools that search many files at once are generally slower if they use memory maps, not faster.

Benchmark
Benchmark

I used to use git grep whenever I was inside a git repo and sift for everything else. That alone, was a huge step up from regular grep. Granted, almost all my git repos are small enough that regular git grep is faster than I can perceive many times. But with ripgrep I can just add --no-ignore-vcs and it searches in all the files mentioned in .gitignore too. That's useful when you want to search in your own source as well as the files in node_modules.

The installation instructions are easy. I installed it with brew install ripgrep and the best way to learn how to use it is rg --help. Remember that it has a lot of cool features that are well worth learning. It's written in Rust and so far I haven't had a single crash, ever. The ability to search by file type gets some getting used to (tip! use: rg --type-list) and remember that you can pipe rg output to another rg. For example, to search for all lines that contain query and string you can use rg query | rg string.

Please post a comment if you have thoughts or questions.

How to unset aliases set by Oh My Zsh

Linux, MacOSX

I use Oh My Zsh and I highly recommend it. However, it sets some aliases that I don't want. In particular, there's a plugin called git.plugin.zsh (located in ~/.oh-my-zsh/plugins/git/git.plugin.zsh) that interfers with a global binary I have in $PATH. So when I start a shell the executable gg becomes...:

▶ which gg
gg: aliased to git gui citool

That overrides /usr/local/bin/gg which is the one I want to execute when I type gg. To unset that I can run...:

unset gg

▶ which gg
/usr/local/bin/gg

To override it "permanently", I added, to the end of ~/.zshrc:

# This unsets ~/.oh-my-zsh/plugins/git/git.plugin.zsh
# So my /usr/local/bin/gg works instead
unalias gg

Now whenever I start a new terminal, it defaults to the gg in /usr/local/bin/gg instead.

Please post a comment if you have thoughts or questions.

How to NOT start two servers on the same port

Linux, Web development

First of all, you can't start two servers on the same port. Ultimately it will fail. However, you might not want a late notice of this. For example, if you do this:

# In one terminal
$ cd elasticsearch-6.1.0
$ ./bin/elasticsearch
...
$ curl localhost:9200
...
"version" : {
    "number" : "6.1.0",
...
# In *another* terminal
$ cd elasticsearch-6.2.4
$ ./bin/elasticsearch
...
$ curl localhost:9200
...
"version" : {
    "number" : "6.1.0",
...

In other words, what happened to the elasticsearch-6.2.4/bin/elasticsearch?? It actually started on port :9201. But that's a rather scary thing because as you jump between project in different tabs or you might not notice that you have Elasticsearch running with docker-compose somewhere.

To remedy this I use this curl one-liner:

$ curl -s localhost:9200 > /dev/null && echo "Already running!" && exit || ./bin/elasticsearch

Now if you try to start a server on a used port it will exit early.

To wrap this up in a script, take this:

#!/bin/bash

set -eo pipefail

hostandport=$1
shift
curl -s "$hostandport" >/dev/null && \
  echo "Already running on $hostandport" && \
  exit 1 || exec "$@"

...and make it an executable called unlessalready.sh and now you can do this:

$ unlessalready.sh localhost:9200 ./bin/elasticsearch

Please post a comment if you have thoughts or questions.

How I found out where a bash alias was set up

Linux

I wanted to install a command line tool called gg. But for some reason, gg was already tied to an alias. No problem, I'll just delete that alias. I looked in ~/.bash_profile and I looked in ~/.zshrc and it wasn't there!

But here's how I managed to figure out where it came from:

▶ which gg
gg: aliased to git gui citool

Then I copied the git gui citool part of that output and ran:

▶ rg --hidden 'git gui citool'
.oh-my-zsh/plugins/git/git.plugin.zsh
104:alias gg='git gui citool'
105:alias gga='git gui citool --amend'

A ha! So it was .oh-my-zsh/plugins/git/git.plugin.zsh that was the culprit. Totally forgot about the plugin. It's full of other useful aliases so I just commented out the one(s) I knew I don't need any more.

By the way rg, aka. ripgrep is probably one of the best tools I have. I use it so often that it's attached to my belt rather than in my toolbox.

Please post a comment if you have thoughts or questions.

gtop is best

Linux, MacOSX, JavaScript

https://github.com/aksakalli/gtop

To me, using top inside a Linux server via SSH is all muscle-memory and it's definitely good enough. On my Macbook when working on some long-running code that is resource intensive the best tool I know of is: gtop

gtop in action
gtop in action

I like it because it has the graphs I want and need. It splits up the work of each CPU which is awesome. That's useful for understanding how well a program is able to leverage more than one CPU process.

And it's really nice to have the list of Processes there to be able to quickly compare which programs are running and how that might affect the use of the CPUs.

Instead of listing alternatives I've tried before, hopefully this Reddit discussion has good links to other alternatives

Please post a comment if you have thoughts or questions.

Run something forever in bash until you want to stop it

Linux

I often use this in various projects. I find it very useful. Thought I'd share to see if others find it useful.

Running something forever

Suppose you have some command that you want to run a lot. One way is to do this:

$ ./manage.py run-some-command && \
  ./manage.py run-some-command && \
  ./manage.py run-some-command && \
  ./manage.py run-some-command && \
  ./manage.py run-some-command && \
  ./manage.py run-some-command && \
  ./manage.py run-some-command && \
  ./manage.py run-some-command && \
  ./manage.py run-some-command && \
  ./manage.py run-some-command

That runs the command 10 times. Clunky but effective.

Another alternative is to hijack the watch command. By default it waits 2 seconds between each run but if the command takes longer than 2 seconds, it'll just wait. Running...

$ watch ./manage.py run-some-command

Is almost the same as running...:

$ clear && sleep 2 && ./manage.py run-some-command && \
  clear && sleep 2 && ./manage.py run-some-command && \
  clear && sleep 2 && ./manage.py run-some-command && \
  clear && sleep 2 && ./manage.py run-some-command && \
  clear && sleep 2 && ./manage.py run-some-command && \
  clear && sleep 2 && ./manage.py run-some-command && \
  ...
  ...forever until you Ctrl-C it...

But that's clunky too because you might not want it to clear the screen between each run and you get an un-necessary delay between each run.

The biggest problem is that with using watch or copy-n-paste the command many times with && between is that if you need to stop it you have to Ctrl-C and that might kill the command at a precious time.

A better solution

The important thing is that if you want to stop the command repeater, is that it gets to finish what it's working on at the moment.

Here's a great and simple solution:

#!/usr/bin/env bash
set -eo pipefail

_stopnow() {
  test -f stopnow && echo "Stopping!" && rm stopnow && exit 0 || return 0
}

while true
do
    _stopnow
    # Below here, you put in your command you want to run:

    ./manage.py run-some-command
done

Save that file as run-forever.sh and now you can do this:

$ bash run-forever.sh

It'll sit there and do its thing over and over. If you want to stop it (from another terminal):

$ touch stopnow

(the file stopnow will be deleted after it's spotted once)

Getting fancy

Instead of taking this bash script and editing it every time you need it to run a different command you can make it a globally available command. Here's how I do it:

#!/usr/bin/env bash
set -eo pipefail


count=0

_stopnow() {
    count="$(($count+1))"
    test -f stopnow && \
      echo "Stopping after $count iterations!" && \
      rm stopnow && exit 0 || return 0
}

control_c()
# run if user hits control-c
{
  echo "Managed to do $count iterations"
  exit $?
}

# trap keyboard interrupt (control-c)
trap control_c SIGINT

echo "To stop this forever loop created a file called stopnow."
echo "E.g: touch stopnow"
echo ""
echo "Now going to run '$@' forever"
echo ""
while true
do
    _stopnow

    eval $@

    # Do this in case you accidentally pass an argument
    # that finishes too quickly.
    sleep 1

done

This code in a Gist here.

Put this file in ~/bin/run-forever.sh and chmod +x ~/bin/run-forever.sh.

Now you can do this:

$ run-forever.sh ./manage.py run-some-command

If the command you want to run, forever, requires an operator you have to wrap everything in single quotation marks. For example:

$ run-forever.sh './manage.py run-some-command && echo "Cooling CPUs..." && sleep 10' 

Please post a comment if you have thoughts or questions.