Giorgio Delgado

While working on the Caribou Engineering Blog website (repo), I needed to do some reshuffling of the contents of a particular directory.

The directory looked as follows:

  ▾ blog/
    ▸ nested-route/
      code-sample.md
      deriving-ols-estimator.mdx
      github-markdown-guide.mdx
      guide-to-using-images-in-nextjs.mdx
      introducing-tailwind-nextjs-starter-blog.mdx
      my-fancy-title.md
      new-features-in-v1.mdx
      pictures-of-canada.mdx
      the-time-machine.mdx

What I wanted to do was to move all of the contents of blog into an examples/ directory so that our team could have usage examples of the features that our markdown pages have (the blogging framework we're using has a lot of features such as the ability to render LaTex formulas and more).

So the end goal was to have the following:

  ▾ blog/
    ▸ examples/
      actual-first-blog-post.mdx

Being the incessantly nerdy human being that I am, I did not allow myself to simply open the file management UI on my macbook to select all files and then drag them into an examples directory. I set out to figure out how to do this task via the command line.

The Final Output

First, I want to show the final script and then progressively provide more detail depending on whether you are:

• someone who is already familiar with xargs such as myself, or
• you've never used xargs before and you're not really following what the following bash command is doing

So the final bash command is this:

ls | grep -v 'examples' | xargs -I file mv file ./examples

If you're already familiar with xargs

The only thing you need to do is simply use the -I <reference_name> option and now you can use <reference_name> anywhere in the xargs command. Neat!

What the heck is this bash script .. what even is xargs?

let's break it down step by step, shall we?

First we have ls, which simply lists out the contents of a given directory, or the current working directory if none is specified.

Secondly we have the | character. This is to pipe the output of ls into the next command.

Thirdly, I am doing an inverse search using grep to find me the set of items that do not match the term examples. This then yields a list of terms that doesn't include examples.

If I have a directory structure such as:

cats
dogs
whales
rabbits

And I do ls | grep -v 'dogs', then I'll end up with the following output on my terminal:

cats
whales
rabbits

Lastly, we then use xargs to call some arbitrary command for each line generated by the previous command.

The best way to explain what this means is to give you a real life example. I use xargs a lot as a convenient way to delete old / stale / merged git branches that are on my machine.

In fact, let me actually do this on my machine and show you what the output is:

# In my work directory for our api server

➜  api-server git:(staging) ✗ git branch
  CAR-724-Ability-to-soft-delete-cases-CLEAN-HISTORY
  car-1086-read-only-on-conversations-for-SA
  car-1107-fa-account-activation
  car-1216-create-users-in-stream
  car-1234-include-user-info
  car-1396-add-survey-manager-to-cors-list
  car-1396-persist-survey
  car-648-add-linting-and-formatting
  car-944-create-firm-archiving-email-column
  car-994-exclude-users-from-welcome-message
  car-994-staging-qa-follow-up-fixes
  car-997-restrict-case-contacts-to-same-firm-or-none
  chore/add-test-for-refactored-household-profiles-utils
  chore/drop-unused-case-primary-contact-column
  chore/remove-unused-update-case-contact-code
  chore/upgrade-prisma
  production
* staging

Let's say I want to remove anything that is not the staging or production branch. To do that, I would run:

 git branch | grep -vE 'staging|production' | xargs git branch -D

And poof! I have now run git branch -D on each branch of interest!

Now my branch list looks as follows:

➜  api-server git:(staging) ✗ git branch
  car-994-staging-qa-follow-up-fixes
  production
* staging

Note that my grep call matched on a branch called car-994-staging-qa-follow-up-fixes ... but that's a story for another time.

So if we're paying attention, we see that xargs is taking each line from the previous expression and using it as argument to the expression that we've supplied to xargs.

xargs git branch -D <<branch_name>>

Hopefully now you understand what xargs is doing. Note that the xargs value is being applied AT THE END of our command.

What happens if we have a command that requires the argument from xargs to not be at the end? An example would be with mv. In this case we need a named reference.

So if we go back to the full shell command that I had above that solved my problem:

ls | grep -v 'examples' | xargs -I file mv file ./examples

You'll notice one extra addition, which is the -I file option. What I'm simply doing here is attaching a name to the "variable" or the line that I'm operating on, much like we would in a for loop:

for (int variableThatIAmReferencing = 0; ...) {}

Now that I have a named reference to the line that I'm operating on with xargs I can now use it anywhere in my command as you can see within my call to mv.

And that's it :)

Liked this post? Consider buying me a coffee :)