Bash

The only way to really learn scripting is to write scripts – Mendel Cooper

Writing shell scripts leaves a lot of room to make mistakes, in ways that will cause your scripts to break on certain input, or (if some input is untrusted) open up security vulnerabilities… The simplest step is to avoid using shell at all – MIT Student Information Processing Board

Table of Content

• Create Symbolic Links 🔗
• Extract a substring from a string
• Extract a value from a file
• Padding characters with printf
• Read a JSON stream
• Replace multiline text with Bash
• Throbber \& Spinner
• Wait for several subprocesses to complete
• Tools for Shell Script Development ⚙️
• Features 🔎
• Coding Guidelines, Style, Linter ✍️

Create Symbolic Links 🔗

ln is a command-line utility for creating links between files. By default, the ln command creates hard links. To create a symbolic link, use the -s (–symbolic) option.

The ln command syntax for creating symbolic links is as follows: ln -s [OPTIONS] FILE LINK

e.g.: ln -s source_file symbolic_link

src 👉 Create Symbolic Links

Extract a substring from a string

With grep -Eo and a regex pattern

grep -Eo "http[^ ']+" <<EOF
curl 'https://cloud.sdu.dk/api/ingresses/browse?itemsPerPage=100&includeOthers=true&sortDirection=ascending'
-H 'Accept-Language: en-US,en;q=0.5'
-H 'Accept-Encoding: gzip, deflate, br'
-H 'Authorization: Bearer ****'
-H 'Content-Type: application/json; charset=utf-8'
-H 'Cookie: refreshToken=cf6220bd%2D650c%2D4a48%2D9836%2D58ea9ecb7567'
-H 'Sec-Fetch-Dest: empty'
-H 'Sec-Fetch-Mode: cors'
-H 'Sec-Fetch-Site: same-origin'
-H 'Pragma: no-cache'
-H 'Cache-Control: no-cache'
EOF

will output

https://cloud.sdu.dk/api/ingresses/browse?itemsPerPage=100&includeOthers=true&sortDirection=ascending

src 👉 stackoverflow.com

Extract a value from a file

Example of a pyproject.toml file

[tool.poetry]
name = "docs"
version = "0.1.0"
description = "Collection of Cheatsheets 🗒️ PostgreSQL 🐘, Git..."
repository = "https://github.com/JV-conseil/docs"

With grep and sed

grep -Eo "^repository.+$" pyproject.toml | sed -E 's/^repository = "(.+)"$/\1/'

will output

https://github.com/JV-conseil/docs

Padding characters with printf

for i in $(seq -f "%04g" 1 470)
do
  echo $i
done

will output

0001
0002
...
0469
0470

src 👉 How to zero pad a sequence of integers in bash so that all have the same width

printf '%0.1s' "."{0..10} will output ...........

src 👉 Padding characters in printf and printf syntax

Read a JSON stream

With jq

curl 'https://cloud.sdu.dk/api/ingresses/browse?itemsPerPage=100&includeOthers=true&sortDirection=ascending' \
-H 'Accept-Language: en-US,en;q=0.5' \
-H 'Accept-Encoding: gzip, deflate, br' -H 'Authorization: Bearer *****' \
-H 'Content-Type: application/json; charset=utf-8' \
--compressed | jq '.items[].specification.domain' >~/Downloads/example.json

will output

"app-9a7f8023b8b09392140f3ff9f12c91f2.cloud.sdu.dk"
"app-githubbing.cloud.sdu.dk"
"app-health-check.cloud.sdu.dk"
"app-health-status.cloud.sdu.dk"
"app-mission-ocean.cloud.sdu.dk"
"app-research-funding.cloud.sdu.dk"
"app-santa-maria-josefina-do-coracao-de-jesus-sancho-de-guerra.cloud.sdu.dk"
"app-thalassa.cloud.sdu.dk"
"app-yerun.cloud.sdu.dk"

src 👉 jq/manual

Replace multiline text with Bash

With ed line editor replace / update a part of multiline text in a text file with automatically generated content (eg: Table of contents) 👇

_lines=""

for i in {1..10}; do
  _lines+=$'\n'"${i}. ""$(openssl rand -hex 12)"
done

ed -s "./sample.md" <<EOF
/## BEGIN GENERATED/+,/## END GENERATED/-d
/## BEGIN GENERATED/a
${_lines}

.
wq
EOF

src 👉 gist

Throbber & Spinner

A throbber, also known as a loading icon, is an animated graphical control element used to show that a computer program is performing an action in background — wikipedia

# Usage: throbber & my_long_process ; kill %-1
throbber() {
  printf "Please wait..."
  while true; do
    printf "."
    sleep .3
  done
}

Add a display to let user wait in bash

• kill %-1 will kill the n-1'th backgrounded process

throbber &
sleep 10
kill %-1

will output

Please wait.............

Spinner alternative

alias spinner='while :; do for s in / - \\ \|; do printf "\n$s"; sleep .1; done; done'

src 👉 How to kill the (last - 1) PID and How to Write Better Bash Spinners

Wait for several subprocesses to complete

declare -A pids=()

# run processes and store pids in array
for i in {1..5}; do
  sleep "${i}" &
  pids["${i}"]=$!
done

# wait for all pids
for pid in "${pids[@]}"; do
  wait "${pid}"
done

src 👉 Wait for several subprocesses to complete

Tools for Shell Script Development ⚙️

	ShellCheck Integrates ShellCheck into VS Code, a linter for Shell scripts
	shell-format A formatter for shell scripts implementing shfmt parser, formatter, and interpreter See Google Shell Style Guide

General 📚

• Bash Reference Manual - The definitive reference on shell behavior
• Advanced Bash-Scripting Guide - An in-depth exploration of the art of shell scripting by Mendel Cooper
• Awesome Bash - A curated list of delightful Bash scripts and resources
• Shell Scripting for Beginners - How to Write Bash Scripts in Linux

Features 🔎

• The Ultimate Guide to Modularizing Bash Script Code by Shinichi Okada
• How to use a key-value dictionary in Bash
• Bash Parameter expansions
• How To Use Bash Parameter Substitution Like A Pro
• Bash Associative Array Cheat Sheet

Coding Guidelines, Style, Linter ✍️

• Google Shell Style Guide revision 2.02
• The Unofficial Bash Strict Mode - These lines deliberately cause your script to fail. Wait, what? Believe me, this is a good thing…
• Writing Safe Shell Scripts - MIT Student Information Processing Board

Sponsorship

If this project helps you, you can offer me a cup of coffee ☕️ :-)