Professional Bash scripting: automating Linux administration
Exit

Lesson 1 of 12 · The language in depth

Variables, expansions and quoting: slice, replace and compute without ever breaking a file name

1 h Rocky Linux 9

Objective

Process strings inside bash itself (file name, extension, host name, case, default values, arithmetic), then know exactly what bash does with a line before running the command, so you choose the right kind of quotes every time. This is the foundation for the rest of the course, and the first cause of bugs in operations scripts.

Key points

  • #/## remove a pattern at the start (shortest / longest), %/%% at the end; ${v/a/b} replaces, ${v^^} and ${v,,} change case, ${#v} gives the length.
  • ${v:-default}, ${v:=default}, ${v:?message}, ${v:+other}: with the colon, an empty variable counts as missing.
  • $(( … )) computes with integers; printf -v var formats into a variable without a subshell.
  • Bash transforms a line in a fixed order: expansions, then word splitting, then globs. Double quotes block the last two steps.
  • "$@" passes each argument intact; printf %q protects a string that another shell will read again (ssh, replayable log).

This lesson assumes you know simple variables, ${1:-default} and the globs seen in the "Fundamentals" course (lessons 1.11, 1.13 and 1.17). We go much further.

1. Why avoid basename, cut or sed?

In a loop that processes 20,000 log files, every $(basename "$f") starts a new process. The expansion ${f##*/} does the same job inside bash, hundreds of times faster.

f=/var/backups/ml-web-01-2026-10-20.tar.gz

echo "${#f}"            # 40           length in characters
echo "${f:13:9}"        # ml-web-01    9 characters from position 13 (counting from 0)
echo "${f: -6}"         # tar.gz       the last 6 (the space before - is mandatory)

2. Removing a pattern at the start or at the end

Patterns are shell globs (*, ?, […]), not regular expressions.

Expansion Removes… Memory aid
${v#pattern} the shortest matching prefix # is to the left of $ on the keyboard
${v##pattern} the longest prefix
${v%pattern} the shortest suffix % is to the right
${v%%pattern} the longest suffix 
f=/var/backups/ml-web-01-2026-10-20.tar.gz
echo "${f##*/}"         # ml-web-01-2026-10-20.tar.gz   (= basename)
echo "${f%/*}"          # /var/backups                  (= dirname)
nom=${f##*/}
echo "${nom%.tar.gz}"   # ml-web-01-2026-10-20          exact extension removed
echo "${nom#*.}"        # tar.gz                        the "full" extension
echo "${nom##*.}"       # gz                            only the last extension

fqdn=ml-db-01.masterlinux.lab
echo "${fqdn%%.*}"      # ml-db-01                      short name
echo "${fqdn#*.}"       # masterlinux.lab               domain

for f in /srv/exports/*.CSV; do
    mv -- "$f" "${f%.CSV}.csv"       # bulk rename, without rename or sed
done

3. Replacing and changing case

ligne="agence de fes;agence de rabat;agence de oujda"
echo "${ligne/agence/AGENCE}"     # AGENCE de fes;agence de rabat;...   first one only
echo "${ligne//;/ | }"            # agence de fes | agence de rabat | ...  all of them
echo "${ligne//agence de /}"      # fes;rabat;oujda                     delete = replace with nothing

chemin=/srv/site/
echo "${chemin/#\/srv/\/data}"    # /data/site/     # = anchored at the START
echo "${chemin/%\//}"             # /srv/site       % = anchored at the END

role=web ; ENV=PROD
echo "${role^} ${role^^} ${ENV,,}"   # Web WEB prod

Case changes require bash 4 or later: every current distribution has it, but not Debian's sh (dash).

4. Default values, mandatory variables

Expansion If v is missing If v is empty If v has a value
${v-d} d empty the value
${v:-d} d d the value
${v:=d} d, and v receives d d, and v receives d the value
${v:?msg} error, the script stops error the value
${v:+x} empty empty x 
#!/bin/bash
# /usr/local/bin/ml-purge — configuration through environment variables
: "${RETENTION_JOURS:=14}"           # ":" does nothing: only the assignment matters
CIBLE=${1:?Usage : ml-purge HOTE (exemple : ml-app-01)}
echo "Purge sur $CIBLE au-delà de $RETENTION_JOURS jours${DRY_RUN:+ (simulation)}"

${v:+…} adds an option only when it makes sense: rsync -a ${VERBOSE:+-v} …. And indirection ${!nom} reads the variable whose name is stored in nom (nom=seuil_db; echo "${!nom}"); as soon as cases multiply, an associative array (lesson 2.2) is cleaner.

5. Computing: $(( )) and printf -v

echo $(( 45 * 100 / 60 ))                      # 75    INTEGER division (truncated)
echo $(( 2 * 100 / 3 ))                        # 66    while 66.67 rounds to 67
echo $(( (2 * 100 + 3 / 2) / 3 ))              # 67    rounding: add half of the divisor
echo $(( 17 % 5 )) $(( 2 ** 10 ))              # 2 1024
(( compteur += 1 ))                            # no $ needed inside (( ))

printf -v jour '%(%F)T' -1            # today's date, without running the date command
printf -v hote 'ml-web-%02d' 3        # hote=ml-web-03

Bash only knows integers: for amounts, compute in cents (12350 for 123.50 MAD) or hand the calculation to awk.

6. What bash does with a line

Take a variable containing a space and a star, in a directory that contains rapport de juin.txt and simple.txt:

motif="ventes  *"
echo $motif         # ventes rapport de juin.txt simple.txt     ← split THEN the star replaced
echo "$motif"       # ventes  *                                 ← intact

Without quotes, bash has: 1) replaced $motif with its value; 2) split it on spaces (the two spaces are gone); 3) replaced * with the list of files. Double quotes prevent steps 2 and 3. Golden rule: every expansion inside double quotes, unless you precisely want the splitting.

f="rapport de juin.txt"
wc -c $f          # wc: rapport: No such file or directory  (3 files requested)
wc -c "$f"        # 4 rapport de juin.txt

7. The three kinds of quotes

hote=ml-web-01
echo 'Serveur : $hote'          # Serveur : $hote          nothing is interpreted
echo "Serveur : $hote"          # Serveur : ml-web-01      $ is interpreted
echo "Prix : 120 \$"            # Prix : 120 $             \ protects one character inside "…"
echo $'Colonne1\tColonne2'      # $'…' understands \t, \n, \e
echo 'l'\''agence'              # l'agence   (close, \' , reopen: the only way to get ' inside '…')
Syntax $v, $( ) \n Typical use
'…' no no regexes, awk programs, sed patterns
"…" yes no almost everything else
$'…' no yes special characters (IFS=$'\n', tabs)

8. "$@", "$*" and IFS

# The script receives three arguments: "Rabat centre" Fes Agadir
for a in "$@"; do echo "[$a]"; done     # [Rabat centre] [Fes] [Agadir]     3 iterations
for a in "$*"; do echo "[$a]"; done     # [Rabat centre Fes Agadir]         1 iteration
for a in $@;   do echo "[$a]"; done     # [Rabat] [centre] [Fes] [Agadir]   4 iterations: broken

A wrapper script that adds options then hands over writes exec rsync -a --exclude='*.tmp' "$@": arguments intact, and the exit code is directly the one from rsync.

IFS (space, tab, newline by default) is used for word splitting and by read. Change it for a single command:

IFS=: read -r nom ip ville <<< "ml-db-01:10.0.0.20:casablanca"
echo "$nom / $ip / $ville"                  # ml-db-01 / 10.0.0.20 / casablanca

9. Globs under control, and names starting with -

By default, a glob that matches nothing stays as is: for f in /srv/vide/* runs once with the literal value /srv/vide/*.

Option Effect
shopt -s nullglob a glob with no match disappears: the loop does not run
shopt -s failglob a glob with no match is an error
shopt -s dotglob * includes hidden files
shopt -s globstar **/*.log goes down into every subdirectory
shopt -s extglob extended patterns: !(*.gz), `@(web

A file named -rf is taken as an option by rm *. Fixes: rm -- * (-- = end of options) or rm ./*.

10. printf %q: re-injecting a string safely

ssh joins its arguments and has them read again by the remote shell:

chemin="/srv/partage/compta 2026;reboot"
printf '%q\n' "$chemin"                             # /srv/partage/compta\ 2026\;reboot
ssh ml-web-01 rm -f "$chemin"                       # DANGER: the remote shell runs "reboot"
ssh ml-web-01 "rm -f $(printf '%q' "$chemin")"      # safe

A file name supplied by a user and passed as is to ssh lets someone run a command on the remote server: it is a real security hole.

11. Real case: the purge script that emptied the wrong directory

On a MasterLinux file server, a cleanup script contained rm -rf $DOSSIER/*. One day it was started without an argument by a badly written scheduled task: $DOSSIER was empty, and the command became rm -rf /*. Three protections, each one enough on its own:

DOSSIER=${1:?Usage : purge.sh DOSSIER}     # 1. mandatory argument
[ -d "$DOSSIER" ] || exit 1                # 2. it must exist
rm -rf -- "${DOSSIER:?}"/*                 # 3. ${v:?} refuses an empty value, quotes everywhere

Common pitfalls

  • Mixing up # and %, or forgetting the colon: ${PORT-8080} lets PORT="" through.
  • ${v: -3} without the space: ${v:-3} is a default value, not "the last three characters".
  • Unquoted variables: [ $v = oui ] with an empty v becomes [ = oui ]; for f in $(ls) breaks names. Always "$v", and for f in ./*.
  • Quotes stored inside a variable do nothing: opts='-a "mon dossier"'; cmd $opts passes "mon and dossier". For a list of arguments, use an array (lesson 2.2).

Lab

On Rocky Linux 9, you write six small scripts in /root/bash:

  1. nom-archive.sh: the name of an archive without directory or extension, without external commands;
  2. court.sh: the short name of a machine, in uppercase;
  3. conf.sh: default values that survive empty variables;
  4. pourcent.sh: a correctly rounded integer percentage;
  5. compte-args.sh: the arguments received, intact thanks to "$@";
  6. cmd-distante.sh: a safe ssh command with printf %q.

Quiz

  1. With f=/srv/www/index.old.html, what do ${f##*/}, ${f%.*} and ${f%%.*} give?
  2. What is the difference between ${PORT-80} and ${PORT:-80}?
  3. In which order does bash apply variable expansion, word splitting and globs?
  4. What is the difference between "$@" and "$*"?
  5. Why is ssh host rm -f "$f" dangerous if $f comes from a user?
Answers
  1. index.old.html; /srv/www/index.old; /srv/www/index.
  2. The first only replaces when PORT is missing; the second also replaces when PORT is empty.
  3. First the expansions ($v, $( ), $(( ))), then word splitting according to IFS, then globs. Double quotes prevent the last two steps.
  4. "$@" gives one word per argument, intact; "$*" joins all the arguments into a single string separated by the first character of IFS.
  5. ssh joins the arguments and has the remote shell interpret them again: a name containing ; or $( ) becomes a command there. The value must be protected with printf %q.

Practical exercises

To do on Rocky Linux 9

Do this lab on your own machine

The terminal is yours: a Linux VM (VirtualBox, VMware, Hyper-V, WSL…) or Docker/Podman. The ml tool prepares the exercise and checks your work; the lesson is validated here automatically.

Create a free account to get your personal command and have your labs checked automatically.

Create my account

Tasks

  1. Create the executable script /root/bash/nom-archive.sh PATH: it prints the archive name without its directory and without the .tar.gz extension. Example: /var/backups/ml-web-01-2026-10-20.tar.gz gives ml-web-01-2026-10-20. Not allowed: basename, dirname, sed, awk, cut.
    Show a hintTwo steps: f=${1##*/} removes everything up to the last /, then ${f%.tar.gz} removes the extension.
    Show the solution
    mkdir -p /root/bash && printf '%s\n' '#!/bin/bash' 'f=${1##*/}' 'echo "${f%.tar.gz}"' > /root/bash/nom-archive.sh && chmod +x /root/bash/nom-archive.sh

    Check : test "$(/root/bash/nom-archive.sh /var/backups/ml-web-01-2026-10-20.tar.gz 2>/dev/null)" = ml-web-01-2026-10-20 && test "$(/root/bash/nom-archive.sh archive.tar.gz 2>/dev/null)" = archive && ! grep -Eq "basename|dirname|sed|awk|cut" /root/bash/nom-archive.sh

  2. Create the executable script /root/bash/court.sh NAME: it prints the short name of a machine in UPPERCASE. ml-db-01.masterlinux.lab gives ML-DB-01; ml-web-01 (no domain) gives ML-WEB-01.
    Show a hint${1%%.*} keeps what comes before the first dot; ${h^^} converts to uppercase.
    Show the solution
    mkdir -p /root/bash && printf '%s\n' '#!/bin/bash' 'h=${1%%.*}' 'echo "${h^^}"' > /root/bash/court.sh && chmod +x /root/bash/court.sh

    Check : test "$(/root/bash/court.sh ml-db-01.masterlinux.lab 2>/dev/null)" = ML-DB-01 && test "$(/root/bash/court.sh ml-web-01 2>/dev/null)" = ML-WEB-01

  3. Create the executable script /root/bash/conf.sh: it prints port=P env=E. P comes from the PORT environment variable (8080 if it is missing OR empty); E comes from ENV_ML (dev if missing or empty). Example: PORT=9090 ENV_ML=prod /root/bash/conf.sh prints port=9090 env=prod.
    Show a hintWith the colon, ${PORT:-8080} also handles the empty variable. To set ENV_ML: : "${ENV_ML:=dev}"
    Show the solution
    mkdir -p /root/bash && printf '%s\n' '#!/bin/bash' 'port=${PORT:-8080}' ': "${ENV_ML:=dev}"' 'echo "port=$port env=$ENV_ML"' > /root/bash/conf.sh && chmod +x /root/bash/conf.sh

    Check : test "$(env -u PORT -u ENV_ML /root/bash/conf.sh 2>/dev/null)" = "port=8080 env=dev" && test "$(PORT=9090 ENV_ML=prod /root/bash/conf.sh 2>/dev/null)" = "port=9090 env=prod" && test "$(PORT= ENV_ML= /root/bash/conf.sh 2>/dev/null)" = "port=8080 env=dev"

  4. Create the executable script /root/bash/pourcent.sh USED TOTAL: it prints the ROUNDED integer percentage (1 3 gives 33, 2 3 gives 67, 45 60 gives 75). If TOTAL is 0, an error message on standard error and exit code 1.
    Show a hintInteger division truncates: add half of the divisor before dividing, $(( (100 * u + t / 2) / t )).
    Show the solution
    mkdir -p /root/bash && printf '%s\n' '#!/bin/bash' 'u=$1 t=$2' 'if (( t == 0 )); then' '    echo "Erreur : TOTAL ne peut pas valoir 0" >&2' '    exit 1' 'fi' 'echo $(( (100 * u + t / 2) / t ))' > /root/bash/pourcent.sh && chmod +x /root/bash/pourcent.sh

    Check : test "$(/root/bash/pourcent.sh 1 3 2>/dev/null)" = 33 && test "$(/root/bash/pourcent.sh 2 3 2>/dev/null)" = 67 && test "$(/root/bash/pourcent.sh 45 60 2>/dev/null)" = 75 && ! /root/bash/pourcent.sh 5 0 >/dev/null 2>&1

  5. Create the executable script /root/bash/compte-args.sh: it prints the number of arguments received, then each argument on its own line, in square brackets, WITHOUT changing it. Example: compte-args.sh "*" "x y" "" prints 3, [*], [x y], [] (the star is not replaced by files, both spaces stay).
    Show a hintecho "$#" then for a in "$@"; do echo "[$a]"; done — the quotes around $@ keep each argument whole.
    Show the solution
    mkdir -p /root/bash && printf '%s\n' '#!/bin/bash' 'echo "$#"' 'for a in "$@"; do' '    echo "[$a]"' 'done' > /root/bash/compte-args.sh && chmod +x /root/bash/compte-args.sh

    Check : test "$(cd / && /root/bash/compte-args.sh "*" "x y" "" 2>/dev/null)" = "$(printf "3\n[*]\n[x y]\n[]")" && test "$(/root/bash/compte-args.sh 2>/dev/null)" = 0

  6. Create the executable script /root/bash/cmd-distante.sh PATH: it PRINTS (without running it) the command ssh ml-web-01 rm -f PATH, where PATH is protected with printf %q. Example: "/tmp/a b" gives ssh ml-web-01 rm -f /tmp/a\ b.
    Show a hintprintf "ssh ml-web-01 rm -f %q\n" "$1"
    Show the solution
    mkdir -p /root/bash && printf '%s\n' '#!/bin/bash' 'printf "ssh ml-web-01 rm -f %q\n" "$1"' > /root/bash/cmd-distante.sh && chmod +x /root/bash/cmd-distante.sh

    Check : test "$(/root/bash/cmd-distante.sh "/tmp/a b" 2>/dev/null)" = "ssh ml-web-01 rm -f /tmp/a\ b" && test "$(/root/bash/cmd-distante.sh "/tmp/x;reboot" 2>/dev/null)" = "ssh ml-web-01 rm -f /tmp/x\;reboot" && grep -q "%q" /root/bash/cmd-distante.sh