bash Style Guide

bash Style Guide

(Based originally on Google Shell Style Guide r1.26.)

Background

Why this Guide?

The essential purpose of this guide is to ensure a consistency of style, an adherence to and improvement of best practices, and a generally positive direction for writing code.  As best practices are changed or discovered, we will endeavor to update this guide.  Nothing here is etched in stone and community involvement in this evolutionary process is encouraged (and likely essential).  After the Conclusion there is a section where we can offer code examples which fall outside the other structures of this guide.

Have fun!

Which Shell to Use

Scripts must start with #!/usr/bin/env bash and a minimum number of flags. Use set to set shell options so that calling your script as bash script_name does not break its functionality.

When to Use bash

While shell scripting isn’t a development language, it is used for writing various utility scripts.

This style guide is more a guide to best practices during its use rather than a suggestion that it be used for any particular deployment.

Further bash Guidelines

If you’re mostly calling other utilities and are doing relatively little data manipulation, shell is an acceptable choice for the task.

If performance matters, use something other than (faster than) shell.

Consider using Python for scripts that perform multiple parsing or transformation operations on collections/arrays.

Prefer shell if writing the script in Python would require several subprocess calls (provided the complexity doesn’t make using shell prohibitive).

For a script that is more than 100 lines long, Google prefers Python, bearing in mind that scripts grow.

Rewrite your script in another language early to avoid a time-consuming rewrite at a later date.

Prefer printf over echo as echo has a number of issues when run in scripts (especially portability).

When using printf, remember that you must explicitly add newline ( \n ) characters.

Shell Files and Interpreter Invocation

File Extensions

Executables should have an sh extension (script.sh).

Libraries must have an sh extension (library.sh) and should not be executable.

SUID/SGID

SUID and SGID are forbidden in shell scripts.

There are too many security issues with shell that make it nearly impossible to secure sufficiently to allow SUID/SGID.

While bash does make it difficult to run SUID, it’s still possible on some platforms which is why we’re being explicit about banning it.

Use sudo to provide elevated access if you need it.

Environment

STDOUT v STDERR

All error messages should go to STDERR.  This makes it easier to separate normal status from actual issues.

## 
function func_error() {
    printf '%s\n' "[$( date +'%Y-%m-%d_%H:%M:%S%z' )]: $@" >&2
} 

if ! do_something; then
    err "Unable to do_something"
    exit "${error_did_nothing}"
fi 
## 

Comments

File Header

Begin each file with a description of its contents.

Every file must have a top-level comment including a brief overview of its contents.

A ticket number for tracking is required. If one does not exist, create it.

A copyright notice and author information are optional.

## 
#! /usr/bin/env bash
#
# Purpose: Perform hot backups of Oracle databases.
# Ticket: https://ticket.site/Ticket/Display.html?id=12345 
## 

Function Comments

Any function that is not both obvious and short must be commented.

Any function in a library must be commented regardless of length or complexity.

It should be possible for someone else to learn how to use your program (or to use a function in your library) by reading the comments (and self-help, if provided) without reading the code.

All function comments should contain:

  • Description of the function
  • Global variables used and modified
  • Arguments taken
  • Returned values other than the default exit status of the last command run
#! /usr/bin/env bash
#
# Perform hot backups of Oracle databases.

export path='/usr/xpg4/bin:/usr/bin:/opt/csw/bin:/opt/goog/bin'

#######################################
# Cleanup files from the backup dir
# Globals:
# c_BACKUP_DIR
# c_ORACLE_SID
# Arguments:
# None
# Returns:
# None
####################################### 

cleanup() {
    ...
} 
## 

Implementation Comments

Comment tricky, non-obvious, interesting, or important parts of your code.  This follows general coding comment practice.

Don’t comment everything.  If there’s a complex algorithm or you’re doing something out of the ordinary, put a short comment.

ToDo Comments

Use ToDo comments for code that is temporary, a short-term solution, or good-enough but not perfect.  This approximates the convention in Google’s C++ Guide.

ToDo’s should include the string ToDo, followed by your username in parentheses.  It’s preferable to put a ticket number next to the ToDo item as well.

## 
# ToDo (john.doe):  Handle the unlikely edge cases (ticket:  ####) 
## 

Formatting

Always use the following for any new code; for existing code, follow the style that’s already there – unless the existing style violates current best-practices or is otherwise problematic.  In that case, update all instances of the existing style.

Indentation

Indentations should be 4-spaced tabs.  Set the indentations in your IDE accordingly. Keep indentation consistent.

Use blank lines between blocks to improve readability.

Line Length and Long Strings

Avoid line lengths longer than 80 characters.  If you have to write strings that are longer than 80 characters, consider using a here document or an embedded newline if possible.

Literal strings that have to be longer than 80 chars and can’t sensibly be split are ok, but it’s strongly preferred to find a way to make it shorter.

## 
# DO use here documents
cat <<END;
I am an exceptionally long
string.
END 

# Embedded newlines are ok too
long_string="I am an exceptionally
long string." 
## 

Pipelines

Pipelines should be split one per line if they don’t all fit on one line.  Indent for clarity.  If a pipeline all fits on one line, it should be on one line.

This applies to a chain of commands combined using the | as well as to logical compounds using the double || and &&.  Line these elements up where sensible.  Be sure to include the trailing \ which causes the executed code to act as though it is on a single line (by escaping the newline).

## 
# All fits on one line
command1 | command2

# Long commands (with escaped newlines)
command1 \
| command2 \
| command3 \
| command4 
## 

Loops

Put ; do and ; then on the same line as the while, for, or if.

Loops should follow the same principles for braces as when declaring functions.  However, else should be on its own line (and closing statements should be on their own lines vertically aligned with their opening statements).

## 
for dir in ${dirs_to_cleanup} ; do
    if [[ -d "${dir}/${c_ORACLE_SID}" ]] ; then
        log_date "Cleaning up old files in ${dir}/${c_ORACLE_SID}"
        rm "${dir}/${c_ORACLE_SID}/"*
        if [[ "$?" -ne 0 ]] ; then
            error_message
        fi
    else
        mkdir -p "${dir}/${c_ORACLE_SID}"
        if [[ "$?" -ne 0 ]] ; then
            error_message
        fi
    fi
done 
## 

case

Indent alternatives.

A one-line alternative needs a space after the close parenthesis of the pattern and before the ;;

Long or multi-command alternatives should be split over multiple lines with the pattern, actions, and ;; on separate lines.

The matching expressions are indented one level from the case and esac.  Multiline actions are indented another level.

Pattern expressions should not be preceded by an open parenthesis.

Avoid the ;& and ;;& notations.

## 
case "${expression}" in
a )
variable="..."
some_command "${variable}" "${other_expr}" ...
;;
absolute )
actions="relative"
another_command "${actions}" "${other_expr}" ...
;;
* )
error "Unexpected expression '${expression}'"
;;
esac 
## 

Simple commands may be put on the same line as the pattern and the ;; as long as the expression remains readable.  This is often appropriate for single-letter option processing.

When on the same line as the actions, use a space after the close parenthesis of the pattern and another before the ;;

## 
verbose='false'
aflag=''
bflag=''
files=''
while getopts 'abf:v' flag ; do
    case "${flag}" in
        a ) aflag='true' ;;
        b ) bflag='true' ;;
        f ) files="${optarg}" ;;
        v ) verbose='true' ;;
        * ) error "Unexpected option ${flag}" ;;
    esac
done 
## 

Variable Expansion

In order of precedence:  stay consistent with what you find; quote your variables; prefer “${var}” over “$var” – but see details.  Avoid brace-quote single character shell specials or positional parameters, unless strictly necessary or avoiding deep confusion.

## 
# Section of recommended cases.

# Preferred style for 'special' variables:
printf '%s\n' "Positional: $1" "$5" "$3"
printf '%s\n' "Specials: !=$!, -=$-, _=$_. ?=$?, #=$# *=$* @=$@ \$=$$ ..."

# Braces necessary:
printf '%s\n' "many parameters: ${10}"

# Braces avoiding confusion:
# Output is "a0b0c0"
set -- a b c
printf '%s\n' "${1}0${2}0${3}0"

# Preferred style for other variables:
printf '%s\n' "PATH=${PATH}, PWD=${PWD}, mine=${some_var}"
while read f ; do
    printf '%s\n' "file=${f}"
done <<(ls -l /tmp)

# Section of discouraged cases

# Unquoted vars, unbraced vars, brace-quoted single letter
# shell specials.
printf '%s\n' a=$avar "b=$bvar" "PID=${$}" "${1}"

# Confusing use:  this is expanded as "${1}0${2}0${3}0",
# not "${10}${20}${30}
set -- a b c
printf '%s\n' "$10$20$30" 
## 

Quoting

Always quote strings containing variables, command substitutions, spaces, or shell meta characters (unless careful unquoted expansion is required).

In short, unless there is a reason not to do it, format your variables as “${variable}” when calling them.

Prefer quoting strings that are words (as opposed to command options or path names).

Never quote literal integers as this makes an integer into a string.

Be aware of the quoting rules for pattern matches in [[

Use “$@” unless you have a specific reason to use $*

## 
# 'Single' quotes indicate that no substitution is desired.
# "Double" quotes indicate that substitution is required/tolerated.

# Simple examples
# "quote command substitutions"
flag="$(some_command and its args "$@" 'quoted separately')"

# "quote variables"
printf '%s\n'  "${flag}"

# "never quote literal integers"
value=32
# "quote command substitutions", even when you expect integers
number="$(generate_number)"

# "prefer quoting words", not compulsory
readonly const_USE_INTEGER='true'

# "quote shell meta characters"
printf '%s\n' 'Hello, stranger, and well met.  Earn lots of $$$'
printf '%s\n' "Process $$:  Done making \$\$\$."

# "command options or path names"
# ($1 is assumed to contain a value here)
grep -li Hugo /dev/null "$1"

# Less simple examples
# "quote variables, unless proven false": ccs might be empty
git send-email --to "${reviewers}" ${ccs:+"--cc" "${ccs}"}

# Positional parameter precautions: $1 might be unset
# Single quotes leave regex as-is.
grep -cP '([Ss]pecial|\|?characters*)$' ${1:+"$1"}

# For passing on arguments,
# "$@" is right almost everytime, and
# $* is wrong almost everytime:
#
# * $* and $@ will split on spaces, clobbering up arguments
# that contain spaces and dropping empty strings;
# * "$@" will retain arguments as-is, so no args
# provided will result in no args being passed on;
# This is in most cases what you want to use for passing
# on arguments.
# * "$*" expands to one argument, with all args joined
# by (usually) spaces,
# so no args provided will result in one empty string
# being passed on.
# (Consult 'man bash' for the nit-grits ;-)
set -- 1 "2 two" "3 three tres"; printf $# ; set -- "$*"; printf "$#, $@")
set -- 1 "2 two" "3 three tres"; printf $# ; set -- "$@"; printf "$#, $@") 
## 

Features and Bugs

Command Substitution

Nested backticks require escaping the inner backticks with \

Prefer $( command ) instead of backticks.  The $( command ) format doesn’t change when nested and is easier to read.

## 
# This is preferred:
var="$( command "$( command1 )" )"

# This is not:
var="`command \`command1\``" 
## 

test, [, and [[

[[ … ]] is preferred over [, test, and /usr/bin/[

[[ … ]] reduces errors as no pathname expansion or word splitting takes place between [[ and ]]

plus [[ … ]] allows for regular expression matching where [ … ] does not.

## 
# This ensures the string on the left is made up of characters in the
# alnum character class followed by the string name.
# Note that the RHS should not be quoted here.
# For the gory details, see
# E14 at https://tiswww.case.edu/php/chet/bash/FAQ
if [[ "filename" =~ ^[[:alnum:]]+name ]] ; then
    printf '%s\n' "Match"
fi

# This matches the exact pattern "f*" (Does not match in this case)
if [[ "filename" == "f*" ]] ; then
    printf '%s\n' "Match"
fi

# This gives a "too many arguments" error as f* is expanded to the
# contents of the current directory
if [ "filename" == f* ] ; then
    printf '%s\n' "Match"
fi 
## 

Testing Strings

Since bash is smart enough to deal with an empty string in a test (and given that the code is much easier to read), use tests for empty/non-empty strings or empty strings rather than filler characters.

Use quotes rather than filler characters where possible.

## 
# Do this:
if [[ "${my_var}" = "some_string" ]] ; then
do_something
fi

# -z (string length is zero) and -n (string length is not zero) are
# preferred over testing for an empty string
if [[ -z "${my_var}" ]] ; then
    do_something
fi

# This is ok (ensure quotes on the empty side) but not preferred:
if [[ "${my_var}" = "" ]] ; then
    do_something
fi

# Not this:
if [[ "${my_var}X" = "some_stringX" ]] ; then
    do_something
fi
Avoid confusion about your test by explicitly using -z or -n
# Use this
if [[ -n "${my_var}" ]] ; then
do_something
fi

# Instead of this as errors can occur if ${my_var} expands to a test
# flag
if [[ "${my_var}" ]] ; then
    do_something
fi 
## 

Wildcard Expansion of File Names

Use an explicit path when doing wildcard expansion of filenames.

As filenames can begin with a -, it’s a lot safer to expand wildcards with ./* instead of *

## 
# Here's the contents of the directory:
# -f -r somedir somefile

# This deletes almost everything in the directory by force
psa@bilby$ rm -v *
removed directory: `somedir'
removed `somefile'

# As opposed to:
psa@bilby$ rm -v ./*
removed `./-f'
removed `./-r'
rm: cannot remove `./somedir': Is a directory
removed `./somefile' 
## 

eval

Avoid eval.  It munges the input when used for assignment to variables and can set variables without making it possible to check what those variables were.

## 
# What does this set?
# Did it succeed?  In part or whole?
eval $( set_my_variables )

# What happens if one of the returned values has a space in it?
variable="$( eval func_someFunction )" 
## 

Pipes to while

Use process substitution or for loops in preference to piping to while.  Variables modified in a while loop do not propagate to the parent because the loop’s commands run in a subshell.  The implicit subshell in a pipe to while can make it difficult to track down bugs.

## 
last_line='NULL'
your_command | while read line ; do
last_line="${line}"
done

# This will output 'NULL'
printf '%s\n'  "${last_line}" 
## 

Use a for loop if you are confident that the input will not contain spaces or special characters (usually, this means not user input).

## 
total=0
# Only do this if there are no spaces in return values.
for value in $( command ) ; do
    total+="${value}"
done 
## 

Using process substitution allows redirecting output but puts the commands in an explicit subshell rather than the implicit subshell that bash creates for the while loop.

## 
total=0
last_file=
while read count filename ; do
    total+="${count}"
    last_file="${filename}"
done <<(your_command | uniq -c) 

# This will output the second field of the last line of output from
# the command.
printf '%s\n'  "Total = ${total}"
printf '%s\n' "Last one = ${last_file}" 
## 

Use while loops where it is not necessary to pass complex results to the parent shell; this is typically where some more complex parsing is required.

Beware that simple examples are probably more easily done with a tool such as awk.  This may also be useful where you specifically don’t want to change the parent scope variables.

## 
# Trivial implementation of awk expression:
# awk '$3 == "nfs" { print $2 " maps to " $1 }' /proc/mounts
cat /proc/mounts | while read src dest type opts rest ; do
    if [[ ${type} == "nfs" ]] ; then
        printf '%s\n' "NFS ${dest} maps to ${src}"
    fi
done 
## 

Naming Conventions

Function Names

Names for both functions and variables should be clear as to purpose and origin.

Lower-case with underscores to separate words are preferred (while camel case is also acceptable).

So that functions are easily identifiable at all locations in the code it is preferred to prepend them with either f_ or func_ (so f_my_function or func_my_function).

Separate libraries with ::

Parentheses are required after the function name.

If you’re writing a package, separate package names with ::

Braces will typically be on the same line as the function name.

The function keyword is extraneous when the () is present after the function name, but this enhances quick identification of functions and thus is preferred.  (It also allows easy transformation of functions when porting to another shell.)

## 
# Single function
function func_my_function() {
    ...
}

# Part of a package
mypackage::func_my_function() {
    ...
} 
## 

Variable Names

Names for both functions and variables should be clear as to purpose and origin.

There are three kinds of variables which are marked with prepends:  locals, constants (read-only), and arrays (both types).

Local variables  get the l_ or loc_ prepend (l_variable or loc_variable).  Constants or read-only variables get c_ or const_ prepend (c_variable, c_VARIABLE, const_variable, or const_VARIABLE); if it is your preference to ALL-CAPS your constants (a la C languages), only do so after the prepend.  Finally, arrays and associative arrays should get a_ aa_ (or A_ or AA_ but be sure to avoid an all-caps array name), respetively.

Variable names within loops should be similarly named for any variable they are looping through.

## 
for zone in ${zones} ; do
    something_with "${zone}"
done 
## 

Constants and Environment Variable Names

Names for both functions and variables should be clear as to purpose and origin.

Environment and system variables are all-caps variables; as such it is best to avoid using all-caps for your own variables (and thus avoid conflicts).

So that constants (read-only variables) are easily identifiable at all locations in the code it is preferred to prepend them with either c_ or const_ (so c_my_read-only).

Other languages use all caps for constants and so some may prefer that, but we can still avoid conflict by prepending those with c_ or const_ (so either const_VARIABLE or c_VARIABLE).

Declarations for these variables should come near the top of the file.

## 
# Constant
readonly c_PATH_TO_FILES='/some/path'

# Both constant and environment
declare -xr c_ORACLE_SID='PROD' 
## 

Some things become constant at their first setting (for example, via getopts).  Thus, it’s ok to set a constant in getopts or based on a condition, but it should be made read-only immediately afterwards.  Note that declare doesn’t operate on global variables within functions, so readonly or export are recommended instead.

## 
c_VERBOSE='false'
while getopts 'v' flag ; do
    case "${flag}" in
        v) c_VERBOSE='true' ;;
    esac
done
readonly c_VERBOSE 
## 

Source File Names

Lowercase with underscores to separate words if desired.  (Prefer underscore style over camel case, though both are acceptable.)  Avoid spaces in file paths!

Be consistant.

Read-Only Variables (aka Constants)

Names for both functions and variables should be clear as to purpose and origin.

When you declare a variable that is meant to be read-only, make this explicit.

Use readonly or declare -r to ensure they’re read only.

As globals are widely used in shells, it’s important to catch errors when working with them.

## 
c_zip_version="$( dpkg --status zip | grep Version: | cut -d ' ' -f 2 )"
    if [[ -z "${c_zip_version}" ]] ; then
        error_message
    else
        readonly c_zip_version
    fi 
## 

Local Variables

Names for both functions and variables should be clear as to purpose and origin.

To make local variables easy to parse and locate within the code they are prepended with l_ or loc_ (as l_my_local_variable or loc_my_local_variable).

Declare function-specific variables with local.  Ensure that local variables are only seen inside a function and its children by using local when declaring them.

Declaration and assignment should be on different lines.  This is especially true when the assignment value is provided by a command substitution as local does not propagate the exit code from the command substitution.

## 
func_my_function2() {
    local loc_name="$1"

    # Separate lines for declaration and assignment:
    local loc_my_var
    loc_my_var="$( func_my_function )" || return

# DO NOT do this:  $? contains the exit code of 'local' not func_my_function
    local loc_my_variable="$(func_my_function)"
    [[ $? -eq 0 ]] || return
    ...
} 
## 

Function Location

Put all functions together at the top of the file just below the constants.

Don’t hide executable code between functions!

Only includes, set statements, and constants should be done before declaring functions.

main

A function called main is required for scripts long enough to contain at least one other function.  (A script containing only one function ought to be main regardless.)

In order to easily find the start of the program, put the main program in a function called main as the bottom most function.

This provides consistency with the rest of the code base as well as allowing you to define more variables as local (which can’t be done if the main code is not a function).

The last non-comment line in the file should be a call to main:

main “$@”

Calling Commands

Checking Return Values

Always check return values and give informative return values.

For unpiped commands use $? or check directly via an if statement to keep it simple.

## 
if ! mv "${file_list}" "${dest_dir}/" ; then
    printf '%s\n' "Unable to move ${file_list} to ${dest_dir}" >&2
    exit "${error_bad_move}"
fi 
## 

bash also has the PIPESTATUS variable that allows checking of the return code from all parts of a pipe.  If it’s only necessary to check success or failure of the whole pipe, then the following is acceptable:

## 
tar -cf - ./* | ( cd "${dir}" && tar -xf - )
if [[ "${PIPESTATUS[0]}" -ne 0 || "${PIPESTATUS[1]}" -ne 0 ]]; then
    printf '%s\n' "Unable to tar files to ${dir}" >&2
fi 
## 

However, PIPESTATUS will be overwritten as soon as you do any other command.

If you need to act differently on errors based on where it happened in the pipe, you’ll need to assign PIPESTATUS to another variable immediately after running the command.

Remeber that [ is a command and will wipe out PIPESTATUS.

## 
tar -cf - ./* | ( cd "${dir}" && tar -xf - )
return_codes=(${PIPESTATUS[*]})
    if [[ "${return_codes[0]}" -ne 0 ]] ; then
        do_something
    fi
    if [[ "${return_codes[1]}" -ne 0 ]] ; then
        do_something_else
    fi 
## 

Built-in Commands v External Commands

Given the choice between invoking a shell built-in and invoking a separate process, choose the built-in.

Prefer the use of built-ins such as the Parameter Expansion functions in bash(1) as it’s more robust and portable (especially when compared to things like sed).

## 
# Prefer this:
addition=$((${x} + ${y}))
substitution="${string/#foo/bar}"

# Instead of this:
addition="$(expr ${x} + ${y})"
substitution="$( printf "${string}" | sed -e 's/^foo/bar/' )"
## 

Conclusion

Use common sense and be consistent!

Code Examples and Other Useful Tips

read

There are some scripts which make unusual use of read.  Here are some suggestions for fixing these and for future scripts.

## 
# Offer a default.  (The -e is required for the -i to work.)
read -e -p "Enter the path to the file:  " -i "/usr/local/etc/ " file_path
# or
read -ep "You may accept the default or delete it and type another: " -i "${mail_to}" mail_to

# Add a newline.
read -p "Please Enter a Message: "$'\n' message
printf '%s\n' "${message}"

# Do this...
read -e -p "Do you want to go again?  [y/n] "
printf '%s\n' "${REPLY}"
# ... or possibly this...
read -ep "Do you want to go again?  [y/n] " answer
printf '%s\n' "${answer}"
# ... but never this.
read "Do you want to go again?  [y/n] "
answer=$REPLY
printf '%s\n' "${answer}"
## 

printf

As mentioned above, we should be using printf in scripts and avoiding echo.  Here are some pointers for printf.

## 
# New lines.
printf '%s\n' "" "Make some space if you want your message to stand out. " ""

# More than one line.
printf "You can
make multi-line
prints as well.
"

# As printf has no included newline it must be specified.
printf "Just remember printf behaves like echo -n. "
printf '%s\n' "So if you want a new line, specify it using \\\n. " ""

# Though there is likly no issue with using echo to output an empty line, you may as well use a good habit.
printf '\n'

# You want some color?
# (A list exists here:  https://gist.github.com/chrisopedia/8754917 )
printf "Here is some \e[0;31mred\e[0m text. "
printf "Here is some \e[31mred\e[m text. "
printf '\n' 

## 

$?

This can be a very useful variable.  You can use it to test the results of a previous command as it contains the return status code of whatever came before it.

## 
# $? contains the return status code of the previous command.
$ echo
$ echo $?
0
$ [ ]
$ echo $?
1
$ []
[]: command not found
$ echo $?
127 
## 

You may want to change the case of a string you have stored in a variable.  This is the simplest way we’ve found.

## 
lowercase="${UPPERCASE,,}"
UPPERCASE="${lowercase^^}" 
## 

Of course you can also force case in your variable declaration.

## 
# for lower-case letters conversion
declare -l lowerCaseLetters
# for upper-case letters conversion
declare -u upperCaseLetters
# any text input for variables thus declared will convert to the specified case 
## 

decimal numeration and math

Let’s look at how we might handle decimal numeration.

## 
declare numberWithDecimals
numberWithDecimals="16.687"
# print it as it is
printf '%s\n' "Original number: ${numberWithDecimals}"
# print the number with the decimal places removed
printf '%s\n' "Clipped: ${numberWithDecimals%.*}"
# printf the number rounded
printf '%s' "Rounded: " ; printf '%.0f' "${numberWithDecimals}" ; printf '\n' 
## 

What about more basic math problems?

## 
# addition should work similarly
firstNumber=7
secondNumber=4
# the set of double parentheses provide a space for doing math
printf '%d\n' "$(( ${firstNumber} - ${secondNumber} ))" 
## 

Suppose you like decimals.

## 
# get a start and stop time (or any pair of decimal numbers)
time_start="$( date +%s.%N )"
time_end="$( date +%s.%N )"
# subtract them via bc into another variable
time_elapsed="$( printf '%b\n' "${time_end} - ${time_start}" | bc )"
# and trim that output to three decimal places
time_elapsed="$( printf '%0.3f' "${time_elapsed}" )" 
## 

Common Functions Farm

Function for pausing input to wait for the user to continue.
## 
function func_EnterToContinue() {
# just waits for the user before proceeding; a timer could be added later
read -rp "Press [Enter] to continue... "
} 
## 
Check root and sudo.

It is a good idea to test for root use. It may also be useful to ensure a user is using or not using sudo when running a script.

## 
function func_test_root(){
    if [ ! "${USER}" = root ] ; then
        printf '%s\n' "If this is not run with sudo, something something. " ""
        exit 0
    elif [[ "${USER}" == root ]] ; then # check if some naughty monster is logged in as root
        if [[ "${SUDO_USER}" == "" ]] ; then # sudo is ok
        printf '%s\n' "" "It is a bad practice to log in as root. Use sudo instead. " ""
        exit 0
        fi
    fi
} 
## 

Or you may want to get the current user while testing for root and sudo.

## 
function func_GetCurrentUser() {
    if [[ ! "${USER}" == root ]] ; then
        scriptUser_linux="${USER}"
    elif [[ "${USER}" == root ]] ; then # check if some naughty monster is logged in as root
        if [[ "${SUDO_USER}" == "" ]] ; then # sudo is ok
            printf '%s\n' "" "It is a bad practice to log in as root. " "Log in as yourself and use sudo if necessary. " ""
            exit 0
        fi
        scriptUser_linux="${SUDO_USER}"
    fi
} 
## 
Get a directory for working.

Function for obtaining working directory from user input.

## 
function func_getContainingFolder() {
    # obtain directory in which to work
    printf '%s\n' "Hello. " ""
    printf '%s\n' "Crtl-c at any time abandons any changes and exits the script. " ""
    while [ ! -d "${containingFolderPath}" ] ; do
        read -rep "Please provide the containing folder for the files to be renamed: " -i "${containingFolderPath}" containingFolderPath
        # expand the ~/ if it gets submitted
        containingFolderPath="${containingFolderPath/#~/${HOME}}"
        # fix spaces to be used in a quoted variable
        containingFolderPath="${containingFolderPath//\\/}"
        if [ -d "${containingFolderPath}" ] ; then
            printf '%s\n' "I have confirmed this is a directory. "
            printf '%s\n' "${containingFolderPath}" ""
        else
            printf '%s\n' "I cannot confirm this is a directory. "
            printf '%s\n' "${containingFolderPath}" ""
        fi
    done
} 
## 
Get numeric input.

You may want to ensure input is numeric.  Here is a pair of functions for testing ticket numbers.

## 
function func_TestTicket_isNumeric() {
    ticket_number=$( printf '%s' "${1}" | grep -E -o '[0-9]+' )
    if [[ "${1}" =~ ^[0-9]+$ ]] ; then # Check passed value as integer with a regex
        return 0 ;
    else
        return 1 ;
    fi
} 

function func_TestTicket_isNumeric_loop() {
    until func_TestTicket_isNumeric "${ticket_number}" ; do
        read -rep "Ticket number: " -i "${ticket_number}" ticket_number
    done
} 
## 

Enhance bash

Functions can also be used to enhance bash itself.  Here is a function for launching two scripts.  (Since this lives in a bashrc file and not in a script it lacks the func_ prepend.)

## 
# add to /etc/bash.bashrc file for a function for all users

# function for using termuser_auto and termuser_manual shell scripts
function termuser() {
case "${1}" in
-a|--auto) termuser_auto.sh ; return 0 ;;
-m|--manual) termuser_manual.sh ; return 0 ;;
*)
printf '%s\n' "In future you can call this funciton using -a or -m. "
read -rp "Automated or manual? (a/m) " -n1
if [[ "${REPLY}" == "a" ]] ; then
termuser_auto.sh
return 0
elif [[ "${REPLY}" == "m" ]] ; then
termuser_manual.sh
return 0
fi ;;
esac
printf '%s\n' "Oops. "
return 1
} 
## 
Share

Function v Method

I’ve been trying to get my head around the difference between a function and a method.  Part of the trouble in clearing this distinction is that, depending on the language in question, the naming conventions are all over the place.  C never calls anything a method while Java and C# never call anything a function.  Yet each does have both?  Yeah.  A rose by any other name would make people scratch their heads.

In essence, we can rephrase this problem as Classless Functions v Classed Functions.  A Classless Function is a function not associated with a class (nor a struct, C; nor a type, Go) while a Classed Function is a function tied to a class.  To call a classless, one merely calls the function by name, while to call a classed, one must call it using dot notation (class.function).

A Classless Function may be called a function or a static method and a Classed Function may be called a member function or a method.  See how this adds to the confusion?  No wonder developers get paid extra.

This question page have several answers which attempt to dive into this problem:

What’s the difference between a method and a function?

Share

Bookmarklets

Err… what’s a Bookmarklet?

A bookmarklet is a bit of JavaScript disguised as a bookmark that, when run, can perform a specific bit of work via your browser.  These below are generally built to satisfy some search need, to make these searches easier, more seamless, and less annoying.  Try one; try them all!

I have tested these and they all work as-is in the latest (2023-01-19) versions of Opera, Chrome, and Firefox.

If you run into trouble, try updating your browser or disabling all extensions before altering the JavaScript.

(If clicking the bookmarklet dialog does not spawn a new tab with your target, I have seen Firefox act a fool and need changed to %27.  But probably check your updates and extensions.)

HowTo:  Install and Use a Bookmarklet

The short version:

  • Triple-click the line of JavaScript code to highlight it.
  • Drag and drop that line of code to your browser’s Bookmarks Bar to create a bookmark.
    • If drag-and-drop doesn’t work, created any bookmark and edit to substitute the js as the destination and name it accordingly.
  • Right-click that newly created bookmark to edit the name.
    • Suggested names are provided above each JS line.  I like to keep the names short so more fit on the bar.
  • Rinse and repeat for all you’d like below.  Or create your own.

Usage

Once you have a bookmarklet installed, there are two ways to use it.  You can either highlight any text on the page and click the bookmarklet, or (if no text is highlighted on the page) a search dialog appears when the bookmarklet is clicked.

Of note, iframe contents cannot be seen, only the wrapping page.  In these circumstances a dialog will always spawn.

These are all built to open a new tab, which should open next to (to the right of) your current working tab.

These can be leveraged for many searches one does throughout the day, thus greatly increasing the efficiency of said searches.  I’ve been building and using them since 1999 and will continue to do so.

Bookmarklets

GitHub

// 
// GitHub https://github.com/search?q=searchthis 
// gitH 
javascript:q=&amp;quot;&amp;quot;+(window.getSelection?window.getSelection():document.getSelection?document.getSelection():document.selection.createRange().text);if(!q)q=prompt(&amp;quot;Search GitHub for… &amp;quot;).replace(/\s\+/g,&amp;quot;%252B&amp;quot;);if(q!=null)window.open(&amp;quot;https://github.com/search?q=&amp;quot;+q);void(0); 
// 

Amazon

// 
// Amazon https://www.amazon.com/s?k=searchthis&amp;amp;crid=2SHRP43MFNSJ8&amp;amp;sprefix=searchthis%2Caps%2C386&amp;amp;ref=nb_sb_noss 
// Amazon+s 
javascript:q=&amp;quot;&amp;quot;+(window.getSelection?window.getSelection():document.getSelection?document.getSelection():document.selection.createRange().text);if(!q)q=prompt(&amp;quot;Search Amazon for… &amp;quot;).replace(/\s\+/g,&amp;quot;%252B&amp;quot;);if(q!=null)window.open(&amp;quot;https://www.amazon.com/s?k=&amp;quot;+q);void(0); 
// 

LinkedIn

// 
// LinkedIn https://www.linkedin.com/search/results/all/?keywords=searchthis&amp;origin=GLOBAL_SEARCH_HEADER&amp;sid=cVD  
// LI+s 
javascript:q=&quot;&quot;+(window.getSelection?window.getSelection():document.getSelection?document.getSelection():document.selection.createRange().text);if(!q)q=prompt(&quot;Search Amazon for… &quot;).replace(/\s\+/g,&quot;%252B&quot;);if(q!=null)window.open(&quot;https://www.linkedin.com/search/results/all/?keywords=&quot;+q);void(0); 
// 

Confluence

Adjust URL for your instance.

// 
// Confluence 
// https://esentire.atlassian.net/wiki/search?text=narf 
// C+s 
javascript:q=&amp;quot;&amp;quot;+(window.getSelection?window.getSelection():document.getSelection?document.getSelection():document.selection.createRange().text);if(!q)q=prompt(&amp;quot;Confluence [try words]…&amp;quot;).replace(/\s\+/g,&amp;quot;%252B&amp;quot;);if(q!=null)window.open('https://esentire.atlassian.net/wiki/search?text='+q);void(0); 
// 

SolarWinds

Adjust URL for your instance.

// 
// SolarWinds https://YOURINSTANCE/ui/search?q=%s 
// SW+s 
javascript:q=&amp;quot;&amp;quot;+(window.getSelection?window.getSelection():document.getSelection?document.getSelection():document.selection.createRange().text);if(!q)q=prompt(&amp;quot;SolarWinds [try a sensor number]…&amp;quot;).replace(/\s\+/g,&amp;quot;%252B&amp;quot;);if(q!=null)window.open('https://YOURINSTANCE/ui/search?q='+q);void(0); 
// 

ServiceNow

Adjust URL for your instance.

// 
// ServiceNow ticket search https://YOURINSTANCE.service-now.com/nav_to.do?uri=%2F$sn_global_search_results.do%3Fsysparm_search%3D%s 
// SN+s 
javascript:q=&amp;quot;&amp;quot;+(window.getSelection?window.getSelection():document.getSelection?document.getSelection():document.selection.createRange().text);if(!q)q=prompt(&amp;quot;Service-Now [try a sensor number]…&amp;quot;).replace(/\s\+/g,&amp;quot;%252B&amp;quot;);if(q!=null)window.open('https://YOURINSTANCE.service-now.com/nav_to.do?uri=%2F$sn_global_search_results.do%3Fsysparm_search%3D'+q);void(0); 

// ServiceNow direct ticket search (no iframe) 
// this takes you to a ticket directly if search results are unique 
// SN+cs 
javascript:q=&amp;quot;&amp;quot;+(window.getSelection?window.getSelection():document.getSelection?document.getSelection():document.selection.createRange().text);if(!q)q=prompt(&amp;quot;ServiceNow Case Number: &amp;quot;).replace(/\s\+/g,&amp;quot;%252B&amp;quot;);if(q!=null)window.open('https://YOURINSTANCE.service-now.com/text_search_exact_match.do?sysparm_search='+q);void(0); 

// ServiceNow knowledge base article search 
// SN+KB 
javascript:q=&amp;quot;&amp;quot;+(window.getSelection?window.getSelection():document.getSelection?document.getSelection():document.selection.createRange().text);if(!q)q=prompt(&amp;quot;Search for a KB&amp;quot;).replace(/\s\+/g,&amp;quot;%252B&amp;quot;);if(q!=null)window.open('https://esentire.service-now.com/selfservice?id=search&amp;amp;spa=1&amp;amp;q='+q);void(0); 
// 

Under the Hood

As you will note from the JavaScript, each of these bookmarklets performs a variable substitution (here named q) into the URL sent back to the server performing the search.  As such, one would perform any URL substitution.

For example, at one job I was able to create several bookmarklets for accessing various config pages on devices by substituting the device names.

// 
// so if you begin with a URL like  
// https://devicename.company.internal:5555/path/to/configX 
// you can massage that into a bookmarklet something like  
javascript:q=&amp;quot;&amp;quot;+(window.getSelection?window.getSelection():document.getSelection?document.getSelection():document.selection.createRange().text);if(!q)q=prompt(&amp;quot;Config X for device…&amp;quot;).replace(/\s\+/g,&amp;quot;%252B&amp;quot;);if(q!=null)window.open(&amp;quot;https://&amp;quot;+q+&amp;quot;.company.internal:8834/path/to/configX&amp;quot;);void(0); 
// 

The possibilities are pretty limitless.

Share

Quickly Make Directories Based on Files in Location

I needed to make a bunch of folders for a slew of singles I’d downloaded.  I wanted each single to have its own folder but when you download singles they tend to offer just the flac file and no containing folder.  Reasonable, I suppose.

Anyway, I wrote this to create the folders.

##
for f in ./* ; do ff=$( printf '%s\n' "${f//.\//}" ) ; mkdir "${ff//.flac/}" ; done 
##

Good enough.  I could have added something to move each file (“${f}”) into the newly created folder (./”${ff}”) but didn’t think of that until I was done.  Probably an addition like this (untested):

##
for f in ./* ; do ff=$( printf '%s\n' "${f//.\//}" ) ; mkdir "${ff//.flac/}" ; mv "${f}" "./${ff}/${f}" ; done 
##

Enjoy! Maybe I’ll get around to testing the move version at some point.

Share

Sorting Lines in VSCode

Stay tuned for a more elegant version…

copy some lines then

This comes up at the top of searches still so I’ll add this latest solution. If you are running a current VS Code (and why wouldn’t you?) you can use the built-in sorter by hitting ctrl-shift-p (Mac is cmd-shift-p) and type “sort” in the subsequent search box. There are many options.

https://stackoverflow.com/questions/3350793/sort-selected-text-from-visual-studio-context-menu

Share

High Bitrate ISO Music to FLAC

Here is the process I used to convert some DVD-audio ISO files into proper FLAC files for inclusion in my main collection.  No compromise in channels nor in audio quality required.

These instructions are for Ubuntu but will likely work on most Linux distributions (since ffmpeg and flac are so common).  Mac?  Maybe.  Win10 with bash?  Probably.

The basics are as follows:

  • mount the ISO (or extract the relevant files)
  • use ffmpeg to extract audio data into wav files
  • stitch multiple wav files together using Audacity or ffmpeg
  • split that full-album wav into individual songs
  • convert those wav songs into FLAC files
  • tag and move into the main collection

Often mounting an ISO is a matter of right-clicking and choosing Mount; regardless I leave it up to you to make that happen in whatever manner you choose.  All that matters is that we can access the files through a specific file path. (Alternatively you can extract the relevant files to an accessible file path for the subsequent commands.)

You will need to fun ffmpeg against each audio file from the mounted ISO in order to get a complete collection of wav files to conjoin.  My most recent ISO had two and this command is for the second extraction:

##
ffmpeg -i ~/Desktop/ISO/AUDIO_TS/ATS_01_2.AOB ~/Desktop/RW/02_end.wav 

# run ^^this^^ line for each file containing audio and with a unique file name (in order) 

# you can use the following line to stitch two (or more by modification) wav files into a single long file 

ffmpeg -i ./01_start.wav -i ./02_end.wav -filter_complex '[0:0][1:0]concat=2:v=0:a=1[out]' -map '[out]' full.wav 

# you can see above you'd need to add more -i (input) files and you'd want to alter the bit before and the bit after concat to correspond with the correct number of files 
##

Either use the above command to stitch all files together or open all your files in Audacity (or whatever you’d like) and paste each track against the end of the previous track until you have one project containing the entire audio content.  Export that as a single wav file.

Audacity will default to exporting as a two-channel 16×44 wav file.   You can change the bitrate in the lower-left of the project window.  For multi-channel exports, you’ll need to navigate to Edit → Preferences → Import/Export → When exporting tracks to an audio file → Use custom mix (radio button).  Now you are set to wave the full wave file.

Next you get to struggle with dividing the full file into individual songs.  This can be made all the more challenging if your album is progressive rock.  Who needs gaps between so-called songs?  Anyway, I can not offer much advice though there is this handy page from Audacity that could help.

You can do it!

Converting into FLAC is really easy.  It’s probably the easiest step in the whole process.  (Splitting into individual songs is probably the least easy if you’re keeping track.)

##
# cd into the folder where your newly created wav files are located and run flac itself 

flac --best --channel-map=none -V *.wav 

# you only need --channel-map=none if your full-length file has more than two tracks  
##

Once that has finished, you can tag your music.  I use EasyTag because it’s easy.

Share

Password Chaos

Creating a memorable password runs against most of the rules implemented for creating a strong password.  Much fine work has been done parodying this interesting fact.  That being said, the folks who implement the rules and subsequently announce those rules to the users have lost their minds.  Take a look at this description of the required rules I happened upon recently.

Required Overlap
Required Overlap

Let’s pair some of these.

Length:

Between 8 and 64 characters

Increase the length from 12-20 characters

Case:

Use both uppercase and lowercase letters.

A lowercase or uppercase letter

Repetition:

Not repeat any character more than 3 times in a row.

Not be a sequence of 4 characters in a row.

In each of the above pairs, the first line is all that is required to articulate the apparent rule.  The second line can be dropped as superfluous (and confusing).

In the length pair, the “increase” line is essentially unparsable.  This is to say I can formulate no meaning for that line which aligns that line with the other lines in a logically consistent fashion.  If the minimum is 8 then there would never be a reason to increase by 12.  If you are 20 away from 64 (the maximum) there is no reason to increase the length.

In the case pair, the and and the or cannot be conjoined.  If you must use both (and) then you cannot use one or the other (or).

In the repetition pair, if you cannot have three in a row you necessarily can’t have four in a row.  Further, if the minimum is 8 it must be longer than 4 regardless.

Then there is the order of the list.  Makes me wonder if that order could have been arrayed in a way that would be more confusing.  Could it?

Never mind that five lines are sentences (ending in periods) and the other three lines are not (unpunctuated).

It’s like they took a poll of the IT staff and just listed out selections from their various responses.

Just… think it through a bit.  UI/UX isn’t something that requires a specialized developer.  Think.

2024 Update

I came across this lovely gem of password instructions and wanted to share it as well.

Password Policy
Password Policy

Why limit a password to 18 characters?  That’s just plain silly.  It’s not even a power of two.  “Hey, let’s arbitrarily limit passwords!”.  Idiots.

Why restrict spaces?  Again, arbitrary and silly.

Both of these rules limit passwords making them less secure rather than more.  Stop it.  Just don’t.  Fix your shit.

Share