nixCraft MENU

How To Use Bash Parameter Substitution

Like A Pro
last updated December 1, 2018 in Linux, Shell scripting, UNIX

T he $ character is used for parameter expansion, arithmetic

expansion and command substitution. You can use it for
manipulating and expanding variables on demands without using
external commands such as perl, python, sed or awk. This guide shows
you how to use parameter expansion modi ers to transform Bash shell
variables for your scripting needs.

Syntax

You can use variables to store data and con guration options. For example:

dest="/backups"

Use echo or printf command to display variable value:

echo "$dest"

OR

printf "$dest\n"
The parameter name or symbol such as $dest to be expanded may be enclosed in braces

echo "Value ${dest}"

It is optional but serve to protect the variable to be expanded from characters

immediately following it which could be interpreted as part of the name.

1. Setting Up Default Shell Variables Value

The syntax is as follows:

${parameter:-defaultValue}
var=${parameter:-defaultValue}

If parameter not set, use defaultValue. In this example, your shell script
takes arguments supplied on the command line. You’d like to provide
default value so that the most common value can be used without
needing to type them every time. If variable $1 is not set or passed, use
root as default value for u:

u=${1:-root}

Consider the following example:

#!/bin/bash
_jail_dir="${1:-/home/phpcgi}"
echo "Setting php-cgi at ${_jail_dir}..."
# rest of the script ...

You can now run this script as follows:

./script.sh /jail # <--- set php jail at /jail dir

./script.sh /home/httpd/jail # <---- set php jail at /home/httpd/jail di
./script.sh # <--- set php jail dir at /home/phpcgi (de

Here is another handy example:

_mkdir(){
local d="$1" # get dir name
local p=${2:-0755} # get permission, set default to 0755
[ $# -eq 0 ] && { echo "$0: dirname"; return; }
[ ! -d "$d" ] && mkdir -m $p -p "$d"
}

Use this substitution for creating failsafe functions and providing missing command line
arguments in scripts.

#1.1: Setting Default Values

The syntax is as follows:

${var:=value}
var=${USER:=value}

The assignment (:=) operator is used to assign a value to the variable if it doesn’t
already have one. Try the following examples:

echo $USER

Sample outputs:

vivek

Now, assign a value foo to the $USER variable if doesn’t already have one:

echo ${USER:=foo}

Sample outputs:
 vivek

Unset value for $USER:

unset USER
echo ${USER:=foo}

Sample outputs:

foo

This make sure you always have a default reasonable value for your script.

Tip: ${var:-defaultValue} vs ${var:=defaultValue}

Please note that it will not work with positional parameter arguments:

var=${1:=defaultValue} ### FAIL with an error cannot assign in this way

var=${1:-defaultValue} ### Perfect

2. Display an Error Message If $VAR Not Passed

If the variable is not de ned or not passed, you can stop executing the Bash script with
the following syntax:

${varName?Error varName is not defined}

${varName:?Error varName is not defined or is empty}
${1:?"mkjail: Missing operand"}
MESSAGE="Usage: mkjail.sh domainname IPv4" ### define error me
_domain=${2?"Error: ${MESSAGE}"} ### you can use $MESSAGE too

This is used for giving an error message for unset parameters. In this example, if the $1
command line arg is not passed, stop executing the script with an error message:

_domain="${1:?Usage: mknginxconf domainName}"

Here is a sample script:

#!/bin/bash
# Purpose: Wrapper script to setup Nginx Load Balancer
# Author: Vivek Gite
_root="/nas.pro/prod/scripts/perl/nginx"
_setup="${_root}/createnginxconf.pl"
_db="${_root}/database/text/vips.db"
_domain="${1:?Usage: mknginxconf domainName}" ### die if domainName is

[ ! -f $_db ] && { echo "$0: Error $_db file not found."; exit 1; }
line=$(grep "^${_domain}" $_db) || { echo "$0: Error $_domain not found in

# Get domain config info into 4 fields:

# f1 - Domain Name|
# f2 - IPv4Vip:httpPort:HttpsPort, IPv6Vip:httpPort:HttpsPort|
# f3 - PrivateIP1:port1,PrivateIP2,port2,...PrivateIPN,portN|
# f4 - LB Type (true [round robin] OR false [session])
# ------------------------------------------------------------------------
IFS='|'
read -r f1 f2 f3 f4 <<<"$line"

# Do we want ssl host config too?

IFS=':'
set -- $f2
ssl="false"
[ "$3" == "443" ] && ssl="true"

# Build query
d="$f1:$ssl:$f4"
IFS=','
ips=$f3

# Call our master script to setup nginx reverse proxy / load balancer (LB)
$_setup "$d" $ips

2.1. Display an Error Message and Run Command

If $2 is not set display an error message for $2 parameter and run cp command on y as
follows:

#!/bin/bash
_file="$HOME/.input"
_message="Usage: chkfile commandname"

# Run another command (compact format)

_cmd="${2:? $_message $(cp $_file $HOME/.output)}"

$_cmd "$_file"

3. Find Variable Length

You can easily nd string length using the following syntax:

${#variableName}
echo ${#variableName}
len=${#var}

Here is a sample shell script to add a ftp user:

#!/bin/bash
# Usage : Add a ftp user
_fuser="$1"
_fpass="$2"

# die if username/password not provided

[ $# -ne 2 ] && { echo "Usage: addftpuser username password"; exit 1;}

# Get username length and make sure it always <= 8

[[ ${#_fuser} -ge 9 ]] && { echo "Error: Username should be maximum 8 char

# Check for existing user in /etc/passwd

/usr/bin/getent passwd "${_fuser}" &>/dev/null

# Check exit status

[ $? -eq 0 ] && { echo "Error: FTP username \"${_fuser}\" exists."; exit 3

# Add user
/sbin/useradd -s /sbin/nologin -m "${_fuser}"
echo "${_fpass}" | /usr/bin/passwd "${_fuser}" --stdin

Each Linux or UNIX command returns a status when it terminates normally or abnormally.
You can use command exit status in the shell script to display an error message or take
some sort of action. In above example, if getent command is successful, it returns a code
which tells the shell script to display an error message. 0 exit status means the command
was successful without any errors. $? holds the return value set by the previously
executed command.

4. Remove Pattern (Front of $VAR)

The syntax is as follows:

${var#Pattern}
${var##Pattern}

You can strip $var as per given pattern from front of $var. In this example remove /etc/
part and get a lename only, enter:

f="/etc/resolv.conf"
echo ${f#/etc/}

The rst syntax removes shortest part of pattern and the second syntax removes the
longest part of the pattern. Consider the following example:

_version="20090128"
_url="http://dns.measurement-factory.com/tools/dnstop/src/dnstop-${_versio

You just want to get lename i.e. dnstop-20090128.tar.gz, enter (try to remove shortest
part of $_url) :

echo "${_url#*/}"
Sample outputs:

/dns.measurement-factory.com/tools/dnstop/src/dnstop-20090128.tar.gz

Now try using the longest part of the pattern syntax:

echo "${_url##*/}"

Sample outputs:

dnstop-20090128.tar.gz

This is also useful to get a script name without using /bin/basename:

#!/bin/bash
_self="${0##*/}"
echo "$_self is called"

Create a script called master.info as follows:

#!/bin/bash
# Purpose: Display jail info as per softlink
# Author: Vivek Gite
_j="$@"

# find out script name

_self="${0##*/}"

[ "$VERBOSE" == "1" ] && echo "Called as $_self for \"$_j\" domain(s)"

for j in $_j
do
export _DOMAIN_NAME=$j
source functions.sh
init_jail
# call appropriate functions as per script-name / softlink
case $_self in
uploaddir.info) echo "Upload dir for $j: $(get_domain_uplo
tmpdir.info) echo "/tmp dir for $j: $(get_domain_tmp_dir)"
mem.info) echo "$j domain mem usage (php+lighttpd): $(get_
cpu.info) echo "$j domain cpu usage (php+lighttpd): $(get_
user.info) echo "$j domain user and group info: $(get_doma
diskquota.info) echo "$j domain disk quota info (mysql+dis
*) warn "Usage: $_self"
esac
done

Finally, create softlink as follows:

 # ln -s master.info uploaddir.info
# ln -s master.info tmpdir.info
# ln -s master.info mem.info
....
..

You can now call script as follows:

# ./mem.info example.org
# ./cpu.info example.com example.net

#4.1: Remove Pattern (Back of $VAR)

The syntax is as follows:

${var%pattern}
${var%%pattern}

Exactly the same as above, except that it applies to the back of $var. In this example
remove .tar.gz from $FILE, enter:

FILE="xcache-1.3.0.tar.gz"
echo ${FILE%.tar.gz}

Sample outputs:

xcache-1.3.0

Rename all *.perl les to *.pl using bash for loop as Apache web server is con gured to
only use .pl le and not .perl le names:

for p in /scripts/projects/.devl/perl/*.perl
do
mv "$p" "${p%.perl}.pl"
done

You can combine all of them as follows to create a build scripts:

#!/bin/bash
# Usage: Build suhosin module for RHEL based servers
# Author: Vivek Gite
# ----
# Set default value for $2
VERSION="-${2:-0.9.31}"
URL="http://download.suhosin.org/suhosin${VERSION}.tgz"
vURL="http://download.suhosin.org/suhosin${VERSION}.tgz.sig"

# Get tar ball names

FILE="${URL##*/}"
vFILE="${vURL##*/}"
DLHOME="/opt"
SOFTWARE="suhosin"

# Remove .tgz and get dir name

DEST="${FILE%.tgz}"

# Download software
wget $URL -O "${DLHOME}/$FILE"
wget $vURL -O "${DLHOME}/$vFILE"

# Extract it
tar -zxvf $FILE
cd "$DEST"

# Build it and install it

phpize --clean && phpize && ./configure && make && read -p "Update/Install
shopt -s nocasematch
[[ $answer =~ y|es ]] && make install
shopt -u nocasematch

If you turn on nocasematch option, shell matches patterns in a case-insensitive fashion

when performing matching while executing case or [[ conditional expression.

5. Find And Replace

The syntax is as follows:

${varName/Pattern/Replacement}
${varName/word1/word2}
${os/Unix/Linux}

Find word unix and replace with linux, enter:

x="Use unix or die"

sed 's/unix/linux/' <<<$x

You can avoid using sed as follows:

echo "${x/unix/linux}"
out="${x/unix/linux}"
echo "${out}"

To replace all matches of pattern, enter :

out="${x//unix/linux}"

You can use this to rename or remove les on y

y=/etc/resolv.conf
cp "${y}" "${y/.conf/.conf.bak}"

Here is another example:

# RHEL php modules path

_php_modules="/usr/lib64/php/modules"
for i in $_php_modules/*
do
p="${i##*/}" ## Get module name
ini="/etc/php.d/${p/so/ini}" ## Get ini file by replacing
# make sure file exists
[ ! -f "$ini" ] && echo "$i php module exists but $ini fil
done

The following function installs required modules in chrooted php-cgi process

install_php_modules(){
# get jail name
local n="${_chrootbase}/${d##/}"
local p=""
local ini=""
# enable only ${_php_modules_enabled} php modules and delete other
for i in $_php_modules/*
do
p="${i##*/}"
ini="$n/etc/php.d/${p/so/ini}"
# find out if module is enabled or not
if [[ ${_php_modules_enabled} = *${p}* ]]
then
[ "$VERBOSE" == "1" ] && echo " [+] Enabling php m
$_cp -f "$i" "$n/${_php_modules##/}" ## instal
copy_shared_libs "$i" ## get s
else
[ -f "${ini}" ] && $_rm -f "${ini}" ## if ol
fi
done
}

6. Substring Starting Character

The syntax is as follows:

${parameter:offset}
${parameter:offset:length}
${variable:position}
var=${string:position}

Expands to up to length characters of parameter starting at the character speci ed by

offset.

base="/backup/nas"
file="/data.tar.gz"
#### strip extra slash from $file ####
path="${base}/${file:1}"
Extract craft word only:

x="nixcraft.com"
echo ${x:3:5}"

To extract phone number, enter:

phone="022-124567887"
# strip std code
echo "${phone:4}"

7. Get list of matching variable names

Want to get the names of variables whose names begin with pre x? Try:

VECH="Bus"
VECH1="Car"
VECH2="Train"
echo "${!VECH*}"

8. Convert to upper to lower case or vice versa

Use the following syntax to convert lowercase characters to uppercase:

name="vivek"
# Turn vivek to Vivek (only first character to uppercase)
echo "${name^}"
# Turn vivek to VIVEK (uppercase)
echo "${name^^}"
echo "Hi, $USERNAME"
echo "Hi, ${USERNAME^}"
echo "Hi, ${USERNAME^^}"
# Convert everything to lowercase
dest="/HOME/Vivek/DaTA"
echo "Actual path: ${dest,,}"
# Convert only first character to lowercase
src="HOME"
echo "${src,}"

Only convert rst character in $dest if it is a capital ‘H’:

dest="Home"
echo "${dest,H}"
dest="Fhome"
echo "${dest,H}"

See “Shell Scripting: Convert Uppercase to Lowercase” for more info.

Summary: String Manipulation and Expanding Variables

For your ready references here are all your handy bash parameter substitution operators.
Try them all; enhance your scripting skills like a pro:

${parameter:-defaultValue} Get default shell variables value

${parameter:=defaultValue} Set default shell variables value

Display an error message if parameter is

${parameter:?"Error Message"}
not set

${#var} Find the length of the string

${var%pattern} Remove from shortest rear (end) pattern

${var%%pattern} Remove from longest rear (end) pattern

${var:num1:num2} Substring

${var#pattern} Remove from shortest front pattern

${var##pattern} Remove from longest front pattern

Find and replace (only replace rst

${var/pattern/string}
occurrence)

${var//pattern/string} Find and replace all occurrences

Expands to the names of variables whose

${!prefix*}
names begin with pre x.

${var,}
Convert rst character to lowercase.
${var,pattern}

${var,,}
Convert all characters to lowercase.
${var,,pattern}

${var^}
Convert rst character to uppercase.
${var^pattern}

${var^^}
Convert all character to uppercase..
${var^^pattern}

References:

Linux shell scripting wiki

UNIX / Linux bash build script examples
oo-style string library for bash 4
Bash man page

SHARE ON Facebook Twitter

Moving long distance?

Get a Free Quote from U-
Pack Now.
Ad U-Pack Moving

Learn more
 Posted by: Vivek Gite
The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for
the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin,
Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

Your support makes a big difference:

I have a small favor to ask. More people are reading the nixCraft. Many of you
block advertising which is your right, and advertising revenues are not
suf cient to cover my operating costs. So you can see why I need to ask for
your help. The nixCraft takes a lot of my time and hard work to produce. If
everyone who reads nixCraft, who likes it, helps fund it, my future would be
more secure. You can donate as little as $1 to support nixCraft:

Become a Supporter Make a contribution via Paypal/Bitcoin

Start the discussion at www.nixcraft.com

Historical Comment Archive

31 comment

Jeffrey October 14, 2010 at 3:23 am

I hate to go offtopic, but do you know if these apply to tcsh as well?

Thanks.

nixCraft October 14, 2010 at 4:54 pm

 I’ve not used tcsh / csh much; but should work with ksh.
HTH

Chris F.A. Johnson October 6, 2011 at 2:04 am

These are POSIX shell expansions and bash/ksh93 extensions. They do not apply to
tcsh (which should not be used for scripting).

Lacrymology October 14, 2010 at 4:22 pm

I hate to go offtopic as well, but I love the Internet Explorer 9 ad on the right….. wtf?

nixCraft October 14, 2010 at 4:54 pm

Ads comes from various sources. This is the only way to cover the cost of hosting,
CDN and bandwidth. Hope this helps!

TMW October 14, 2010 at 4:36 pm

So you got post back but deleted all comments? How come?

nixCraft October 14, 2010 at 4:39 pm

Backups are done ones a day. This post was restored from my local open of ce org
cache le. There is not much I can do to restore all 12-13 comments including yours.

jackd October 15, 2010 at 2:07 am

What a great tutorial and reference. I can never remember the syntax when I need it. Will
de nitely be bookmarking this one.

Lucas Vieites October 15, 2010 at 7:29 am

Great tips! There are some functionalities I didn’t even know existed.
thiagoc October 18, 2010 at 2:11 am

cp “${y}” “${y/.conf/.conf.bak}”

a more simple approach:

cp “${y}”{,.bak}

Chris F.A. Johnson October 6, 2011 at 2:07 am

Or, more portably and ef ciently:

cp "$y" "$y.bak"

maedox December 6, 2011 at 11:01 am

cp ${y,.bak}

Chris F.A. Johnson November 8, 2013 at 2:36 pm

In bash, a comma in parameter expansion converts to lower case:

$ y=HELLO
$ echo "${y,}"
hELLO
$ echo "${y,,}"
hello

maedox November 8, 2013 at 2:48 pm

You’re right. Not sure where my brain was that day. One would I assume I
tested it before commenting, but it doesn’t seem to do anything. No error
or warning either. That’s bash for you. 😛

Chris F.A. Johnson November 8, 2013 at 3:19 pm

 There’s no reason for any error message or warning; it’s a perfectly
legal, if nonsensical, expansion.

Matt Thiessen October 22, 2010 at 12:47 pm

Man, did I need this today. Thanks!

gamerh2o December 4, 2010 at 8:12 am

very useful,like this site very much

toto February 16, 2011 at 5:59 am

good tutorial,

thanks

unixgeek February 23, 2013 at 1:52 am

Thanks for this excellent HowTo! One minor correction: sed ‘s/unix/linux’ <<<$x should be
sed 's/unix/linux/g' <<<$x

Otherwise you get an unterminated `s' command error.

nixCraft February 23, 2013 at 6:16 pm

Thanks for the heads up!

Tom June 24, 2013 at 6:30 pm

on the bash-shell-parameter-substitution-2.html page, _mkdir() example, fourth line you

write:
[ $# -eq 0 ] &shouldn’t this be
[ $# -eq 0 ] &## [0—] is four digit octal permissions

or equivalent to allow user to set permissions other than the default 0755 value?
Tom June 24, 2013 at 6:35 pm

prior message was strangely truncated in places. Resubmit:

on the bash-shell-parameter-substitution-2.html page, _mkdir() example, fourth line you

write:

[ $# -eq 0 ] &shouldn’t this be:

[ $# -eq 0 ] &## [0—] is four digit octal permissions

to inform user to set permissions other than the default 0755 value if desired?

Tom June 24, 2013 at 6:38 pm

Last attempt: on the bash-shell-parameter-substitution-2.html page, _mkdir() example,

4th line:
[ $# -eq 0 ] &shouldn’t this be:
[ $# -eq 0 ] &## [0—] is octal permissions
to inform user to set other than the default 0755 value if desired?

srinivasu November 8, 2013 at 5:25 am

Nice! learnt something from this tutorial..

Zigmund November 12, 2013 at 9:55 pm

Something interesting to note on the substring syntax is that you can address the offset
backwards as well. I am not sure about how portable this syntax is, but I have found it
very useful:

echo “${VAR: -4}” # Prints the last 4 characters, note the space between : and – in this
example.

echo “${VAR:1:-1}” # The space is optional on the second offset, but not the rst

khoan September 20, 2014 at 3:45 pm

${bam:=value}
-bash: value: command not found
 maedox September 25, 2014 at 11:07 am

@khoan, yes, what did you expect? There is no command named ‘value’. Put echo in
front and it should output ‘value’ unless $bam is already set.

wd40 January 25, 2015 at 7:44 pm

Just what i wanted to learn. Thanks!

Ikem February 3, 2015 at 4:50 pm

How much does that apply to “dash”?

Chris F.A. Johnson February 7, 2015 at 7:30 pm

Ikem, all the POSIX expansions (#1..#4) will work in dash (but not the shopt command).

Alhadis February 13, 2015 at 2:12 pm

… aaaaand BOOKMARKED. *punches Command+D* Thanks for the helpful reference! 😀

    Have a question? Post it on our forum!

Previ ous post:

Download Ubuntu 10.10 (Maverick Meerkat) CD ISO / DVD Images

N ext post:

Download Fedora 14 CD / DVD ISO

Tagged as: assignment operator, command line arguments, command substitution, default shell, parameter
expansion, shell variables
@2000-2019 nixCraft. All rights reserved.

PRIVACY

TERM OF SERVICE

CONTACT/EMAIL

DONATIONS

SEARCH