What is shuf?
shuf is a command-line utility like sort included in Coreutils. You may have guessed that it is used to pseudo randomize a given input in the same way you would shuffle a deck of cards. You guessed right!
Here we will cover the shuf command along with alternatives just in case you find yourself stuck without a shuf.
Where to find help on shuf
Once you know what shuf is, the next step is knowing how to use it. Like most command-line utilities, shuf comes with an –help long option.
Command
Usage: shuf [OPTION]… [FILE]
or: shuf -e [OPTION]… [ARG]…
or: shuf -i LO-HI [OPTION]…
Write a random permutation of the input lines to standard output.
With no FILE, or when FILE is -, read standard input.
Mandatory arguments to long options are mandatory for short options too.
-e, –echo treat each ARG as an input line
-i, –input-range=LO-HI treat each number LO through HI as an input line
-n, –head-count=COUNT output at most COUNT lines
-o, –output=FILE write result to FILE instead of standard output
–random-source=FILE get random bytes from FILE
-r, –repeat output lines can be repeated
-z, –zero-terminated line delimiter is NUL, not newline
–help display this help and exit
–version output version information and exit
The Three Ways of shuf
There are three ways to use the shuf command which are:
- file shuf
- list shuf
- range shuf
Each way has its advantages. Knowledge of all the ways beforehand may reduce the need to use other external commands in conjunction with the shuf command.
file shuf
File shuf is the most common way shuf is used in command line. When the -e or -i option are not included in the options, shuf will operate as file shuf. That is, the input to be shuffled will be a file whether standard input or any given file. The last word in the parameter list may be a filename. In case this parameter is omitted file is taken to be standard input from the shell or piped. A – may be included following the convention that standard input is inferred.
Here follow usage and ways to specify the file in file shuf.
Usage
Ways to specify a file
There is more than one way to specify the file using file shuf. Here are example commands for each way.
Implicit file as standard input
In this way we omit file from the arguments of the shuf command. Following the convention, you may opt to include a – in place of file to indicate the file to be taken as standard input.
Commands
seq 3 | shuf
}
Output
3
2
Explicit file as standard input
Commands
seq 3 | shuf –
}
Output
1
2
Notes
(1) Adding a – at the end of shuf serves as a visual cue improving readability in bash scripts.
File as the name of file
In this way, we specify a filename as file in the arguments of the shuf command. Here follow a few file shuf examples using files.
Shuffle input lines from the terminal
Commands
shuf /dev/fd/1
}
asdf
sdf
df
f
Ctrl-D
Output
f
asdf
sdf
Notes
(1) The above shuf command shuf /dev/fd/1 is equivalent to shuf –
(2) Termination of input lines through Ctrl-D is required
Shuffle lines in file
Commands
seq 3 > file;
shuf file;
rm -f file
}
Output
1
3
list shuf
In the last way to shuf, we operated on a file or input piped into the shuf command. In this way to shuf, we allow input lines to be specified as arguments of the shuf command using the -e option, forcing shuf to operate as list shuf.
Usage
Ways to specify list args
Type input as args
Commands
shuf -e 1 2 3
}
Output
3
2
Notes
(1) The above shuf command shuf -e 1 2 3 is equivalent to seq 3 | shuf –
Variable as args
Commands
var="1 2 3";
shuf -e ${var}
}
<strong>Output</strong>
[cc lang="bash"]
3
1
2
Parameter expansion as args
Commands
shuf -e {1..3}
}
Output
2
3
Command substitution as args
Commands
shuf -e $( seq 3 )
}
Output
2
1
range shuf
This last way to shuf is unlike the previous ways introduced. Instead of specifying a file or args in the command line, it requires a range of integers. The -i option, forces shuf to operate as range shuf.
Range shuf produces a range of integers in random order.
Usage
Ways to specify range
The one way: LO-HI
Commands
shuf -i 1–3
}
Output
3
1
Notes
(1) The shuf command shuf -i 1-3 is equivalent to all previous command using the sequence 1 2 3
Advanced shuf options
Here are some of the advanced options for shuf that may prove useful in bash programming.
Limit number of output lines
To limit the number of lines in the output, we use the -n option followed by an integer as follows.
Commands
shuf -i 1–3 -n 1
}
Output
Notes
- The shuf command shuf -i 1-3 -n 1 is equivalent to shuf -i 1-3 | head -1
Specify a file to write output lines
To specify a file to write output lines, we use the -o option followed by a filename as follows.
Commands
shuf -i 1–3 -n 1 -o file;
cat file;
rm -f file
}
Output
Notes
(1) The shuf command shuf -i 1-3 -n 1 -o file is equivalent to shuf -i 1-3 -n 1 > file using I/O redirection
Stream output lines
To create a continuous stream of output lines, we use the -r option as follows.
Commands
shuf -e {0,1} -r | xargs -i echo -n "{}"
}
Output
Use the zero byte instead of newline as line delimiter
To use zero-terminated lines, we use the -z option as follows.
Commands
seq 3 | tr ‘n’ ” | shuf -z
}
Output
Notes
(1) The output contains non-printing zero byte between digits
How to shuf in bash the easy way
The easy way to shuf is to use the shuf command as discussed above. However, if you are a little curious about how you would shuf otherwise using other external commands or even pure bash, read on.
How to shuf the hard way
I have compiled a list of hard ways to shuf. Don’t worry they are not that hard. They just don’t make use of the shuf command.
Using sort
A common alternative to using file shuf is to use the sort command. Here’s how:
Commands
seq 3 | sort -r
}
Output
3
1
Notes
(1) The shuf seq 3 | sort -r is equivalent to shuf -i 1-3
(2) shuf -i 1-3 is faster
Using gawk
Another alternative to using file shuf is to use the gawk command. Here’s how:
Script
gawk -v random=${RANDOM} ‘
function randInt() {
return int(rand()*1000)
}
function case_numeric_compare(i1, v1, i2, v2, l, r) {
l = int(v1)
r = int(v2)
if(l<r) return -1
else if(l==r) return 0
else return 1
}
BEGIN {
count=1
srand(random)
}
{
rank[count]=randInt()
line[count]=$(0)
count++
}
END {
asorti(rank,order,"case_numeric_compare")
for(i=0;i<count;i++) {
print line[order[i]]
}
}
‘ –
}
if [ ${#} -eq 0 ]
then
true
else
exit 1 # wrong args
fi
gawk-shuf
Source: gawk-shuf.sh
Commands
seq 3 | bash gawk-shuf.sh
}
Output
3
1
Notes
(1) You may also use awk
(2) gawk-shuf.sh must exist in the working directory of commands
Pure bash
Script
local line
local -a lines
while read -r line
do
lines[RANDOM]=${line}
done
for line in ${lines[@]}
do
echo ${line}
done
}
if [ ${#} -eq 0 ]
then
true
else
exit 1 # wrong args
fi
pure-bash-shuf
Source: pure-bash-shuf.sh
Commands
seq 3 | bash pure-bash-shuf.sh
}
Output
3
1
Notes
- The above script does not handle the case in which RANDOM happens to occur more than once. That is left as an exercise.
dJackblck17 The Game
I have put together a card game start called Jackblck17 based on blackjack for the purpose of showing the shuf command in action.
Script
echo {A,2,3,4,5,6,7,8,9,10,J,Q,K}–{club,diamond,heart,spade}
}
banner() {
cat << EOF
bbbbbbbb
jjjj b::::::b lllllll kkkkkkkk 1111111 77777777777777777777
j::::j b::::::b l:::::l k::::::k 1::::::1 7::::::::::::::::::7
jjjj b::::::b l:::::l k::::::k 1:::::::1 7::::::::::::::::::7
b:::::b l:::::l k::::::k 111:::::1 777777777777:::::::7
jjjjjjj aaaaaaaaaaaaa ccccccccccccccccb:::::bbbbbbbbb l::::l
cccccccccccccccc k:::::k kkkkkkk 1::::1 7::::::7
j:::::j a::::::::::::a cc:::::::::::::::cb::::::::::::::bb
l::::l cc:::::::::::::::c k:::::k k:::::k 1::::1 7::::::7
j::::j aaaaaaaaa:::::a c:::::::::::::::::cb::::::::::::::::b l::::l
c:::::::::::::::::c k:::::k k:::::k 1::::1 7::::::7
j::::j a::::ac:::::::cccccc:::::cb:::::bbbbb:::::::b l::::l
c:::::::cccccc:::::c k:::::k k:::::k 1::::l 7::::::7
j::::j aaaaaaa:::::ac::::::c cccccccb:::::b b::::::b l::::l c::::::c
ccccccc k::::::k:::::k 1::::l 7::::::7
j::::j aa::::::::::::ac:::::c b:::::b b:::::b l::::l c:::::c k:::::::::::k
1::::l 7::::::7
j::::j a::::aaaa::::::ac:::::c b:::::b b:::::b l::::l c:::::c
k:::::::::::k 1::::l 7::::::7
j::::ja::::a a:::::ac::::::c cccccccb:::::b b:::::b l::::l c::::::c
ccccccc k::::::k:::::k 1::::l 7::::::7
j::::ja::::a a:::::ac:::::::cccccc:::::cb:::::bbbbbb::::::bl::::::lc:::::::
cccccc:::::ck::::::k k:::::k 111::::::111 7::::::7
j::::ja:::::aaaa::::::a c:::::::::::::::::cb::::::::::::::::b l::::::l
c:::::::::::::::::ck::::::k k:::::k 1::::::::::1 7::::::7
j::::j a::::::::::aa:::a cc:::::::::::::::cb:::::::::::::::b
l::::::l cc:::::::::::::::ck::::::k k:::::k 1::::::::::1 7::::::7
j::::j aaaaaaaaaa aaaa ccccccccccccccccbbbbbbbbbbbbbbbb
llllllll cccccccccccccccckkkkkkkk kkkkkkk11111111111177777777
j::::j
jjjj j::::j
j::::jj j:::::j
j::::::jjj::::::j
jj::::::::::::j
jjj::::::jjj
jjjjjj
EOF
}
score() {
case ${1} in
A) echo 0 ;;
[2–9]|10) echo ${1} ;;
J|Q|K) echo 10 ;;
esac
}
score-hand() {
local card
local points
local -i candidate_points
local -i aces
aces=0
for card in ${hand}
do
let points+=$( score ${card/-*/} )
test ! "${card/-*/}" = "A" || {
let aces+=1
}
done
test ! ${aces} -le 0 || {
echo ${points}
}
while [ ${aces} -gt 0 ]
do
for point in ${points}
do
new_point=""
for ace in 1 11
do
candidate_points=$(( point + ace ))
test ! ${candidate_points} -le 21 || {
echo "${candidate_points}"
new_points="${new_points} ${candidate_points}"
}
done
done
let aces-=1
points="${new_points}"
done | sort -nur | head -1
}
jackblck17() {
local deck
local card
local -i turn
local hand
deck=$( shuf -e $( deck ) )
banner
echo -e " Press enter key to continue"
read
turn=1
for card in ${deck}
do
test ! ${turn} -gt 2 || {
echo -e "nhit or stay? (h) or s "
read
test ! "${REPLY}" = "s" || {
break
}
}
echo -e "n Your hand: n"
hand="${hand} ${card}"
echo " ${hand}"
test ! $( score-hand ) -gt 21 || {
echo -e "nBust!n"
exit
}
let turn++
sleep 1
done
echo -e "Dealer’s hand: 17n"
echo -e "Your hand: $( score-hand )n"
test ! $( score-hand ) -gt 17 && {
echo -e "Dealer winsn"
true
} || {
echo -e "You win!n"
}
}
if [ ${#} -eq 0 ]
then
true
else
exit 1 # wrong args
fi
jackblck17
Source: jackblck17.sh
Commands
Output
jjjj b::::::b lllllll kkkkkkkk 1111111 77777777777777777777
j::::j b::::::b l:::::l k::::::k 1::::::1 7::::::::::::::::::7
jjjj b::::::b l:::::l k::::::k 1:::::::1 7::::::::::::::::::7
b:::::b l:::::l k::::::k 111:::::1 777777777777:::::::7
jjjjjjj aaaaaaaaaaaaa ccccccccccccccccb:::::bbbbbbbbb l::::l
cccccccccccccccc k:::::k kkkkkkk 1::::1 7::::::7
j:::::j a::::::::::::a cc:::::::::::::::cb::::::::::::::bb l::::l
cc:::::::::::::::c k:::::k k:::::k 1::::1 7::::::7
j::::j aaaaaaaaa:::::a c:::::::::::::::::cb::::::::::::::::b l::::l
c:::::::::::::::::c k:::::k k:::::k 1::::1 7::::::7
j::::j a::::ac:::::::cccccc:::::cb:::::bbbbb:::::::b l::::l
c:::::::cccccc:::::c k:::::k k:::::k 1::::l 7::::::7
j::::j aaaaaaa:::::ac::::::c cccccccb:::::b b::::::b l::::l
c::::::c ccccccc k::::::k:::::k 1::::l 7::::::7
j::::j aa::::::::::::ac:::::c b:::::b b:::::b l::::l
c:::::c k:::::::::::k 1::::l 7::::::7
j::::j a::::aaaa::::::ac:::::c b:::::b b:::::b l::::l
c:::::c k:::::::::::k 1::::l 7::::::7
j::::ja::::a a:::::ac::::::c cccccccb:::::b b:::::b l::::l
c::::::c ccccccc k::::::k:::::k 1::::l 7::::::7
j::::ja::::a a:::::ac:::::::cccccc:::::cb:::::bbbbbb::::::bl::::::lc
:::::::cccccc:::::ck::::::k k:::::k 111::::::111 7::::::7
j::::ja:::::aaaa::::::a c:::::::::::::::::cb::::::::::::::::b l::::::l
c:::::::::::::::::ck::::::k k:::::k 1::::::::::1 7::::::7
j::::j a::::::::::aa:::a cc:::::::::::::::cb:::::::::::::::b l::::::l
cc:::::::::::::::ck::::::k k:::::k 1::::::::::1 7::::::7
j::::j aaaaaaaaaa aaaa ccccccccccccccccbbbbbbbbbbbbbbbb
llllllll cccccccccccccccckkkkkkkk kkkkkkk11111111111177777777
j::::j
jjjj j::::j
j::::jj j:::::j
j::::::jjj::::::j
jj::::::::::::j
jjj::::::jjj
jjjjjj
Your hand:
3-heart
Your hand:</span>
3-heart 4-spade
hit or stay? (h) or (s)
Your hand:
3-heart 4-spade 9-heart
hit or stay? (h) or s
s
Dealer’s hand: 17
Your hand: 16
Dealer wins
The bottom line on shuf in bash
In this tutorial, we covered all you need to know about the shuf command and more. No longer will you have to result to do things the hard way, now that you have shuf.
To be honest, before writing this, I knew little that there was a command called shuf that could be used to randomize the order of any given input. After taking a deep dive into the shuf command for bash programming, Now, I can honestly say that it was worth it; shuf is more useful than I thought.
I hope that you enjoyed reading this as much as I enjoyed writing it and that it helps you in your career or homework. If it does, let me know.
Thanks,
