awk - unix

Simple Examples

Extract and print a first column from the input

ps | awk '{print $1}'

Extract and print a multiple columns from the input with space between each column

ps | awk '{print $1, $3}'

Extract and print a multiple columns from the input without space

ps | awk '{print $1$3}'

Extract and print a multiple columns from the input with various formatting

ps | awk '{print $1">>"$2"\t"$3}'

Extract and print multiple on new lines

ps | awk '{print $1; print $2;}'

Extract and print a columns without newline at the end

ps | awk '{printf $1">>"$2">>"$3"**"}'

BEGIN block

This block is executed only once before processing first row in the input

ps | awk 'BEGIN { print "Header" } {print $1}'

ps | awk 'BEGIN { print "Header"; print "SubHeader"; } {print $1}'

END block

This block is executed only once before processing first row in the input

ps | awk '{print $1} END { print "Footer" }'

ps | awk '{print $1} END { print "SubFooter"; print "Footer"; }'

Both BEGIN and END blocks

ps | awk 'BEGIN { print "Header" } {print $1} END { print "Footer" }'

Special variables

$0 - Entire input record/row

$NF - Last field/column in the given row

NF - Total number of fields/columns in the given row

NR - Record/row number, starting with 1

Add total columns to the front in each row

ps | awk '{print NF,$0}'

Add row number to the front in each row

ps | awk '{print NR,$0}'

Print last column column in each row

ps -c | awk '{print $NF}'

Customise Field and Record Separator

FS - Input Field/Column Separator

OFS - Output Field/Column Separator

RS - Input Record/Row Separator

ORS - Output Record/Row Separator

Print second column from a csv file

cat input.csv | awk 'BEGIN{FS=","} {print $2}'

Convert ps output to csv format

ps -c | awk 'BEGIN{OFS=","} {print $1,$2}'

Convert a comma separated file with has | for separating each record

cat input.csv | awk 'BEGIN{FS=",";OFS=":";RS="|";ORS="**";}{print $1,$2}'

Conditions and Operators

Print the rows with third column (gender) equal to Male

cat input.csv | awk '{if ($3 == "Male") { print $0 }}'

Print the rows with third column (gender) not equal to Male

cat input.csv | awk '{if ($3 != "Male") { print $0 }}'

Print the rows with fourth column (age) greater than or equal to 40

cat input.csv | awk '{if ($4 >= 40) { print $0 }}'

Print the rows with fourth column (age) lesser than or equal to 60

cat input.csv | awk '{if ($4 <= 60) { print $0 }}'

Print the rows with fourth column (age) between 40 and 60

cat input.csv | awk '{if ($4 >= 40 && $4 <= 60) { print $0 }}'

Print the rows with fourth column (age) less than 18 or greater than 60

cat input.csv | awk '{if ($4 < 18 || $4 > 60) { print $0 }}'

if else

cat input.csv | awk '{if ($3 == "Male") { print $2 } else { print $3 }}'

if else if else

cat input.csv | awk '{if ($3 == "Male") { print $2 } else if ($4 >= 40) { print $2 } else { print $3 }}'

Regex

Displays row matching the given regex

ps | awk '{if ($0 ~ /docker/) { print $0 }}'

Displays row not matching the given regex

ps | awk '{if ($0 !~ /docker/) { print $0 }}'

Global scope variable - calculate total of all rows

cat data.csv | awk 'BEGIN{FS=",";count=0;} {count = count + $2} END { print "Sum: "count}'

Record scope variable - calculate total of eac row

cat data.csv | awk 'BEGIN{FS=","} {count=0;col=1; while (col <= NF) {count = count + $col; col = col + 1;}  print count }'