What if the text has tabs, punctuation or capital letters?

Q: What if the text has tabs, punctuation or capital letters?

Normalise first: tr -cs 'A-Za-z' '\n' turns every run of non-letters into a newline, and tr 'A-Z' 'a-z' lowercases. Without this, Deploy, deploy, and deploy are counted as three different words.

Bash and Linux

Word frequency count

mediumsort, uniq, wc and pipes

Given a text file of lowercase words separated by one or more spaces, print each word and how many times it appears, most frequent first, in the format word count. Assume no two words have the same count.

words.txt

TEXT

deploy api deploy web deploy web
api rollback api deploy

Examples

Example 1

Input: Run the command on `words.txt`.

Output: deploy 4 api 3 web 2 rollback 1

Hints

Approach

tr -s ' ' '\n' translates every space into a newline, and -s squeezes runs of them into one, so each word lands on its own line with no empty lines in between. sort | uniq -c counts each word. sort -rn orders by count, highest first. uniq -c prints count word, so the final awk '{print $2, $1}' swaps the two columns into the required word count format and also drops the padding.

Bash

tr -s ' ' '\n' < words.txt | sort | uniq -c | sort -rn | awk '{print $2, $1}'

Follow-up questions

Print only the top 10 words.
Ignore common stop words such as the and a using grep -vwFf stopwords.txt.

Frequently asked questions

Normalise first: tr -cs 'A-Za-z' '\n' turns every run of non-letters into a newline, and tr 'A-Z' 'a-z' lowercases. Without this, Deploy, deploy, and deploy are counted as three different words.