DevOps NetworkDevOpsNetwork
Join Free
Bash and Linux

Mask Secrets in Logs

mediumsed and config editing

Problem statement

Hide passwords, tokens and card numbers in a log before sharing it, using sed -E with capture groups so the key names and the last digits stay readable. Logs get pasted into tickets, chats and vendor support cases, and a leaked password in a log is a real incident.

app.log

TEXT
2026-10-01 10:00:01 login user=asha password=hunter2 ok
2026-10-01 10:00:02 api call token=abc123XYZ path=/orders
2026-10-01 10:00:03 payment card=4111111111111111 amount=20
2026-10-01 10:00:04 json {"user":"ravi","password":"s3cr3t!"}
2026-10-01 10:00:05 retry PASSWORD=Admin1
2026-10-01 10:00:06 health ok
  1. Show what goes wrong with a greedy pattern password=.* on the first line.
  2. Mask the log: replace the values of password and token (any case) with ****, mask the JSON password, and keep only the last 4 digits of the card.

Expected output:

TEXT
== greedy .* eats the rest of the line ==
2026-10-01 10:00:01 login user=asha password=****
== masked log ==
2026-10-01 10:00:01 login user=asha password=**** ok
2026-10-01 10:00:02 api call token=**** path=/orders
2026-10-01 10:00:03 payment card=************1111 amount=20
2026-10-01 10:00:04 json {"user":"ravi","password":"****"}
2026-10-01 10:00:05 retry PASSWORD=****
2026-10-01 10:00:06 health ok

Hints

Hint 1: [^ ]+ means "one or more characters that are not a space", which stops at the end of the value. .* runs to the end of the line.

Approach

Optimal: sed -E with capture groups

Covers: sed -E, capture groups ( ) and \1, alternation |, [^ ]+ vs .*, {n} counts, the g and I flags, several -e commands.

Keep the label, hide the value. A good mask keeps everything that helps debugging (the key name, the user, the last card digits) and hides only the secret. That means the pattern must find the exact value and nothing more.

%%{init: {"flowchart": {"padding": 18, "nodeSpacing": 30, "rankSpacing": 40, "htmlLabels": true}, "themeVariables": {"fontSize": "18px"}}}%% flowchart TB IN["card=4111111111111111"]:::red --> P(["match card= then 12 digits
capture the last 4"]):::purple P --> OUT["card=************1111"]:::green IN2["password=hunter2 ok"]:::red --> P2(["[^ ]+ stops at the space"]):::purple P2 --> OUT2["password=**** ok"]:::green classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px

Greedy .* eats too much. .* matches as many characters as it can. s/password=.*/password=****/ replaces hunter2 ok, so the ok you needed for debugging is gone too. [^ ]+ means "one or more characters that are not a space", so it stops at the end of the value. For JSON, [^"]* stops at the closing quote in the same way.

Capture groups put parts back. With -E (extended patterns), anything inside ( ) is remembered. In the replacement, \1 is the first group, \2 the second.

Command Keeps Hides
s/(password|token)=[^ ]+/\1=****/Ig the key name, in its original case the value
s/("password":")[^"]*"/\1****"/g "password":" the JSON value
s/card=[0-9]{12}([0-9]{4})/card=************\1/g the last 4 digits the first 12

(password|token) matches either word. [0-9]{12} means exactly 12 digits, and ([0-9]{4}) captures the last 4. g masks every match on a line, not just the first. I makes the match ignore case, so PASSWORD=Admin1 is caught too; \1 puts back PASSWORD exactly as it was.

Walking through the code. The # Setup: lines only create the sample log, so skip past them.

  1. The greedy version on line 1 shows password=**** with ok gone.
  2. The real mask runs three -e commands, one per kind of secret, in one pass over the file. The health ok line has nothing to mask and is printed unchanged.

Edge cases. Values with spaces, like password="my secret", need a pattern for quoted values. A key spelled differently (passwd, api_key, secret) is not caught until you add it to the list. The I flag is GNU only; on macOS, write the case variants out, like [Pp][Aa][Ss][Ss][Ww][Oo][Rr][Dd].

# Setup: create the sample log in a fresh temporary folder
cd "$(mktemp -d)"
cat > app.log << 'LOG'
2026-10-01 10:00:01 login user=asha password=hunter2 ok
2026-10-01 10:00:02 api call token=abc123XYZ path=/orders
2026-10-01 10:00:03 payment card=4111111111111111 amount=20
2026-10-01 10:00:04 json {"user":"ravi","password":"s3cr3t!"}
2026-10-01 10:00:05 retry PASSWORD=Admin1
2026-10-01 10:00:06 health ok
LOG

echo "== greedy .* eats the rest of the line =="
sed -E 's/password=.*/password=****/' app.log | head -n 1

echo "== masked log =="
sed -E \
  -e 's/(password|token)=[^ ]+/\1=****/Ig' \
  -e 's/("password":")[^"]*"/\1****"/g' \
  -e 's/card=[0-9]{12}([0-9]{4})/card=************\1/g' \
  app.log

Interview follow-ups

  • Mark every line that was changed with a comment underneath.

    GNU sed can append text after a line with a\, but it should only run when a substitution happened. The t command jumps when the last s changed something: sed -E -e 's/(password|token)=[^ ]+/\1=****/Ig' -e 't masked' -e 'b' -e ':masked' -e 'a # masked'. Here t masked jumps to the label for changed lines, b ends the cycle for the others, and a adds the comment line. It is a good example of sed being a tiny programming language, though awk is often clearer for this.

Frequently asked questions

It is a useful last line of defence, for example before attaching a log to a ticket. The better fix is not to log secrets at all: apps should never print passwords or tokens, and logging libraries can redact known fields before writing. Logs are often shipped to other systems within seconds, so a secret written once may already be copied elsewhere. Use sed masking for sharing, and fix the app so the secret is never written.

In basic mode (no -E), ( ), { }, +, ? and | are plain characters, and you must write \( \), \{12\} and so on to make them special. With -E they are special as written, which is much easier to read. -E works in both GNU and BSD sed. Old scripts often use -r, which is the GNU-only name for the same thing.

Usually because the group was written in basic mode, so ( was a plain character and there was no group to refer to. Add -E, or escape the brackets as \( \). Another cause is double quotes in the shell: inside "...", the shell may change backslashes, so prefer single quotes around sed commands unless you need a shell variable.