Is it safe to cut at every # character?

Q: Is it safe to cut at every # character?

No. A # can be part of a value, such as a URL fragment, a quoted string or a colour code. For config formats where that happens, only strip # that starts a line or follows whitespace (s/[[:space:]]+#.*$// with -E), or use a parser for the format.

Bash and Linux

Strip comments and blank lines from a config

mediumsed

You want to diff the effective settings of two nginx configs, so all the comments and empty lines have to go. Remove full-line comments (including indented ones), inline comments at the end of a line together with the spaces before them, and every line that is blank or becomes blank. Keep indentation of the remaining lines.

nginx.conf

Bash

# upstream settings
upstream api {
    server 10.0.0.11:8080;  # primary
    server 10.0.0.12:8080;
 
    # server 10.0.0.13:8080;  (retired)
}
 
  # timeouts
proxy_read_timeout 60s;

Examples

Example 1

Input: Run the command on `nginx.conf`.

Output: upstream api { server 10.0.0.11:8080; server 10.0.0.12:8080; } proxy_read_timeout 60s;

Hints

Approach

Two expressions run in order on each line. The first, s/[[:space:]]*#.*$//, deletes a #, everything after it, and any whitespace just before it. On a full-line comment that leaves the line empty or whitespace-only. On server 10.0.0.11:8080; # primary it trims the inline comment. The second, /^[[:space:]]*$/d, deletes any line that is now empty or only whitespace, covering both the original blank lines and the lines emptied by step one. Because the deletion happens after the substitution, one pass handles everything.

Bash

sed -e 's/[[:space:]]*#.*$//' -e '/^[[:space:]]*$/d' nginx.conf

Follow-up questions

How would you use this to diff two configs in one command? (diff <(sed ... a.conf) <(sed ... b.conf))
How would you print only the lines between upstream api { and }?

Frequently asked questions

No. A # can be part of a value, such as a URL fragment, a quoted string or a colour code. For config formats where that happens, only strip # that starts a line or follows whitespace (s/[[:space:]]+#.*$// with -E), or use a parser for the format.