Skip to main content
Bash and Linux

Clean Up a Config

mediumsed and config editing

Problem statement

Clean a messy config so only the real settings are left: remove Windows line endings, comment lines (even indented ones) and blank lines, then print just one [section] of it. You do this to see what a config really sets, to compare two servers' configs, or to fix a file copied from Windows that breaks scripts.

config.ini (every line ends with a Windows \r)

Bash
# main config
[server]
port = 8080
# indented comment
host = 0.0.0.0
[database]
user = app
; old style comment
pass = secret
[cache]
size = 64
  1. Count the lines that end with a Windows \r.
  2. Print only the real settings and section headers: no \r, no comments (# or ;, indented or not), no blank lines.
  3. Print only the [database] section.
  4. Save the cleaned file and count the \r lines again.

Expected output:

TEXT
== lines ending in a Windows \r ==
12
== real settings only ==
[server]
port = 8080
host = 0.0.0.0
[database]
user = app
pass = secret
[cache]
size = 64
== only the [database] section ==
[database]
user = app
pass = secret
== lines ending in \r after cleaning ==
0

Hints

Hint 1: sed '/pattern/d' deletes every line that matches. ^[[:space:]]*# matches a line that starts with optional spaces and then #.

Approach

Optimal: sed addresses: d, p and ranges

Covers: sed addresses (/pattern/, line numbers, $, ranges /a/,/b/), commands d and p, -n, -e, [[:space:]], s/\r$//, grep -c $'\r'.

Every sed command can have an address. An address picks which lines the command works on. Without one, the command runs on every line. With one, it only runs on lines that match:

%%{init: {"flowchart": {"padding": 18, "nodeSpacing": 30, "rankSpacing": 40, "htmlLabels": true}, "themeVariables": {"fontSize": "18px"}}}%% flowchart TB L["next line"]:::gray --> A{{"address matches?"}}:::purple A --> CMD["run the command
d, p or s"]:::green A --> SKIP["leave the line
as it is"]:::gray classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px
Address Picks
3 line 3
$ the last line
/^#/ lines that start with #
/^\[database\]/,/^\[/ from the [database] line to the next line starting with [
1! every line except line 1 (! flips the address)
Command Does
d delete the line (do not print it)
p print the line
s/a/b/ substitute, as on the previous page

Deleting comments and blanks. [[:space:]] matches a space or a tab, and * means "any number of them, including none". So:

  • /^[[:space:]]*[#;]/d deletes lines that start with optional spaces, then # or ;. That catches # indented comment too.
  • /^[[:space:]]*$/d deletes lines that are empty or only spaces.

The Windows \r problem. Windows ends each line with two characters, \r\n; Linux uses only \n. A file copied from Windows keeps an invisible \r at the end of every line. It breaks scripts (bash: $'\r': command not found), makes grep 'secret$' fail, and makes values like 8080\r invalid. s/\r$// removes it. grep -c $'\r' counts lines that contain one; the $'\r' form is how bash writes a real \r character.

Printing a range with -n and p. -n turns off sed's normal printing, so only lines you p (print) appear. /^\[database\]/,/^\[/p starts printing at [database] and stops after the next line that starts with [, which is [cache]. The brackets are escaped as \[ because [ is special in a pattern. The second sed '1!{/^\[/d}' deletes header lines on every line except the first, which removes the extra [cache].

%%{init: {"flowchart": {"padding": 18, "nodeSpacing": 30, "rankSpacing": 40, "htmlLabels": true}, "themeVariables": {"fontSize": "18px"}}}%% flowchart TB subgraph R["sed -n '/^\[database\]/,/^\[/p'"] direction TB S1["[database]
range starts"]:::green ~~~ S2["user = app"]:::green ~~~ S3["pass = secret"]:::green ~~~ S4["[cache]
range ends, removed after"]:::red end classDef blue fill:#dbeafe,stroke:#2563eb,color:#1e3a8a,stroke-width:2px classDef yellow fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:2px classDef green fill:#d1fae5,stroke:#059669,color:#064e3b,stroke-width:2px classDef red fill:#fee2e2,stroke:#dc2626,color:#7f1d1d,stroke-width:2px classDef purple fill:#ede9fe,stroke:#7c3aed,color:#4c1d95,stroke-width:2px classDef gray fill:#f3f4f6,stroke:#6b7280,color:#111827,stroke-width:2px linkStyle default stroke:#94a3b8,stroke-width:2px style R fill:transparent,stroke:#7c3aed,stroke-width:2px

Walking through the code. The # Setup: line uses printf with \r\n so every line really ends the Windows way, so skip past it.

  1. grep -c $'\r' counts 12 lines with a \r.
  2. The clean function runs three -e commands: strip \r, delete comment lines, delete blank lines.
  3. The range command prints the [database] block.
  4. clean config.ini > clean.ini saves the result, and the count is now 0.

Edge cases. A comment after a value, like port = 8080 # main, is not removed by these commands; add s/[[:space:]]+#.*// with -E if your format allows that. If [database] is the last section, the range runs to the end of the file, which is still correct.

# Setup: write the sample config with Windows line endings (\r\n) in a fresh temporary folder
cd "$(mktemp -d)"
printf '%s\r\n' '# main config' '[server]' 'port = 8080' '    # indented comment' 'host = 0.0.0.0' '' \
  '[database]' 'user = app' '; old style comment' 'pass = secret' '[cache]' 'size = 64' > config.ini

clean() {
  sed -e 's/\r$//' \
      -e '/^[[:space:]]*[#;]/d' \
      -e '/^[[:space:]]*$/d' "$1"
}

echo "== lines ending in a Windows \\r =="
grep -c $'\r' config.ini

echo "== real settings only =="
clean config.ini

echo "== only the [database] section =="
clean config.ini | sed -n '/^\[database\]/,/^\[/p' | sed '1!{/^\[/d}'

clean config.ini > clean.ini
echo "== lines ending in \\r after cleaning =="
grep -c $'\r' clean.ini || true

Interview follow-ups

  • Also strip trailing spaces and print the settings sorted, to compare two servers.

    Add one more command, -e 's/[[:space:]]*$//', which removes spaces and tabs at the end of every line. Then sort: clean server1.ini | sort > a.txt and the same for server2. diff a.txt b.txt now shows only real differences, not changes in comments, blank lines or order. This clean-sort-diff trick is a quick way to find config drift between machines.

Frequently asked questions

cat -A file shows them: a \r appears as ^M, a tab as ^I, and the end of each line as $. file config.ini also tells you "with CRLF line terminators" when a file has Windows endings. Many editors can show them too. Once you know they are there, sed -i 's/\r$//' file or dos2unix file removes them.

Plain sed prints every line unless you delete it with d, so you say what to throw away. sed -n prints nothing unless you print it with p, so you say what to keep. Use d when most lines stay, like removing comments. Use -n ... p when you want a small part, like one section or a line range. Mixing p without -n prints matching lines twice, which is a common surprise.

For comments and blank lines, grep -v -E '^[[:space:]]*([#;]|$)' does the same job in one step, and that is fine. sed earns its place when you also need to change lines, like removing the \r, and when you want ranges such as "from [database] to the next header", which grep cannot do. Knowing both lets you pick the shortest clear command.