Pipes

You can chain any number of commands by separating them with |. Data flows from left to right. Each command runs in its own subshell.

Each command in a pipeline runs as an independent subshell (child process). This means that variables defined inside the pipeline and directory changes made with cd do not affect the outer shell (the calling shell).

kof_pipe_basic.sh

#!/bin/bash
# -----------------------------------------------
#  Process KOF character data using pipelines.
#  Basic examples.
# -----------------------------------------------

# Define character data using a here-string.
# Format: name,team,country
CHARACTERS="テリー・ボガード,餓狼伝説チーム,アメリカ
庵,八神チーム,日本
クリス,オロチチーム,不明
草薙京,草薙チーム,日本
キング,アートオブファイティングチーム,フランス
テリー・ボガード,餓狼伝説チーム,アメリカ
草薙京,草薙チーム,日本"

echo "=== Original data ==="
echo "${CHARACTERS}"
echo ""

# -----------------------------------------------
#  1. Filter with grep.
#     Extract only rows where the country is Japan.
# -----------------------------------------------
echo "=== 1. Characters from Japan (grep) ==="
echo "${CHARACTERS}" | grep "日本"
echo ""

# -----------------------------------------------
#  2. Sort with sort.
#     Sort by character code order.
# -----------------------------------------------
echo "=== 2. Sort by name (sort) ==="
echo "${CHARACTERS}" | sort
echo ""

# -----------------------------------------------
#  3. Remove duplicates with sort | uniq.
#     Sort first to make duplicates adjacent, then
#     remove them with uniq.
# -----------------------------------------------
echo "=== 3. Remove duplicate lines (sort | uniq) ==="
echo "${CHARACTERS}" | sort | uniq
echo ""

# -----------------------------------------------
#  4. Count lines with wc -l.
#     Output the number of unique characters.
# -----------------------------------------------
echo "=== 4. Number of unique characters (sort | uniq | wc -l) ==="
count=$(echo "${CHARACTERS}" | sort | uniq | wc -l)
echo "${count} characters"
echo ""

# -----------------------------------------------
#  5. Extract a specific column with awk.
#     Display only the first column (name) of the
#     comma-separated data.
# -----------------------------------------------
echo "=== 5. Extract name list only (awk) ==="
echo "${CHARACTERS}" | sort | uniq | awk -F',' '{print $1}'
echo ""

$ chmod +x kof_pipe_basic.sh
$ ./kof_pipe_basic.sh
=== Original data ===
テリー・ボガード,餓狼伝説チーム,アメリカ
庵,八神チーム,日本
クリス,オロチチーム,不明
草薙京,草薙チーム,日本
キング,アートオブファイティングチーム,フランス
テリー・ボガード,餓狼伝説チーム,アメリカ
草薙京,草薙チーム,日本

=== 1. Characters from Japan (grep) ===
庵,八神チーム,日本
草薙京,草薙チーム,日本
草薙京,草薙チーム,日本

=== 2. Sort by name (sort) ===
キング,アートオブファイティングチーム,フランス
クリス,オロチチーム,不明
テリー・ボガード,餓狼伝説チーム,アメリカ
テリー・ボガード,餓狼伝説チーム,アメリカ
庵,八神チーム,日本
草薙京,草薙チーム,日本
草薙京,草薙チーム,日本

=== 3. Remove duplicate lines (sort | uniq) ===
キング,アートオブファイティングチーム,フランス
クリス,オロチチーム,不明
テリー・ボガード,餓狼伝説チーム,アメリカ
庵,八神チーム,日本
草薙京,草薙チーム,日本

=== 4. Number of unique characters (sort | uniq | wc -l) ===
5 characters

=== 5. Extract name list only (awk) ===
キング
クリス
テリー・ボガード
庵
草薙京

kof_pipe_advanced.sh

#!/bin/bash
# -----------------------------------------------
#  Explore advanced pipeline patterns using
#  KOF character data.
# -----------------------------------------------

# -----------------------------------------------
#  Data setup: create a match log for characters.
#  Format: date,winner,loser
# -----------------------------------------------
LOG_FILE="kof_match.log"

cat > "${LOG_FILE}" <<'EOF'
2026-03-01,草薙京,庵
2026-03-01,テリー・ボガード,クリス
2026-03-02,庵,キング
2026-03-02,草薙京,テリー・ボガード
2026-03-03,クリス,庵
2026-03-03,テリー・ボガード,草薙京
2026-03-04,キング,クリス
2026-03-04,庵,草薙京
2026-03-05,草薙京,キング
2026-03-05,テリー・ボガード,庵
EOF

echo "=== Full match log ==="
cat "${LOG_FILE}"
echo ""

# -----------------------------------------------
#  1. Extract winners with grep | awk and count them.
#     Tally the number of wins per character.
# -----------------------------------------------
echo "=== 1. Win count ranking (sort | uniq -c | sort -rn) ==="
# Extract the 2nd column (winner) and count duplicates.
# sort -rn sorts numerically in descending order.
awk -F',' '{print $2}' "${LOG_FILE}" | sort | uniq -c | sort -rn
echo ""

# -----------------------------------------------
#  2. Count matches for a specific character with
#     grep | wc -l.
#     Count matches where Kyo Kusanagi appears
#     as either winner or loser.
# -----------------------------------------------
echo "=== 2. Number of matches involving 草薙京 (grep | wc -l) ==="
match_count=$(grep "草薙京" "${LOG_FILE}" | wc -l)
echo "${match_count} matches"
echo ""

# -----------------------------------------------
#  3. Use tee to save a log while continuing the
#     pipeline.
#     Save Iori's wins to a file and display the count.
# -----------------------------------------------
echo "=== 3. Save Iori's wins with tee and display line count ==="
awk -F',' '$2=="庵" {print $0}' "${LOG_FILE}" | tee iori_wins.log | wc -l
echo "Contents of iori_wins.log:"
cat iori_wins.log
echo ""

# -----------------------------------------------
#  4. Variables inside a pipeline do not escape
#     the subshell.
#     Verify that a variable assigned inside a pipe
#     is not visible outside it.
# -----------------------------------------------
echo "=== 4. Subshell scope check ==="
WINS=""
# Assigning to WINS inside the pipe has no effect outside.
echo "テリー・ボガード" | while read name; do
    WINS="${name}: counting wins..."
    echo "WINS inside pipe: ${WINS}"
done
echo "WINS outside pipe: ${WINS}"   # will be empty
echo "(Variable assignments inside a pipe are not reflected in the calling shell.)"
echo ""

# -----------------------------------------------
#  5. Detect errors mid-pipeline with pipefail.
# -----------------------------------------------
echo "=== 5. Effect of set -o pipefail ==="

# Without pipefail: $? reflects only the last command even if an earlier one fails.
cat nonexistent_kof.log 2>/dev/null | wc -l
echo "Without pipefail: \$? = $?"

# With pipefail: $? becomes non-zero if any command in the pipeline fails.
set -o pipefail
cat nonexistent_kof.log 2>/dev/null | wc -l
echo "With pipefail: \$? = $?"
set +o pipefail   # disable pipefail
echo ""

# -----------------------------------------------
#  Cleanup: remove temporary files.
# -----------------------------------------------
rm -f "${LOG_FILE}" iori_wins.log
echo "=== Temporary files removed ==="

$ chmod +x kof_pipe_advanced.sh
$ ./kof_pipe_advanced.sh
=== Full match log ===
2026-03-01,草薙京,庵
2026-03-01,テリー・ボガード,クリス
2026-03-02,庵,キング
2026-03-02,草薙京,テリー・ボガード
2026-03-03,クリス,庵
2026-03-03,テリー・ボガード,草薙京
2026-03-04,キング,クリス
2026-03-04,庵,草薙京
2026-03-05,草薙京,キング
2026-03-05,テリー・ボガード,庵

=== 1. Win count ranking (sort | uniq -c | sort -rn) ===
      3 テリー・ボガード
      3 草薙京
      2 庵
      1 キング
      1 クリス

=== 2. Number of matches involving 草薙京 (grep | wc -l) ===
5 matches

=== 3. Save Iori's wins with tee and display line count ===
2
Contents of iori_wins.log:
2026-03-02,庵,キング
2026-03-04,庵,草薙京

=== 4. Subshell scope check ===
WINS inside pipe: テリー・ボガード: counting wins...
WINS outside pipe:
(Variable assignments inside a pipe are not reflected in the calling shell.)

=== 5. Effect of set -o pipefail ===
0
Without pipefail: $? = 0
0
With pipefail: $? = 1

=== Temporary files removed ===

The pipe operator (|) in shell scripts connects the standard output of the command on the left directly to the standard input of the command on the right. Because data flows without any temporary files, you can efficiently process text using patterns like grep | sort | uniq or awk | sort | uniq -c | sort -rn. Each command in a pipeline runs in its own subshell, so variable assignments inside the pipeline are not reflected in the calling shell. By default, only the exit status of the last command is captured in $?; use set -o pipefail if you also want to detect failures in earlier commands. To send standard error through the pipeline as well, combine it with 2>&1. For details on how pipes compare to redirects, see Redirect.