CodePark/ questions / Linux commands & permissions
START PRACTICINGPRACTICE
Backend question · OS, networking & Linux › Linux commands & permissions

Explain what this pipeline does, step by step, and why pipes are useful for log analysis.

awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -5

The pipeline finds the five IPs that made the most requests. A pipe | connects the stdout (output) of one process to the stdin (input) of the next. All the commands run at the same time. Data streams between them through a buffer in the kernel.

Step by step:

  • awk '{print $1}' prints the first field of each line. Fields are split by whitespace. In the common access log format, the first field is the client IP.
  • sort puts identical IPs next to each other. This matters because uniq only merges duplicates that are adjacent (next to each other).
  • uniq -c turns each group of the same IP into one line, with its count in front.
  • sort -rn sorts those lines by number, in reverse, so the largest count comes first.
  • head -5 keeps the top five lines.

Why pipes help with logs: each tool does one small job. Together they answer quick, one-off questions about huge files. You don't load the file into memory, and you don't write a script. For example, start with grep ' 500 ' access.log instead, and the same pipeline ranks clients by server errors.

  • A pipe sends one command's stdout into the next command's stdin.
  • The commands run at the same time and stream the data.
  • uniq only merges adjacent lines, so you must sort first.
  • uniq -c counts, sort -rn ranks, and head limits the output.