File Descriptor

Linux hub

A file descriptor is a non-negative integer that the Linux kernel assigns to an open file, socket, pipe, or device. Every process has a table of file descriptors. 0 is stdin, 1 is stdout, 2 is stderr. All I/O in Linux happens through file descriptors.

What Is a File Descriptor in Simple Terms

When you open a file, the kernel does not give you the file directly. It gives you a numbered ticket — a file descriptor. To read or write, you present your ticket. When you are done, you close the ticket (close the file descriptor) and it becomes available for reuse.

This abstraction is what makes Unix so elegant: files, network sockets, pipes, and devices are all represented the same way. The same read() and write() system calls work on all of them.

How It Works

Every process has its own file descriptor table. The kernel maintains a system-wide open file table that the process tables reference.

◈ DIAGRAM

Process file descriptor table:
+----+----------------------------------------+
| FD | Points to                              |
+----+----------------------------------------+
|  0 | stdin (keyboard or pipe input)         |
|  1 | stdout (terminal or pipe output)       |
|  2 | stderr (terminal error output)         |
|  3 | /var/log/app.log (opened by the app)   |
|  4 | socket (TCP connection to database)    |
|  5 | /etc/config.yaml (config file)         |
+----+----------------------------------------+

The three standard file descriptors:

TEXT

0  stdin   Standard Input
          Default: keyboard in interactive shell
          In scripts: first positional input
 
1  stdout  Standard Output
          Default: terminal screen
          Redirect: command > file.txt
 
2  stderr  Standard Error
          Default: terminal screen (separate from stdout)
          Redirect: command 2> errors.txt

I/O redirection uses file descriptors:

Bash

## Redirect stdout (fd 1) to file
command > output.txt
command 1> output.txt  ## same thing
 
## Redirect stderr (fd 2) to file
command 2> errors.txt
 
## Redirect both stdout and stderr to same file
command > all.txt 2>&1
## 2>&1 means: redirect fd 2 to wherever fd 1 is currently pointing
 
## Append
command >> output.txt
 
## Discard output
command > /dev/null         ## discard stdout
command 2> /dev/null        ## discard stderr
command > /dev/null 2>&1    ## discard both
 
## Redirect stdin from file
command < input.txt
 
## Here-document: inline stdin
cat << 'EOF'
This is line 1
This is line 2
EOF

Practical Commands

Bash

## List open file descriptors for a process
ls -la /proc/$$/fd/
## Shows stdin, stdout, stderr and any open files
 
## List open files for all processes (verbose)
lsof
 
## List open files for a specific process
lsof -p 1234
 
## List open files by a specific user
lsof -u www-data
 
## Find which process has a file open
lsof /var/log/nginx/access.log
 
## Find processes with deleted files still open
lsof | grep deleted
## These files consume disk space but show as deleted in ls
 
## Check file descriptor limit
ulimit -n
## Typical: 1024 (too low for high-traffic servers)
 
## Increase limit for current session
ulimit -n 65536
 
## Set system-wide limit in /etc/security/limits.conf
cat /etc/security/limits.conf
## www-data soft nofile 65536
## www-data hard nofile 65536
 
## Check current usage for a process
cat /proc/1234/limits | grep 'open files'
ls /proc/1234/fd | wc -l  ## count open FDs

Too many open files error:

TEXT

Error: EMFILE: too many open files
Error: accept4: Too many open files
 
This means the process has hit its file descriptor limit.
Each network connection uses one file descriptor.
A busy web server handling 10000 connections needs
at least 10000 file descriptors plus more for log files,
configuration files, and other resources.

Bash

## Increase for nginx in /etc/nginx/nginx.conf
worker_rlimit_nofile 65536;
 
## Increase for systemd services in unit file
[Service]
LimitNOFILE=65536
 
## Verify the service picked up the new limit
cat /proc/$(pidof nginx)/limits | grep 'open files'

Troubleshooting

Symptom	Command	What to Look For
"Too many open files" error	`lsof -p PID	wc -l`
Disk space not freed after delete	`lsof	grep deleted`
Cannot delete file in use	`lsof filename`	Process with file open
High FD usage	`ls /proc/PID/fd	wc -l`

Tip
When a file is deleted but disk space is not freed, a process still has the file descriptor open. The data blocks are not released until all file descriptors referencing the file are closed. lsof | grep deleted finds these. Restarting the process closes its file descriptors and frees the space.

Remember
File descriptor exhaustion is a common cause of production failures in high-traffic services. The default limit of 1024 is appropriate for a developer laptop but completely inadequate for a production web server handling thousands of concurrent connections. Always increase the limit as part of production configuration.

Frequently Asked Questions

Why do file descriptors matter for debugging 'too many open files' errors in production?

Each process has a per-process file descriptor limit (often 1024 by default on Linux, tunable via ulimit or systemd LimitNOFILE) covering not just files but sockets, pipes, and epoll instances. A server handling many concurrent connections or that leaks unclosed sockets/files will eventually exhaust this limit and start failing to accept new connections or open files, throwing EMFILE errors — a classic production incident cause in long-running services.

What's a common cause of file descriptor leaks in application code?

Opening a file or socket without a corresponding close in an error path — e.g. an exception thrown between open() and close() skips the cleanup — steadily leaks descriptors until the process hits its limit. Using language constructs that guarantee cleanup (Python's `with`, Java's try-with-resources, Go's `defer`) avoids this; monitoring `lsof -p <pid>` or `/proc/<pid>/fd` count over time is the standard way to catch a leak before it causes an outage.