CodePark/ questions / Debugging a server in production
START PRACTICINGPRACTICE
Backend question · OS, networking & Linux › Debugging a server in production

Your service on a new server can't connect to the database at db.internal:5432. How do you narrow down the cause, layer by layer?

getent hosts db.internal
nc -vz db.internal 5432
psql "host=db.internal user=app dbname=app" -c "select 1"

Check one layer at a time, from the bottom up. The error at each step tells you which layer failed.

1. Name resolution (DNS). Run getent hosts db.internal or dig. Check that the name resolves, and that it resolves to the address you expect. Private names often resolve only inside the right network or VPC (Virtual Private Cloud: your private network in the cloud).

2. Reachability (TCP). Run nc -vz db.internal 5432. It tries a TCP handshake to that port. Read the result:

  • An immediate "connection refused" means the host answered, but nothing listens on that port. Postgres is down, or it listens only on localhost because of its listen_addresses setting.
  • A hang and then a timeout means something drops the packets. The cause is a security group, a firewall or a routing problem. Or the IP is wrong.

3. Application (Postgres). Once TCP connects, try psql. Now the errors come from Postgres itself:

  • pg_hba.conf rejects the client's IP.
  • The credentials are wrong.
  • The server requires TLS.
  • too many clients: all max_connections are in use.

Finally, check which settings the app really loaded. An old, stale DATABASE_URL is a common cause.

  • First check that DNS resolves the name to the expected IP.
  • Test TCP with nc -vz and see if you get "refused" or a timeout.
  • "Refused" means the service is down or listening on the wrong address.
  • A timeout means a firewall, a security group or a routing problem.
  • Then check auth and config: pg_hba.conf, credentials, TLS and max_connections.