unix shell script: count the file handles for a bunch of processes

Note: this was originally posted April 22, 2009 on my old blog.

We are using WebLogic with the JRockit JVM and noticed tons of issues after a few weeks of uptime - spurious errors and non-responsiveness.  The cause?  Too many open file handles.  Well, actually it's not so much that there's too many open file handles, it's that the OS (Suse Linux Enterprise Server 10) has a very low limit by default.  The max file handles is over 400,000 for our production server (it's pretty high-end) but each process is limited to only 1,024.

After googling around a bit to figure out how to up the limit, I also put together this script to log the number of file handles being used by each of our WebLogic servers.