Linux Performance Hunting Tips - Take Copious Notes (Save Everything)
Probably the most important thing that you can do when investigating a performance problem is to record every output that you see, every command that you execute, and every piece of information that you research. A well-organized set of notes allows you to test a theory about the cause of a performance problem by simply looking at your notes rather than rerunning tests. This saves a huge amount of time. Write it down to create a permanent record.
When starting a performance investigation, create a directory for the investigation, open a new "Notes" file in GNU emacs, and start to record information about the system. Then store performance results in this directory and store interesting and related pieces of information in the Notes file. Suggest that you add the following to your performance investigation file and directory: