Quite some time ago, we discussed how to get compile Minecraft and get it running on FreeBSD. In this article, we take the server as an example how we can monitor system performance.
Minecraft Memory Usage
Minecraft is a Java program. Java programs generally consume more memory than the counterparts made of unmanaged languages. Thankfully, Java programs run inside their own sandboxes and have memory usage allocated and constrained. In the previous article, we defined memory to be 1024 megabytes and expands up to 1024 megabytes only:
java -Xmx1024M -Xms1024M -jar spigot*.jar
It is important not to have Minecraft overrun the system memory. As folklore, a Java program running on a dedicated computer should not be higher than 60% of total memory. For example, my virtual machine has 2048 megabytes of memory and 60% of it is about 1200 megabytes. I deducted myself further for 200 megabytes as safety margin.
General Process Monitoring
FreeBSD provides top(1) utility to check for system saturation and utilisation. Generally speaking, a system is considered saturated if the number of threads ready to run is higher than the number of processor cores, and considered fully utilised if the utilisation numbers are near 100%.
In the top part, there are three numbers in decimal point, representing the load averages in 1, 5, and 15 minutes. This is calculated by average number of threads ready to run and is regarded as the saturation of the processors. In the third and forth row, the overall processor and memory utilisation ratios / rates are shown.
In the picture, the system not yet saturated as there is about 0.6 threads to run per second. It is under some computation load that around 20% processing power and 80% memory are utilised. This is a healthy situation that the system is being used, with some slacks to handle possible usage spikes.
In the bottom part, we have detailed breakdown per process. The fields “TIME” and “WCPU” represents the total time and current portion of processor each process is using. Actual amount of memory usage are represented in the “RES” column.
In the picture, we see the Java process using 44% of current processing power and it has accumulated 295 hours. Although the Java process is instructed to use only 1024 megabytes of memory, eventually it consumes 1368 megabytes. This echoes why the folklores recommend only specify 60% of system memory to a Java virtual machine.
System Input / Output Monitoring
FreeBSD provides systat(1) utility for system usage statistics. Since the general process monitoring can be done by the top(1) command above, it is mostly useful for detailed input and output monitoring. By default, it shows a “pig” screen that refreshes every 10 seconds. You can specify what you want to see by specifying in commands, for example, to see network interface usage every second:
systat -ifstat 1
The screen works like vi(1). You can press the colon sign (:) and to enter a command. The first command you want to try is “help”. It immediate tells you the list of available pages you can switch to. You can then use the colon sign again and enter the page you want.
I would tell you vmstat is my favourite page. I learned it on the very first days I was taught FreeBSD. It shows a lot of comprehensive information from system utilisation to interrupts and disk accesses.
systat -vmstat 1
On the top, we see the system saturation and utilisation as usual. Immediately under it, we see the detailed breakdown system events like context switches (Csw), traps (Trp), system calls (Sys), etc. The system is now having thousands of context switch per second. It would be no good if it were a system for high performance computing, but for our context of gaming with network messages, it is absolutely normal.
Further below, we see an ASCII art of system utilisation break down into system (=), interrupts (+), user space applications (>) and niced user applications (-). The system is using mild amount of processing power and most of it are for the user space, which is a good thing.
The second bottom section shows the name caches of the virtual file system. For a gaming system that uses few files, you can expect the hit rate be near 100%. Otherwise you may want to pay more attention to the name caches. In the picture, we see the system rarely searches for a file and those requests are handled by the cache perfectly.
The bottom section shows the disk utilisation. Utilisation and bandwidth of each of the virtual disks are shown. In the picture, we see the system rarely access the files.
The right most column shows the interrupt statistics. The “timer” interrupts happens almost all the time in order to hint the operating system to context switch and update system clocks. It used to be 100 or 1000 per processor core; thankfully, with the advent of more advanced system clocks, systems no longer need to tick as frequent as before. In the picture, the network card (virtio-network) requires quite some interrupts handling. As long as the network card interrupts does not go to numbers like 50000 or 100000, they are most likely normal.
The list of pages available in the tool, as of today, are:
- pigs: shows the processes which consume the most processing power
- vmstat: (as discussed above)
- swap: shows the system swap situation
- zarc: shows the ZFS adaptive read cache situation
- iostat: raw disk input and output statistics
- netstat: network socket statistics, such as buffered bytes for each of the connections
- sctp: stream control transport protocol statistics
- tcp: transport layer protocol statistics
- ip: internet protocol statistics
- ip6: internet protocol statistics for IPv6
- icmp: internet control message statistics such as ping, etc
- icmp6: internet control message statistics for IPv6
- ifstat: raw network interface utilisation statistics
In this article, we go through some performance monitoring tools that come with FreeBSD. The general process information can be listed by the top(1) command, where you can understand the system saturation and utilisation, and also list the resource consuming processes. More detailed resource utilisations like network, disks, hardware interrupts can be found in the systat(1) command. If in doubt, the “vmstat” page can be a great starting point to look for congested system resources.