OpenSolaris Community: Observability
View the leaders for this community
Community Observers
OpenSolaris Observability
Over the course of history, we've done a lot of debugging in Solaris. Whether
it's processes misbehaving, performance degredations, system panics, or hardware
failure, there is one key element of any approach:
If you cannot observe the problem, you cannot fix it.
To this end, we have spent a great deal of time on robust tools to observe all
aspects of system behavior. Some of these have been around forever; others are
relatively new. They all make the lives of administrators and developers much
easier. This community will serve to explore existing Solaris tools and
features, as well as plan the next generation of OpenSolaris observability
tools.
We've divided this rather large cross section of Solaris functionality into
several specific areas, described here. Most of these just link to the relevant
code. Over time, those that are sufficiently complex will link to more detailed
documentation.
Process Observability
| DTrace |
A fantastic tool that can observe process and system behavior and tie it all
together. Check out the DTrace community for more information. |
| truss |
A tool for examining system calls as well library calls made by an
application. |
| ptools |
A suite of process observability tools. |
| libproc |
A common library used to simplify examining and manipulating
processes. |
| procfs |
The /proc filesystem, upon which libproc and all process tools are
built. |
| MDB |
The modular debugger, the standard debugger for Solaris. See the MDB
community for more information |
| ps |
Basic process monitoring tool. Displays a snapshot of current process state
on the machine. |
| prstat |
The standard process monitoring tool for Solaris, similar to
top. |
| plockstat |
The userland equiavlent of lockstat(1M), used to monitor lock
events and aid in MT scalability. Built on top of DTrace |
System Observability
| DTrace |
Once again, the indispensible system monitoring tool. |
|
kstat
|
A kernel statistics framework used by a variety of tools to examine system
wide staistics. |
|
NUMA
|
Tools for examining NUMA (Non Uniform Memory Architecture) characteristics
on Solaris. |
| ctfs |
The contract filesystem, used to observe and manipulate process contracts. |
| objfs |
The object filesystem, used to examine kernel module symbols and CTF data from
userland. |
mpstat
iostat
vmstat |
Tools to report on basic system statistics. |
| lockstat |
Tools to examine kernel locking statistic, as well as basic profiling data. Implemented using
DTrace |
| intrstat |
Tools to examine kernel interrupt statistics. Implemented using DTrace |
| trapstat |
SPARC-only tool to examine trap statistics. |
Network Observability
| snoop |
The standard tool for observing network packets. |
| netstat |
Tool for observing network connection status. |
| nfsstat |
Basic tool for monitoring NFS statistics. |
Hardware Observability
| FMA |
Fault Management Architecture, a unified system of hardware fault diagnosis, repair, and
reporting. |
| CPC |
A system to gather and report CPU performance counter information. Consists of kernel
components, libcpc,
cpustat, and
cputrack |
| psrinfo |
Report on current processor status and configuration |
| prtconf |
Display the system device tree, optionally in excrutiating detail. |
| prtpicl |
Similar to prtconf, prints out the state of the device nodes maintained by the PICL daemon. |
| prtdiag |
Display summary of attached hardware components. |
| prtfru |
SPARC only - Display FRUID information for system or domain. |
Post Mortem Observability
| MDB |
The standard debugger. See the MDB community for more information. |
| dumpadm |
Configures kernel crash dump generation. |
| coreadm |
Configures process core file generation. |
| CTF |
Compact C Type Format, the format used to store type information in the kernel and userland.
Consumed by MDB and DTrace, among others. |
| process core
generation |
The elfcore() routine, which is responsible for the nuts and bolts of core file
generation. |
| kernel panic
code |
The entry point to the kernel panic code. |
| gcore |
The gcore(1) utility for taking a core dump of a live running process without perturbing the process. |
Unbundled Tools
More to come...
Blogs
May 5, 12:06 AM
This originally was going to be a post-mortem on dtrace.conf, but so much time has passed, that I doubt it qualifies anymore. Back in March, we held the first ever DTrace (un)conference, and I hope ...
Apr 10, 9:00 AM
It was a good run, but Jarod and I didn't make the cut for JavaOne this year... 2005 In 2005, Jarod came up with what he described as a jacked up way to use DTrace to get inside Java. This became the ...
Apr 7, 9:59 PM
I was having a conversation with an OpenBSD user and developer the other day, and he mentioned some ongoing work in the community to consolidate support for RAID controllers. The problem, he was ...
Mar 16, 9:45 PM
dtrace.conf(08) was this past Friday, and (no surprise, given the attendees), it ended up being an incredible (un)conference. DTrace is able to cut across vertical boundaries in the software stack ...
Mar 13, 1:11 AM
The other day, there was an interesting post on the DTrace mailing list asking how to derive a process name from a pid . This really ought to be a built-in feature of D, but it isn't (at least not ...
|