is an easy way to find important stuff in logs. Yes it provides
fast index-based searches when you know what to look for, but its
distinguishing feature is that it helps you find anomalies when you
don't know what to look for. It ranks log files and colorizes
words based on information theory (answering "what is the strangest
logfile, and why?"). It provides useful file and word statistics
in interactive tabular and plot formats. It uses Vaarandi's excellent loghound
program to generate message templates for you (which you can then use
as regular expressions in monitoring programs if you wish). Web
and command line interfaces are included.
Sisyphus has been
developed specifically for use with supercomputer syslogs, based on the
premise that similar computers correctly executing similar workload
should produce similar logs (thus, anomalies warrant
investigation). But it is general enough to be used on other log
types (if your logs are not syslog, you will have to tweak arguments
and/or provide a parser).
Latest Release: 1.7 on 2008-08-28 (readme, changelog, download). License:
- Alert Detection in System Logs. A. J. Oliner, A. Aiken,
and J. Stearley. In Proceedings of the International Conference on
Data Mining (ICDM), Pisa, Italy, 2008.
- Bad Words: Finding Faults in Spirit's Syslogs.
J. Stearley and A. J. Oliner. In Workshop on Resiliency in
High-Performance Computing (Resilience-2008), Lyon, France, 2008.
- What Supercomputers Say: A Study of Five System Logs. A. J. Oliner and J. Stearley. In Proceedings of the International
Conference on Dependable Systems and Networks (DSN), Edinburgh, UK,
- Towards Informatic Analysis of Syslogs.
J. Stearley. IEEE Conference on Cluster Computing, Sepetember 2004.
sisyphus-announce for release announcements, and
for general help and discussion. NOTE: the list archives are not viewable outside Sandia - SORRY!!!.
Email Jon Stearley <email@example.com> for further information.