Help us improving Robinhood FS Monitor, and fit your needs, by answering this survey.

RobinHood - FS Monitor

RobinHood FS Monitor is a tool for monitoring and purging large filesystems. It is available under the CeCILL license, which is a French transposition of the GNU GPL  and is fully GPL-compatible.

Its main purpose is to collect statistics about file system content and trigger purge when disk usage exceeds a given threshold. It implements disk-space fair-share, to ensure quality of service for filesystems, preserve data of nice and responsible users, penalizing "abusers" and preventing from harmfull behaviors... That's why it is called RobinHood :)

RobinHood is designed to perform all its tasks in parallel, so it is particularly adapted for managing large filesystems with millions of entries and petabytes of data.

Main features

• Disk space fair-share
• Policy Engine for temporary filesystems and Lustre-HSM:
  • Purge policies
  • Migration policies
  • Deferred removal policy
  • Old empty directories cleanup policy
• Attribute-based policies, possibly using sophisticated boolean expressions
• Accounting and monitoring
• Aware of Lustre OSTs and pools

Architecture overview

• Since version 2.0, Robinhood uses a database engine for managing its list of entries, which offers a lot of benefits:
  • It can manage larger filesystems, because the size of its list is no more limited by the memory of the machine.
  • Purges can be performed immedialty, even after a system crash, because the list is persistent.
  • Robinhood v2 can be split in several daemons running on several nodes, for more scalability: there is no direct communication between several robinhood instances, they are only clients of the same remote database.
  • Administrators can retrieve custom and complex reports about filesystem content using a very standard language (SQL).
  • It can manage a transactionnal stream of events as input (basically, Lustre's MDT ChangeLog) without loosing events, because it impacts those events in its database using transactions.
• Input management is very modular and many types of information can be processed: namespace scans, streams of events, pre-generated list of entries... Information is processed efficiently, in parallel and asynchronously, thanks to a pipeline architecture.
• Policies are very flexible and easy-to-specify, thanks to its internal Boolean expression engine.
• Large choice of purge triggering: triggers can be specified on global FS usage, OST usage, user or group usage, external command output...

For more information about Robinhood v2, see the documentation section of this site.

Robinhood v2 is available here To run it as Lustre-HSM PolicyEngine, you need to have all Lustre-HSM patches applied on your Lustre version.