Paragon "Good Citizen" Usage Rules

The intent of Sandia National Laboratories is to allow as many users as possible access to the 1824-node Intel Paragon, acoma, for large problems which cannot be run elsewhere. However, priority will be given to users who have contributed toward the purchase of the machine or who contribute to its maintenance. Since time on acoma is a limited resource, all users are expected to observe the following rules. They are designed to balance the needs for development and production use of acoma by a wide community of users. Our intent is to keep acoma busy doing useful work rather than to strictly enforce usage limits. We will succeed as users abide by these rules.

Usage and reservation rules for the 1824-node Intel Paragon (acoma)

Prime time is 8 a.m. to 5 p.m. Albuquerque time on normal Sandia work days. All other time is batch or dedicated time. During prime time, the nodes are divided into interactive nodes and batch nodes. The batch nodes are controlled by the NQS queuing software. During non-prime time, all the nodes are controlled either by NQS or are dedicated to interactive use. Our intent is to encourage batch use rather than interactive use.

During prime time the following guidelines apply:

Interactive Nodes

Interactive nodes should be used primarily for code development rather than for production calculations.
Time limits for jobs or scripts of jobs on the interactive nodes are determined by the following schedule:

         Number of Nodes                   Maximum Duration
         ---------------                   -------------------------------------
             1 to   20                     0   hours*
            21 to  544                     0.5 hour

       *This machine is intended for large jobs; hence the running of small jobs
        on it is discouraged.  Jobs using fewer than 21 nodes may be run for
        brief periods provided they do not interfere with running larger jobs.
        Jobs using fewer than 21 nodes may be terminated without warning and 
        without regard for the effects of such termination.

Ganging jobs either manually or via scripts on the interactive nodes violates the intent of these rules to provide equitable access to the Paragon for both code development and production calculations for the user community.
A job or script of jobs that breaks these rules may be terminated without notice and without regard to the effect of such termination.

Batch Nodes

Batch nodes are accessed via a system of queues. The current queues and their limits are given in the following table.

      Queue Name          Maximum Nodes      Maximum Time
      --------------      -------------      ------------
      backgrnd.day            1024             1.25 hours
      default.day             1024             1.25 hours
      express.day             1024             1.25 hours*
      sandia.day              1024             1.25 hours
      develop.day              256             0.25 hour

     *Requires management request and approval.  The user requesting the use of
      the express.day queue must have the request submitted via e-mail to
      iparagon@cs.sandia.gov by his or her manager.  The request must include
      the project which requires the use of the express queue, must clearly
      state why the normal queues are inadequate for the user's needs, and must
      provide an estimate of how many jobs will be run, how many nodes each job
      will use, and how long each job will run.

      The develop.day queue is intended for code development.  The other queues
      may be used for code development or production calculations.

      Users are permitted access to these queues based on their organizational
      affiliation.  Each user may have up to three jobs submitted per queue;
      excess jobs will be deleted from the queues.

During non-prime time, the following guidelines apply:
Batch Nodes
- Normally NQS controls all the nodes during non-prime time. During this time, jobs are run from the following queues:
Interactive Nodes
- Users can reserve time slots during non-prime time for dedicated interactive use for a specific project subject to management approval.
- Users should make every effort to use the existing queues rather than request dedicated time for interactive nodes.
- All non-prime time is available for reservations; however, reservations for non-prime time during the week are discouraged.
- Allocations are made for one week at a time, Monday morning through Sunday evening.
- Reservation requests must be submitted at least one week in advance. In general, requests should not be made for time slots more than one month away. Requests must include an explanation of why the work cannot be accomplished via the NQS queues. Reservations must be submitted via e-mail to iparagon@cs.sandia.gov.
- Currently, dedicated time must be scheduled by an individual user for an entire block of wall-clock time. These blocks are
  1. from 5 p.m. Albuquerque time on a Tuesday which is also a normal Sandia work day until 8 a.m. the following Thursday.
  2. special blocks which may be made available during Sandia holidays or holiday weekends, nominally from 5 p.m. on the last Sandia work day preceding the holiday until 8 a.m. on the first Sandia work day following the holiday.
The individual scheduled for the dedicated block has control over the entire machine during that block. At their sole discretion, and their own risk, they may allow others to use the unscheduled nodes.
- If there are more requests than slots, slots are granted in order of priority, usually on a first-come, first-served basis.
- Within two working days after the dedicated time, the user must submit a short report to iparagon@cs.sandia.gov describing the work accomplished and any problems encountered.
Use of the parallel disk system is governed by the following:
- Please remember that disk space on acoma and zia is a limited resource.
- We must maintain sufficient storage on the /raid and /pfs file systems to support jobs run on acoma under NQS in non-prime time. Files stored there must remain long enough for users to have a reasonable opportunity to analyze the results contained therein or to move the files to more permanent storage.
- As much as possible, users should spread files over disks 1 through 32 or 1 through 48.
- The following policy governs file storage on the /raid, /pfs, and /usr/tmp file systems on acoma.
The basic policy is that storage on these file systems is scratch storage. It is not backed up, and users can expect only short-term storage there.
- This policy is implemented as follows: Files larger than one megabyte which have not been accessed in more than 14 days and files smaller than one megabyte which have not been accessed in more than 21 days will be deleted.
- Files will be deleted more frequently if the above procedure does not free sufficient space. We will also delete files accessed in less than 14 days (files larger than one megabyte) or less than 21 days (files smaller than one megabyte) if necessary to free sufficient space. In general, we will endeavor to delete older files before newer ones.
- Attempts to defeat this policy by using cron jobs and the touch command to change the access times of files or by any other means will result in the deletion of such files without consideration for the effects of such deletion on the owner of the files.
- We realize that occasionally special circumstances arise where deviations from the policy and its procedures are justified, and when requested to do so we will make every reasonable effort to accommodate the needs of our users. Please notify the staff at iparagon@cs.sandia.gov if you have such special needs.

                                                                   11 April 1996