While the primary role of the process management system is to manage the flow through the system, you will likely need to implement further systems around it to allow for auditing and monitoring.
There is no single solution or this, as the requirements will differ depending on the reasons for implementing it. These include:
- Internal oversight of users’ actions
- Performance monitoring
- External regulatory oversight
While the design for the data structure of the underlying system was made with the principles of data normalisation, eliminating duplicate storage of information, the imperative behind audit and monitoring is very different, and can generate vast amounts of extra data, which should be considered when sizing servers and storage.
You should also consider how long audit information need be kept, and include actions to remove the data after this period.
Audit data can be kept internal to the system, as separate records or tables, or may be external.
External audit mechanisms
A simple example of external audit is sending an email to an audit mailbox, with the email containing information about an event that has just occurred. Such a system has pros and cons:
Pros:
- Simple to implement
- Does not affect the data structure of the core system
- Does not take up any extra capacity on the core system
- Hard to repudiate
Cons:
- May be hard to query or build reports from the data
- May be security implications if emails are sent unencrypted over the public internet
Internal audit mechanism
Internal audit mechanisms are typical constructed when more detailed audit questions need to be answered. In designing the audit aspect of a system it is important to consider exactly what these audit questions will be: it is impractical to simply say you need to ‘audit everything’, as that is rarely what is actually needed. Instead focus on the necessities, and work from that.
For example, a key question is whether you need to audit the action of someone just observing, looking at data, or whether you only ned to consider actions that change the data. In most cases, you will only need to consider changes to the data.
How you proceed from here will depend on the detail of the process being audited. For a simple system where an entity is taken through a few linear stages it may be sufficient to record, for each stage transition, which user triggered the action, and when they did so.
Extra fields method
For each stage, add extra ‘date’ and ‘userid’ fields, to the entity’s record. This is a very simple method, but is not recommended for all but the most trivial of systems.
Pros:
Cons:
- Only stores specified audit data
- Only works for linear flow (if stages are repeated, earlier occurances of that stage are overwritten)
- Harder to query if many stages
Audit table method
More in line with data normalisation, for this method create a new table ‘audit’, and give that fields of ‘date’, ‘userid’, ‘stage’ and a recordlink back to the entity record. This approach is easier to manage should extra stages be added in later.
The above model is assuming the entity record is being updated from time to time, with the ‘stage’ field being changed. The audit table is kept updated with new records being added to it whenever a change is made, but the audit table can otherwise be ignored by other parts of the site (unless audit questions specifically need to be asked).
Pros:
- Easy to implement
- Easy to make subsequent changes
- Audit aspects can be ignored unless needed
Cons:
- Only stores specified audit data
Snapshot records
A more invasive audit approach is to implement a system where the entity record is never changed, but instead, whenever a form is submitted, a new record is created with the changes incorporated within it. This is also known as the ‘Save as’ method, since it is the equivalent of taking say a Word document, and instead of regularly ‘Saving’ it, the user clicks ‘Save as’ and gives the document a new version number each time. Old versions are left as historic records, and the latest version added to the table. At first sight this is an attractive method, since being so comprehensive, the solution is bound to satisfy any question an auditor may ask.
However it is not that simple. This approach requires careful treatment throughout the configuration of the system, as the table is now ‘polluted’ with historic versions that must be explicitly ignored by all operations that are not concerned with the audit history. Simply querying the number of records in the table is now a more complex question. Another aspect to be considered with this approach is when the data structure includes recordlinks between the table being audited this way, and another table. If the recordlink are going from the audited table, the duplicate record will maintain the links correctly, but if the recordlinks are coming in the other direction, towards the audited record, then they will point to the old snapshot record, and not to the latest version. Event actions could be configured to adjust these recordlinks, but it is one more aspect to consider throughout.
Pros:
- Comprehensive audit data stored
Cons:
- Harder to implement
- Harder to make subsequent changes
- Audit record must always be considered even by parts of the system not concerned with audit
