It sounds a bit sinister, but when you use Chromeleon CDS, the audit trails are always watching you. Always – some even if you don’t activate them!
The use and review of CDS audit trails has become a regular topic in Sterling Pharma’s customer and regulatory audits starting with our MHRA (Medicines and Healthcare products Regulatory Agency) inspection in May 2015 where it commented that:
“The data checking procedure did not require staff to check audit trails as part of the data checking process.”
During customer audits I’m regularly asked to show and explain Chromeleon CDS audit trails, and the question, “Are audit trails reviewed as part of the routine data review process?” gets asked more often than not.
Why do we have Audit Trails?
The bottom line is it is a regulatory requirement – you don’t have a choice! 21 CFR Part 11 and EU GMP Annex 11 both detail the requirements for computerised systems to have audit trails. The FDA and MHRA Data Integrity Guidance go even further with the requirements. For example, the MHRA’s GXP Data Integrity Guidance and Definitions March 2018 states:
Section 6.13 Audit trails “Audit Trails (identified by risk assessment as required) should be switched on. Users should not be able to amend or switch off the audit trail. Where a system administrator amends, or switches off the audit trail a record of that action should be retained” “Routine data review should include a documented audit trail review where this is determined by a risk assessment” “Reviewers should have sufficient knowledge and system access to review relevant audit trails, raw data and meta data” |
I’m sure everyone’s risk assessment concludes that CDS audit trails fall under the category of “should be switched on and routinely reviewed,” right?! Your CDS must have audit trails, they must be on, and you must know how to use and interpret them.
So here is a quick 5 step guide to what I believe are the keys to achieving audit trail nirvana:
Step 1: Enable All Audit Trails and Don’t Disable Them Again!
At Sterling the full set of Chromeleon CDS audit trails were turned on immediately after installation, before we generated any data. They record all user actions, instrument communications and system events so we know what has happened at all times.
The IT administrator is the only person that has access to the audit trail control, removing this capability from the laboratory entirely. They received training on data integrity and, together with change control procedures, we can ensure that, without the appropriate authorisation, they could not disable audit trails. However, should the audit trails be disabled in Chromeleon CDS this would be recorded in the Administration Console audit trail, although we have never actually tried this for obvious reasons.
So the audit trails are on and recording what we do but we need to be able to identify who is performing a specific action.
Step 2: Individual Password Protected User Accounts
Everyone at Sterling who is trained and authorised to access Chromeleon CDS has their own password protected account. There are no generic accounts in use, so if, for example, a service engineer requires access, they are given a unique individual account. The password complexity and expiry requirements can be set and managed either in Chromeleon CDS, as we do, or linked to the company Windows LDAP policy.
We use the Administration Console audit trail to record and demonstrate that our accounts are secure and record all administration events – how we do this is probably another topic for another day!
We now know who did what but what about when they did it?
Step 3: Secure Date and Time Settings
The date and time for all computers and servers in the Chromeleon Domain, plus all other laboratory PCs, are controlled by IT. Top level system administration accounts and passwords would be required to change any settings to the date and time including the time zone which are definitely not held within the laboratory.
So we are confident we know when who did what but we also need to know why they did what they did.
Step 4: Explain Why Audit Trail Comments Are Vitally Important
Now the fun starts! The audit trail comment is an analyst’s chance to explain their actions, why they did something. “Why did you do that?” I ask. “It’s my first day!” comes the reply. Not really a valid reason but I’m sure I’m not the only person who has seen some ”unusual” or plainly useless audit trail comments, right?!
Here are some examples. Maybe you have seen something similar?
[caption id="attachment_20496" align="alignnone" width="938"]
For example if a peak needed manual integration to make it consistent with other injections, comments like “ttttttttttttttttttttttt,’’ ”peak integrated” or “manual integration” don’t explain why manual integration was used. Chromeleon CDS already knows what was done (peak manually integrated) and records it, the comment should be to explain why. It is extremely important they take the opportunity to record their reasoning. In fact, inadequate comments draw greater attention to the action in the eyes of an auditor and may be seen as a clear indication you are not following your procedures. They could apply Occam’s Razor or the Duck Test to your data and probably conclude you are “up to no good.”
Occam’s Razor: The more assumptions you have to make, the more unlikely the explanation
Duck Test: If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck
So now you are recording why who did what when but how do you ensure the comments are meaningful and, equally importantly, someone checks or reviews them?
Step 5: Review the Audit Trails
At Sterling, all Chromeleon CDS users (both analysts and reviewers) are trained in how to access and interpret audit trails as part of the routine review process. In fact, our SOP states a review of the audit trails must be carried out during the routine review of CDS data and that the act of signing the data signifies the data and audit trails have been reviewed. It also includes guidance on what to look for to trigger an “alarm bell” such as excessive version numbers of an object.
Report templates are used to summarize version numbers of the sequence entities. The following example template highlights the maximum version number for the injection, chromatogram, instrument and processing methods giving reviewers a clear indication of where to focus.
[caption id="attachment_20495" align="alignnone" width="1448"]
Here you can see that there is only one version of each chromatogram and they are visually verified so there is no need to check the audit trail, but the injections with four versions will require closer inspection.
The QA department also conducts periodic reviews of data and audit trails and, in my experience, once users understand what is recorded and reviewed, and how easy the audit trails are to access, the undesirable nonsensical comments dry up pretty quickly.
There is an excellent white paper on the subject of Chromeleon CDS audit trails by Shaun Quinn:
White Paper 72664: Data Integrity: audit trails with ease of review
Next Steps for Sterling: Privileged Actions and Standardized Comments
Recently we updated to Chromeleon 7.2 CDS Service Release 5 (SR5) and now have some new audit trail “toys” to implement. Previously, enabling audit trail comments required you to make a comment every time you saved a change regardless of the impact of that change.
“Why do I have to add a comment for a view setting change or adding an injection?” Sometimes the act of doing explains the why: “I’ve added an injection because I need to make another injection!” You can quickly reach comment fatigue with users which can then lead to a rise in nonsensical comments.
Now, Privileged Actions enable us to evaluate actions within the CDS and decide how important an action is and whether it needs a comment and/or user authentication – proving who the person doing the action actually is. They allow us the option of adding a list of predefined standard comments for the most frequent actions. They also allow a user to select from their last 20 most recently used comments.
This means for each action we can decide to remove the requirement for a comment, have a default comment with control over whether this can be amended or overwritten, or leave an open field for comments to be added. We can also require the user to authenticate using their logon username and password in addition to, or without the need for, the comment.
There is more information on Privileged Actions in the previously mentioned white paper.
Summary
To summarise this in as quickly as possible I would say:
- In a regulated environment your CDS audit trails will come under auditor scrutiny.
- Turn the audit trails on and leave them on!
- Create a CDS audit trail review SOP and have documented training for everyone who generates or reviews CDS data.
- Encourage the use of clear and sensible audit trail comments.
- Use Privileged Actions to simplify comments and authenticate users performing specific actions.
Finally, the new Chromeleon User Group on LinkedIn is a great forum to share Chromeleon CDS tips, tricks, ideas and experiences – join me there and share your thoughts on audit trails!
Thermo Fisher Scientific
United States
