Auditing

Cloud CMS provides a built-in audit system that automatically logs the activity between users and content through API service methods. The audit system produces an audit trail consisting of audit records that document the access of every user, content object and service method.

Audit records are created automatically if they are enabled for your tenant and for the repository against which the operation occurs.

Auditing is only available for on-premise, Docker customers.

How to Enable Auditing

To enable audit records, the tenant for the currently logged in user must have the auditingEnabled field set to true. Similarly, the repository against which the user interacts must also have the auditingEnabled field set to true.

By default, auditing is turned off for each tenant. This is because auditing is inherently an expensive operation both in terms of computational time as well as disk space. With auditing enabled, some operations may take as much as twice as long to execute.

Audit disk space is non-capped meaning that your allocations and total disk usage will continue to grow over time. The rate of growth is dependent on how frequent users interact with content and content services.

Audit Records

An audit record is an entry that describes a particular interaction between a user and a content object or service. It provides the following fields:

  • scope
  • action
  • principal
  • args
  • method
  • handler
  • return

If the operation involves a data store, the following fields are also filled in:

  • datastoreTypeId
  • datastoreId

If the operation is against a repository branch, the following field is filled in:

  • branchId

In addition, the audit record contains system and doc properties that are consistent with the rest of the product. All objects within Cloud CMS support these properties.

Here is an example of what an audit record might look like:

{
    "_doc" : "87d93692e40edaa9f1ba",
    "action" : "READ",
    "principal" : "joesmith",
    "scope" : "NODE",
    "method" : "read",
    "handler" : "nodeserviceimpl",
    "args" : [{
        "type" : "repository",
        "value" : {
            "platformId" : "9faf2171429839834811",
            "datastoreId" : "21853c33f501377f49e6",
            "datastoreTypeId" : "repository",
            "enableAuditing" : true,
            "enableAuthorities" : true,
            "maxsize" : -1,
            "size" : 0,
            "objectcount" : 0,
            "_system" : {
                "created_on" : {
                    "timestamp" : "04-Apr-2014 18:00:02",
                    "year" : 2014,
                    "month" : 3,
                    "day_of_month" : 4,
                    "hour" : 18,
                    "minute" : 0,
                    "second" : 2,
                    "millisecond" : 246,
                    "ms" : NumberLong("1396634402246")
                },
                "created_by" : "system",
                "created_by_principal_id" : "system",
                "created_by_principal_domain_id" : null,
                "modified_on" : {
                    "timestamp" : "04-Apr-2014 18:00:05",
                    "year" : 2014,
                    "month" : 3,
                    "day_of_month" : 4,
                    "hour" : 18,
                    "minute" : 0,
                    "second" : 5,
                    "millisecond" : 509,
                    "ms" : NumberLong("1396634405509")
                },
                "modified_by" : "system",
                "modified_by_principal_id" : "system",
                "modified_by_principal_domain_id" : null
            },
            "_doc" : "12a970f0f79af0b9b1c3",
            "statisticsDate" : {
                "timestamp" : "01-Jan-1970 00:00:00",
                "year" : 1970,
                "month" : 0,
                "day_of_month" : 1,
                "hour" : 0,
                "minute" : 0,
                "second" : 0,
                "millisecond" : 1,
                "ms" : 1
            },
            "statisticsDirty" : true,
            "title" : "My Content Repository"
        }
    }, {
        "type" : "string",
        "value" : "b67b9df3e739a39cd031"
    }, {
        "type" : "string",
        "value" : "614c4da562daafdd80b6"
    }],
    "datastoreId" : "21853c33f501377f49e6",
    "datastoreTypeId" : "repository",
    "branch" : "67b9df3e739a39cd031b",    
    "_system" : {
        "created_on" : {
            "timestamp" : "04-Apr-2014 18:00:05",
            "year" : 2014,
            "month" : 3,
            "day_of_month" : 4,
            "hour" : 18,
            "minute" : 0,
            "second" : 5,
            "millisecond" : 592,
            "ms" : NumberLong("1396634405592")
        },
        "created_by" : "joesmith",
        "created_by_principal_id" : "39a3b67b9df3e79cd031",
        "created_by_principal_domain_id" : "9cd031b67b9df3e739a3",
        "modified_on" : {
            "timestamp" : "04-Apr-2014 18:00:05",
            "year" : 2014,
            "month" : 3,
            "day_of_month" : 4,
            "hour" : 18,
            "minute" : 0,
            "second" : 5,
            "millisecond" : 592,
            "ms" : NumberLong("1396634405592")
        },
        "modified_by" : "joesmith",
        "modified_by_principal_id" : "39a3b67b9df3e79cd031",
        "modified_by_principal_domain_id" : "9cd031b67b9df3e739a3"
    }    
}

In the audit record above shows that a method called read was invoked on a service called nodeserviceimpl by the user joesmith. It shows the time that the invocation occurred as well as the arguments that were passed into the method.

In this case, the return field is missing which indicates that the method call returned null or empty value. Joe tried to read a node and it wasn't found.

You can also see that the audit engine has written down information about the data store (repository in this case) that was trying to be reached as well as the branch. This information is useful in terms of sifting through the audit records to find operations against data stores for a particular project repository or branch.

Audit Services

If enabled, the audit engine automatically generates audit records for you. No additional setup is required. The audit engine generates audit records for operations against nodes and branches. It groups these audit records into the following action categories:

  • CREATE
  • READ,
  • UPDATE
  • DELETE
  • EXISTS
  • COPY
  • MOVE

Audit Records Logging

Audit records are logged to disk using a configurable Log4j Appender. The default Audit Records appender is a rolling file appender that writes records to disk.

It is defined like this:

<RollingFile name=“AuditRecordsFile” fileName=“${baseDir}/cloudcms-auditrecords.log” filePattern=“${baseDir}/cloudcms-auditrecords/cloudcms-auditrecords-%d{yyyy-MM-dd-HH-mm}-%i.log” append=“false”>
   <PatternLayout>
       <pattern>%msg%n</pattern>
   </PatternLayout>
   <Policies>
       <OnStartupTriggeringPolicy/>
       <TimeBasedTriggeringPolicy interval=“30”/>
       <SizeBasedTriggeringPolicy size=“256 MB”/>
   </Policies>
   <DefaultRolloverStrategy max=“20”/>
</RollingFile>

As such, audit records are written to the ./cloudcms-auditrecords directory (relative to the servlet container root).

The filename for each audit records log file will essentially look like this:

cloudcms-auditrecords-2018-11-15-13-14-1.log

Which is an audit record log file started on Nov 15, 2018 at 1:14pm. 20 rolled over log files are retained before they are cleaned up. The rollover check occurs every 30 minutes or when the current log file reaches 256MB in size (whichever comes first).

S3 Rollover

Cloud CMS supports automatic S3 rollover of audit record files. This generally very desirable since audit record logs tend to grow quite extensively. With S3 Rollover enabled, log files are rolled over on local disk but are also written up to a common S3 bucket. This provides a good way for your third-party tooling to pick up these files and work with them.

To use S3 rollover, you should add a file called log4j2-custom.xml to your classes directory.

The log4j2-custom.xml should look something like this:

<?xml version=“1.0” encoding=“UTF-8"?>

<Configuration status=“info” packages=“org.gitana.platform.services.log”>

   <Properties>
       <Property name=“AUDITRECORDS_S3_ACCESS_KEY”></Property>
       <Property name=“AUDITRECORDS_S3_SECRET_KEY”></Property>
       <Property name=“AUDITRECORDS_S3_REGION”></Property>
       <Property name=“AUDITRECORDS_S3_BUCKET”></Property>
       <Property name=“AUDITRECORDS_S3_PREFIX”></Property>
   </Properties>
   <Appenders>

       <!-- override so that supports S3 rollover -->
       <RollingFile name=“AuditRecordsFile” fileName=“${baseDir}/cloudcms-auditrecords.log” filePattern=“${baseDir}/cloudcms-auditrecords/cloudcms-auditrecords-%d{yyyy-MM-dd-HH-mm}-%i.log” append=“false”>
           <PatternLayout>
               <pattern>%msg%n</pattern>
           </PatternLayout>
           <Policies>
               <OnStartupTriggeringPolicy/>
               <TimeBasedTriggeringPolicy interval=“30"/>
               <SizeBasedTriggeringPolicy size=“256 MB”/>
           </Policies>
           <DefaultRolloverStrategy max=“20"/>
           <S3RolloverStrategy accessKey=“${AUDITRECORDS_S3_ACCESS_KEY}” secretKey=“${AUDITRECORDS_S3_SECRET_KEY}” region=“${AUDITRECORDS_S3_REGION}” bucket=“${AUDITRECORDS_S3_BUCKET}” prefix=“${AUDITRECORDS_S3_PREFIX}” />
       </RollingFile>

   </Appenders>

   <Loggers>

       <!-- override “auditrecords” logger to use our rollingfile appender with support for S3 rollover -->
       <Logger name=“auditRecords” additivity=“false” level=“INFO”>
           <AppenderRef ref=“AuditRecordsFile”/>
       </Logger>

   </Loggers>

</Configuration>

You will need to fill in the values for:

  • AUDITRECORDS_S3_ACCESS_KEY
  • AUDITRECORDS_S3_SECRET_KEY
  • AUDITRECORDS_S3_REGION
  • AUDITRECORDS_S3_BUCKET
  • AUDITRECORDS_S3_PREFIX

And you will further want to adjust the rollover policy to match the needs of your usage. The audit system is intentionally verbose (that is the whole point of it) and so your Audit log files should grow quickly.