Oak offline compaction instructions.md

Note that offline compaction requires a long duration of downtime from 30 minutes to 7 hours (and in extreme cases more time).

Offline Oak compaction with debug logging:

1. Stop Oak / AEM
2. Install the latest Oak hotfix to your AEM instance:

• AEM 6.0 - https://helpx.adobe.com/experience-manager/kb/aem6-available-hotfixes.html
• AEM 6.1 - https://helpx.adobe.com/experience-manager/kb/aem61-available-hotfixes.html
• AEM 6.2 - https://helpx.adobe.com/experience-manager/kb/aem62-available-hotfixes.html
• Later - https://helpx.adobe.com/experience-manager/aem-releases-updates.html

3. Download the oak-run version matching your oak version: https://repo1.maven.org/maven2/org/apache/jackrabbit/oak-run/

• AEM6.0 - use oak-run 1.0.x - if you have Oak 1.0.11 or earlier installed in AEM then use oak-run 1.0.11 otherwise use the version that matches the installed on in AEM.
• AEM6.1 - use latest oak-run 1.2.x version
• AEM6.2 - use latest oak-run 1.4.x version
• AEM6.3 - use latest oak-run 1.6.x version
• AEM6.4 - use latest oak-run 1.8.x version
• AEM6.5 - use latest oak-run 1.10.x version

4. Upload the oak-run jar file to your server
5. Create a file logback-compaction.xml (on the server in the same folder as the oak-run jar)

<configuration>
  <appender name="STDERR" class="ch.qos.logback.core.ConsoleAppender">
    <target>System.out</target>
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>
  <logger name="org.apache.jackrabbit.oak.plugins.segment.Compactor" level="INFO"/>
  <root level="warn">
    <appender-ref ref="STDERR" />
  </root>
</configuration>

Note: Debug logging can be enabled by setting level="DEBUG" 6. Run this command from the same folder as logback-compaction.xml (change the paths and jar file name to match your system’s paths, and tune -Xmx10g to not exceed your server's memory while leaving room for the OS memory):

Linux, Unix, Mac:

  nohup sudo -u crx nohup /usr/java/latest/bin/java -server -Xmx30g -Dtar.memoryMapped=true -Doak.compaction.eagerFlush=true -Dcompaction-progress-log=5000000 -Dupdate.limit=2500000 -Dlogback.configurationFile=logback-compaction.xml -Dcompress-interval=150000000 -jar oak-run-*.jar compact /path/to/segmentstore >> oak-tar-compact-a0.log 2>>oak-tar-error-a0.log &

Windows (note that memoryMapped setting is removed):

  java -server -Xmx30g -Dtar.memoryMapped=false -Doak.compaction.eagerFlush=true -Dcompaction-progress-log=5000000 -Dupdate.limit=2500000 -Dlogback.configurationFile=logback-compaction.xml -Dcompress-interval=150000000 -jar oak-run-*.jar compact C:\path\to\segmentstore >> oak-tar-compact-a0.log

7. Once the process completes then start Oak / AEM

@brace01 try the settings I added with the latest oak-run available. It might help.

-Doak.compaction.eagerFlush=true is a new flag and it helps avoid OutOfMemoryError cases. You would need to download the latest version of oak-run from here http://repo1.maven.org/maven2/org/apache/jackrabbit/oak-run/