Skip to content

Instantly share code, notes, and snippets.

@andrewmkhoury

andrewmkhoury/AEM6.x Migration Instructions.md

Last active June 20, 2023 15:29
Show Gist options
  • Save andrewmkhoury/280c6d0ac1db03b57c58 to your computer and use it in GitHub Desktop.
Save andrewmkhoury/280c6d0ac1db03b57c58 to your computer and use it in GitHub Desktop.
Download ZIP
AEM6.x Migration
Raw
AEM6.x Migration Instructions.md

AEM6.x Migration

  • How to Rebuild a Pre-Existing AEM+Mongo Cluster
  • or migrate from Tar to MongoDB or MongoDB to Tar"
  • or migrate from 5.6.1 or 6.x to 6.x via data migration instead of jar upgrade
  1. (Mongo to Mongo only) Remove one replica from the replica set and delete/recreate the db
    1. Remove the replica node from the set: http://docs.mongodb.org/master/tutorial/remove-replica-set-member/
    2. Validate that no other nodes in the set consider that node to be part of the set anymore. Log into each node in the cluster via mongo shell and run rs.status() to validate that the node that was taken out is not in the cluster any longer.
    3. Drop the aem database on that node http://docs.mongodb.org/manual/reference/command/dropDatabase/
    4. Re-add the database with the correct user permissions (do not add the mongo instance back to the replica set)
    5. Remove one of the AEM cluster nodes from the load balancer or dispatcher config (whichever config is managing the active cluster nodes)
    6. Stop the AEM instance on the inactive node
    7. On the same server install a fresh AEM6.x instance pointed at the new emptied mongo database and install hotfixes (for AEM6.0 install Oak 1.0.27 or later, AEM6.1 install Oak 1.2.11 or later)
    8. Skip step 2 below
  2. Install a fresh AEM6.x instance and all the recent recommended hotfixes
  1. Optimize the instance using the configurations here - https://helpx.adobe.com/experience-manager/kb/performance-tuning-tips.html
  2. Prep the systems for data migration
    1. Prep source system if migrating from CQ5.x version to AEM6.x:
      1. Run consistency check and fix: https://helpx.adobe.com/experience-manager/kb/RepositoryInconsistency.html
      2. Remove Same Name Sibling nodes: https://helpx.adobe.com/experience-manager/kb/find-sns-nodes.html
    2. Prep the destination 6.x system:
      1. Apply any customizations to the AEM start script and restart AEM
      2. Disable workflow launchers for DAM by going to the launcher tab under workflow console url: http://host:port/libs/cq/workflow/content/console.html
        • Disable all launchers that use "DAM Update Asset" and "DAM Metadata Writeback” workflow models.
      3. Disable OSGi components to reduce overhead. 
        com.day.cq.dam.core.impl.event.DamEventAuditListener
com.day.cq.replication.audit.ReplicationEventListener
com.day.cq.wcm.core.impl.event.PageEventAuditListener
com.day.cq.tagging.impl.TagValidatingEventListener
com.day.cq.dam.core.impl.DamChangeEventListener
    3. (6.x to 6.x migration only) On the old/source instance, package out-of-the-box indexes that were disabled or modified and install those. Do not install custom indexes as it is faster to install them after. The objective here is to get any disabled indexes disabled.
    4. Go to CRXDE on the destination instance, browse to /oak:index/lucene and add a String[] property excludedPaths 
      excludedPaths=[/var, /jcr:system, /etc/workflow/instances]
    5. Perform repository maintenance
    6. Monitor the logs to validate when Revision GC completes successfully.
    grep VersionGarbage error*
  3. Migrate the data from the old AEM cluster node to the new one
  4. Migrate users and groups (If users were not imported automatically via LDAP)
  5. Install the application and other required files
    1. Install the application including all configurations
    2. Package the replication agents from production, install the package on the new instance and disable the agents
    3. Package any extra items like /etc/designs and other items
    4. Validate any backend and third party integrations
  6. Install custom lucene property indexes, make sure all indexes have been applied and fully indexed
  7. Do testing (functional and load testing) to validate that the application works perfectly
  8. Create package using a search for pages that changed since the copy was done
    1. Install either one of these tools for packaging the changed content:
    2. Create package using a search for tags that changed since the copy was done. Use this query for tags (but change the date):
    //element(*, cq:Tag)[@jcr:created > xs:dateTime('2015-09-16T00:00:00.000-05:00')]
    1. Create package using a search for assets that changed since the copy was done. Use this query for assets (but change the date):
    //element(*, dam:AssetContent)[@jcr:lastModified > xs:dateTime('2015-09-16T00:00:00.000-05:00')]
    1. Create package using a search for pages that changed since the copy was done. Use this query for pages (but change the date):
    //element(*,cq:PageContent)[@cq:lastModified >= xs:dateTime('2015-09-16T00:00:00.000-05:00')]
    1. Install all the packages to the AEM instance (install the tags first, then the assets then pages)
    2. Account for any other data that might of changed (for example, new users added to the system or other content)
  9. Remove the disabling of OSGi component config that we did in step 3 above to re-allow audit trails and tag validation.
  10. (Mongo to Mongo only) Remove another replica from the original mongo cluster, wipe out the database there and instead add it to the new replica from step A.
    1. Remove the replica node from the set: http://docs.mongodb.org/master/tutorial/remove-replica-set-member
    2. Validate that no other nodes in the set consider that node to be part of the set anymore
    3. Drop the aem database on that node http://docs.mongodb.org/manual/reference/command/dropDatabase/
    4. Add the node as a replica http://docs.mongodb.org/master/tutorial/expand-replica-set/
  11. Do testing (functional and load testing) to validate that the application works perfectly
  12. Validate that the environment is fully in sync and, if needed, repeat step D above with a later date / time as needed
  13. Perform repository maintenance
  14. Take a short outage window and do the cut over to the environment by simply clearing the dispatcher cache and pointing it to the new environment. If no dispatcher is used, then point the load balancer to the new AEM environment instead of the old.

Alternative upgrade approach for CQ5.x to AEM6.x upgrade using Oak level crx2oak tool documented here: http://dev.day.com/content/ddc/en/gems/deep-dive-into-aem-upgrade-process.html

WARNING: the crx2oak (aka oak-upgrade) tool will NOT migrate namespaces that are required for the nodes and properties.

@anupdugar
Copy link

Regarding "Disable OSGi components to reduce overhead"
The link (http://adobe-consulting-services.github.io/acs-aem-commons/features/osgi-disabler.html) from acs commons advises that we need to mention pid of the components.

/apps/mysite/config/com.adobe.acs.commons.util.impl.ComponentDisabler.xml

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:cq="http://www.day.com/jcr/cq/1.0"
    xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0"
    jcr:primaryType="sling:OsgiConfig"
    components="[pid1,pid2]"
    />

components is an array of the OSGi component PIDs to disable.

Is this correct? Wouldn't pids differ from instance to instance? Or just adding component names as array would work?

@andrewmkhoury
Copy link
Author

Hi @anupdugar The PIDs will match from instance to instance as long as the component you are disabling is not a factory instance. You can find the pid by clicking on the component in the /system/console/components UI. Make sure to re-enable the components after the migration.
By the way, you can also configure the list of OSGi components to disable via the /system/console/configMgr/com.adobe.acs.commons.util.impl.ComponentDisabler UI. Then the config would get saved in the repo under /apps/system/config/.

@anurag-aem
Copy link

Great post

@andrewmkhoury
Copy link
Author

Thanks @anurag-aem If you are using this documentation then I just updated it with some extra details for migration from CQ5.x.

@alexkli
Copy link

alexkli commented Feb 24, 2017

Tag validation can be disabled in http://localhost:4502/system/console/configMgr/com.day.cq.tagging.impl.JcrTagManagerFactoryImpl (instead of disabling the TagValidatingEventListener via osgi-disable).

@akrivitzky
Copy link

Hi Andrew. Thanks for the instructions. I was confused by the wording in Step 17 of Migrating Users and Groups. It led me into the weeds for a bit. Once I realized my misinterpretation, I forked this, clarified the instructions, and improved the formatting. I don't see how you can do a pull request with Gist, but you can see it here:
https://gist.github.com/akrivitzky/5b59cee567062149713e88340e8135ec
You might want to review and update yours. Thanks!

@andrewmkhoury
Copy link
Author

@alexkli and @akrivitzky thanks for the updates. I have updated the gist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment