Skip to content

Instantly share code, notes, and snippets.

@akrivitzky

akrivitzky/AEM6.x Migration Instructions.md

Forked from andrewmkhoury/AEM6.x Migration Instructions.md
Last active April 6, 2017 15:52
Show Gist options
  • Save akrivitzky/5b59cee567062149713e88340e8135ec to your computer and use it in GitHub Desktop.
Save akrivitzky/5b59cee567062149713e88340e8135ec to your computer and use it in GitHub Desktop.
Download ZIP
AEM6.x Migration
Raw
AEM6.x Migration Instructions.md

AEM6.x Migration

  • How to Rebuild a Pre-Existing AEM+Mongo Cluster
  • _or migrate from Tar to MongoDB or MongoDB to Tar"
  • or migrate from 5.6.1 or 6.x to 6.x via data migration instead of jar upgrade
  1. (Mongo to Mongo only) Remove one replica from the replica set and delete/recreate the db
    1. Remove the replica node from the set: http://docs.mongodb.org/master/tutorial/remove-replica-set-member/
    2. Validate that no other nodes in the set consider that node to be part of the set anymore. Log into each node in the cluster via mongo shell and run rs.status() to validate that the node that was taken out is not in the cluster any longer.
    3. Drop the aem database on that node http://docs.mongodb.org/manual/reference/command/dropDatabase/
    4. Re-add the database with the correct user permissions (do not add the mongo instance back to the replica set)
    5. Remove one of the AEM cluster nodes from the load balancer or dispatcher config (whichever config is managing the active cluster nodes)
    6. Stop the AEM instance on the inactive node
    7. On the same server install a fresh AEM6.x instance pointed at the new emptied mongo database and install hotfixes (for AEM6.0 install Oak 1.0.27 or later, AEM6.1 install Oak 1.2.11 or later)
    8. Skip step 2 below
  2. Install a fresh AEM6.x instance and all the recent recommended hotfixes

  1. Optimize the instance using the configurations here - https://helpx.adobe.com/experience-manager/kb/performance-tuning-tips.html

  2. Prep the systems for data migration

    1. Prep source system if migrating from CQ5.x version to AEM6.x:
      1. Run consistency check and fix: https://helpx.adobe.com/experience-manager/kb/RepositoryInconsistency.html
      2. Remove Same Name Sibling nodes: https://helpx.adobe.com/experience-manager/kb/find-sns-nodes.html
    2. Prep the destination 6.x system:
      1. Apply any customizations to the AEM start script and restart AEM
      2. Disable workflow launchers for DAM by going to the launcher tab under workflow console url: http://host:port/libs/cq/workflow/content/console.html
        • Disable all launchers that use "DAM Update Asset" and "DAM Metadata Writeback” workflow models.
      3. Disable OSGi components to reduce overhead. 
        com.day.cq.dam.core.impl.event.DamEventAuditListener
com.day.cq.replication.audit.ReplicationEventListener
com.day.cq.wcm.core.impl.event.PageEventAuditListener
com.day.cq.tagging.impl.TagValidatingEventListener
com.day.cq.dam.core.impl.DamChangeEventListener
    3. (6.x to 6.x migration only) On the old/source instance, package out-of-the-box indexes that were disabled or modified and install those. Do not install custom indexes as it is faster to install them after. The objective here is to get any disabled indexes disabled.
    4. Go to CRXDE on the destination instance, browse to /oak:index/lucene and add a String[] property excludedPaths 
      excludedPaths=[/var, /jcr:system, /etc/workflow/instances]
    5. Perform repository maintenance
    6. Monitor the logs to validate when Revision GC completes successfully.
    grep VersionGarbage error*

  3. Migrate the data from the old AEM cluster node to the new one

  4. Migrate users and groups (If users were not impored automatically via LDAP) Package users and groups (2 separate packages) on the old system (excluding admin and anonymous OOTB users)

    1. Go to CRXDE lite app /crx/de/index.jsp and log in as admin user (on the old system)

    2. Go to "Tools" => "Query"

    3. In the bottom "Query" box enter this query to find the admin user: /jcr:root/home/users//element(*,rep:User)[@rep:principalName="admin"]

    4. Click "Execute" and copy the path of the admin user node in the results to a text file

    5. Repeat step 3 with a query for anonymous user: /jcr:root/home/users//element(*,rep:User)[@rep:principalName="anonymous"]

    6. Click "Execute" and copy the path of the anonymous user node in the results to a text file (so now you should have two paths, one for "admin" and one for "anonymous")

      For example:

      • /home/users/Q/QY5FIMXeQIbGpwZtQ3Dv – admin user on the system where I am creating the package
      • /home/users/K/Kj1406Qo9IDODc_nk5Ib – anonymous user on the system where I am creating the package

    7. Go to the "Package Manager", http://host:port/crx/packmgr/index.jsp, and log in as admin

    8. Create a package "users"

    9. Add a filter to the package config for /home/users with these exclude rules (on the /home/users filter):

      • exclude /home/users/.*/.tokens
      • exclude /home/users/Q/QY5FIMXeQIbGpwZtQ3Dv
      • exclude /home/users/K/Kj1406Qo9IDODc_nk5Ib
      • exclude /home/users/a/admin
      • exclude /home/users/a/anonymous
      • exclude /home/users/system
      • exclude /home/users/geometrixx
      • exclude /home/users/media
      • exclude /home/users/projects
      • exclude /home/users/mac

    10. Build the package

    11. Download the package

    12. Unzip the package zip file on your computer

    13. Open the file META-INF/vault/filter.xml in a text editor

    14. Add mode="merge" to the <filter ...> tag, for example:

    <?xml version="1.0" encoding="UTF-8"?>
<workspaceFilter version="1.0">
  <filter root="/home/users" mode="merge">
    <exclude pattern="/home/users/.*/.tokens"/>
    <exclude pattern="/home/users/Q/QY5FIMXeQIbGpwZtQ3Dv"/>
    <exclude pattern="/home/users/K/Kj1406Qo9IDODc_nk5Ib"/>
    <exclude pattern="/home/users/a/admin"/>
    <exclude pattern="/home/users/a/anonymous"/>
    <exclude pattern="/home/users/system"/>
    <exclude pattern="/home/users/geometrixx"/>
    <exclude pattern="/home/users/media"/>
    <exclude pattern="/home/users/projects"/>
    <exclude pattern="/home/users/mac"/>
  </filter>
</workspaceFilter>
    1. Re-zip the modified package contents so it includes the change
    2. Create a "groups" package that contains a filter rule /home/groups
    3. Repeat steps 11-14 for the groups package
    4. (Upgrade only) If performing migration to newer AEM version, then install a fresh local AEM instance of the old version(with nosamplecontent), and install the users package, and then the groups package there. Then, perform an in-place upgrade to the new version on that instance. This converts the users to the new Oak representation. After the in-place upgrade, repackage the users again to port them to your intended upgraded instance. Do the same for the user groups.
    5. Install the users package on the new system
    6. Install the groups package on the new system

  5. Install the application and other required files

    1. Install the application including all configurations
    2. Package the replication agents from production, install the package on the new instance and disable the agents
    3. Package any extra items like /etc/designs and other items
    4. Validate any backend and third party integrations

  6. Install custom lucene property indexes, make sure all indexes have been applied and fully indexed

  7. Do testing (functional and load testing) to validate that the application works perfectly

  8. Create package using a search for pages that changed since the copy was done

    1. Install either one of these tools for packaging the changed content:
    2. Create package using a search for tags that changed since the copy was done. Use this query for tags (but change the date):
    //element(*, cq:Tag)[@jcr:created > xs:dateTime('2015-09-16T00:00:00.000-05:00')]
    1. Create package using a search for assets that changed since the copy was done. Use this query for assets (but change the date):
    //element(*, dam:AssetContent)[@jcr:lastModified > xs:dateTime('2015-09-16T00:00:00.000-05:00')]
    1. Create package using a search for pages that changed since the copy was done. Use this query for pages (but change the date):
    //element(*,cq:PageContent)[@cq:lastModified >= xs:dateTime('2015-09-16T00:00:00.000-05:00')]
    1. Install all the packages to the AEM instance (install the tags first, then the assets then pages)
    2. Account for any other data that might of changed (for example, new users added to the system or other content)

  9. Remove the disabling of OSGi component config that we did in step 3 above to re-allow audit trails and tag validation.

  10. (Mongo to Mongo only) Remove another replica from the original mongo cluster, wipe out the database there and instead add it to the new replica from step A.

    1. Remove the replica node from the set: http://docs.mongodb.org/master/tutorial/remove-replica-set-member
    2. Validate that no other nodes in the set consider that node to be part of the set anymore
    3. Drop the aem database on that node http://docs.mongodb.org/manual/reference/command/dropDatabase/
    4. Add the node as a replica http://docs.mongodb.org/master/tutorial/expand-replica-set/

  11. Do testing (functional and load testing) to validate that the application works perfectly

  12. Validate that the environment is fully in sync and, if needed, repeat step D above with a later date / time as needed

  13. Perform repository maintenance

  14. Take a short outage window and do the cut over to the environment by simply clearing the dispatcher cache and pointing it to the new environment. If no dispatcher is used, then point the load balancer to the new AEM environment instead of the old.

Alternative upgrade approach for CQ5.x to AEM6.x upgrade using Oak level crx2oak tool documented here: http://dev.day.com/content/ddc/en/gems/deep-dive-into-aem-upgrade-process.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment