Skip to content

Instantly share code, notes, and snippets.

@CyberSecutor
Last active November 12, 2025 10:17
Show Gist options
  • Save CyberSecutor/c5f8f85ece76d2ebe087036953515cbe to your computer and use it in GitHub Desktop.
Save CyberSecutor/c5f8f85ece76d2ebe087036953515cbe to your computer and use it in GitHub Desktop.
Magento DB Dump, sanitize and sync media

Dump a sanitized version of the database

Excludes and stips are taken fron the asset snc project: https://bitbucket.youweagency.com/projects/YWEDO/repos/asset-sync/browse/scripts/magento2/db

More excludes can be found in the projects source. They can be specified in the project specific file: app/etc/n98-magerun2.yaml

vendor/bin/n98-magerun2 db:dump \
    --no-tablespaces   \
    -t suffix   \
    -c gzip  \
    --exclude=" *__temp* admin* authorization* buckaroo_magento2_analytics buckaroo_magento2_certificate buckaroo_magento2_invoice buckaroo_magento2_second_chance fastly_modly_manifests integration oauth* tfa_* xtento_orderexport_destination xtento_orderexport_log xtento_orderexport_profile xtento_orderexport_profile_history xtento_xtcore_config_data " \
    --strip="@2fa @aggregated @asset-sync-strip-tables @oauth @search @stripped @trade *_log *_replica *_tmp abeta_login_token admin* adobe_user_profile adyen_invoice adyen_notification adyen_order_payment adyen_payment_response akeneo_connector_family_attribute_relations akeneo_connector_import_log akeneo_connector_import_log* amasty_acart_blacklist amasty_acart_history amasty_acart_quote_email amasty_acart_rule_quote amasty_advanced_review_reminder amasty_advanced_review_reminder_product amasty_advanced_review_unsubscribe amasty_advanced_review_vote amasty_amfollowup_history amasty_amrules_cache_queue amasty_conditions_quote amasty_fpc_activity amasty_fpc_flushes_log amasty_fpc_reports amazon_customer amazon_quote bluesnap_vaulted_shopper buckaroo_magento2_analytics buckaroo_magento2_certificate catalog_compare_item catalog_data_exporter_categories catalog_data_exporter_categories_cl catalog_data_exporter_product_attributes catalog_data_exporter_product_attributes_cl catalog_data_exporter_product_overrides catalog_data_exporter_product_overrides_cl catalog_data_exporter_product_variants catalog_data_exporter_product_variants_cl clang_clang_calllog company company_advanced_customer_entity cron_schedule dealer4dealer_invoice dealer4dealer_invoice_item dealer4dealer_order dealer4dealer_orderaddress dealer4dealer_orderinvoicerelation dealer4dealer_orderitem dealer4dealer_shipment dealer4dealer_shipmentitem dealer4dealer_substituteorders_attachment dpd_shipment_label dpdconnect_shipping_label email_abandoned_cart email_catalog email_contact email_contact_consent email_coupon_attribute email_failed_auth email_importer email_order email_review email_rules email_wishlist experius_emailcatcher fastly_statistics firebear_export_history firebear_export_jobs_event firebear_import_history firebear_importexport_importdata gift_message import_history importexport_importdata inventory_reservation klarna_* login_as_customer login_as_customer_assistance_allowed magento_acknowledged_bulk magento_bulk magento_giftcardaccount magento_giftcardaccount_amount magento_giftcardaccount_history magento_giftcardaccount_pool magento_giftregistry_data magento_giftregistry_entity magento_giftregistry_item magento_giftregistry_item_option magento_giftregistry_label magento_giftregistry_person magento_invitation magento_invitation_status_history magento_invitation_track magento_logging_event magento_logging_event_changes magento_login_as_customer_log magento_operation magento_scheduled_operations mageplaza_reviewreminder_logs mageplaza_smtp_abandonedcart mageplaza_smtp_log mailchimp_errors mailchimp_interest_group mailchimp_stores mailchimp_sync_batches mailchimp_sync_ecommerce mailchimp_webhook_request mediact_googleadminlogin_google_login_attempt mm_webforms_result mm_webforms_result_value mollie_analytics_analytics multisafepay_tokenization password_reset_request_event plumrocket_newsletterpopup_history queue_message queue_message_status review_detail sendfriend_log shopreview_kiyoh_review support_report temp_akeneo* tig_buckaroo_certificate tig_postnl_order tig_postnl_shipment tig_postnl_shipment_barcode tig_postnl_shipment_label tig_vendiro_order tig_vendiro_stock tig_vendiro_track tmp_akeneo* ui_bookmark vat_validation webapi_logs webforms_files webforms_results webforms_results_values yotpo_order_status_history yotpo_sync " \
    var/backups/${HOSTNAME%%.*}-stripped-excluded.sql

Sanitize after import

Delete all config values that are know to be environment specific.

DELETE FROM core_config_data WHERE `path` IN (
    "admin/url/custom",
    "admin/url/custom_path",
    "admin/url/use_custom",
    "admin/url/use_custom_path",
    "agresso_reports/settings/password",
    "aitoc_addressautocomplete/google_places/api_key",
    "akeneo_connector/akeneo_api/base_url",
    "akeneo_connector/akeneo_api/client_id",
    "akeneo_connector/akeneo_api/client_secret",
    "akeneo_connector/akeneo_api/password",
    "akeneo_connector/akeneo_api/username",
    "analytics/general/token",
    "buckaroo_magento2/account/certificate_file",
    "buckaroo_magento2/account/merchant_key",
    "buckaroo_magento2/account/secret_key",
    "carriers/dhl/id",
    "carriers/dhl/password",
    "carriers/dpdpickup/google_maps_api_client",
    "carriers/dpdpickup/google_maps_api_server",
    "carriers/fedex/account",
    "carriers/fedex/key",
    "carriers/fedex/password",
    "carriers/usps/password",
    "carriers/usps/userid",
    "catalog/product_video/youtube_api_key",
    "cataloginventory/source_selection_distance_based_google/api_key",
    "checkout/address_validation_pro6pp/api",
    "cms/pagebuilder/google_maps_api_key",
    "company/general/default_standard_sales_representative",
    "connector/api/endpoint",
    "connector_api_credentials/api/endpoint",
    "connector_api_credentials/api/password",
    "connector_api_credentials/api/username",
    "connector_configuration/tracking/tracking_profile_id",
    "connector_dynamic_content/external_dynamic_content_urls/passcode",
    "currency/fixerio/api_key",
    "dpdshipping/accesstoken_created",
    "dpdshipping/accesstoken_depot",
    "dpdshipping/account_settings/password",
    "dpdshipping/account_settings/username",
    "ec_config/global_setting/client_key",
    "ec_config/global_setting/secret_key",
    "ec_config/global_setting/shop_key",
    "emailcatcher/general/enabled",
    "google/adwords/conversion_id",
    "google/analytics/container_id",
    "google/gtag/analytics4/container_id",
    "googletagmanager/general/account",
    "googleuniversalanalytics/general/account",
    "ingenico_connection/live/password",
    "ingenico_connection/live/pspid",
    "ingenico_connection/live/signature",
    "ingenico_connection/live/user",
    "ingenico_connection/mode/mode",
    "ingenico_connection/test/password",
    "ingenico_connection/test/pspid",
    "ingenico_connection/test/signature",
    "ingenico_connection/test/user",
    "jtw_hardening/general/secret_salt",
    "klaviyo_reclaim_general/general/private_api_key",
    "klaviyo_reclaim_general/general/public_api_key",
    "magmodules_channable/general/token",
    "magmodules_channable_marketplace/item/webhook",
    "magmodules_sooqr/general/token",
    "magmodules_thefeedbackcompany/api/client_id",
    "magmodules_thefeedbackcompany/api/client_secret",
    "magmodules_thefeedbackcompany/api/client_token",
    "mailchimp/general/monkeylist",
    "mailchimp/general/monkeystore",
    "mct_relevant_online/general/authorization_token",
    "mediact_google_admin_login/general/allowed_addresses",
    "mediact_google_admin_login/general/client_id",
    "mediact_google_admin_login/general/client_secret",
    "mediact_googleoptimize/general/container_id",
    "mfblog/general/key",
    "mirakl_api/general/api_key",
    "mirakl_api/general/api_url",
    "msp_securitysuite_recaptcha/general/private_key",
    "msp_securitysuite_recaptcha/general/public_key",
    "msp_securitysuite_twofactorauth/duo/application_key",
    "mtemail/license/key",
    "multisafepay/connect/live_api_key",
    "multisafepay/connect/test_api_key",
    "newrelicreporting/general/api",
    "newrelicreporting/general/insights_insert_key",
    "payment/adyen_abstract/api_key_live",
    "payment/adyen_abstract/api_key_test",
    "payment/adyen_abstract/client_key_live",
    "payment/adyen_abstract/client_key_test",
    "payment/adyen_abstract/live_endpoint_url_prefix",
    "payment/adyen_abstract/notification_hmac_key_live",
    "payment/adyen_abstract/notification_hmac_key_test",
    "payment/amazon_payments/simplepath/privatekey",
    "payment/braintree/private_key",
    "payment/braintree/public_key",
    "payment/braintree/sandbox_private_key",
    "payment/braintree/sandbox_public_key",
    "payment/eway/live_api_password",
    "payment/eway/sandbox_api_password",
    "payment/mollie_general/apikey_live",
    "payment/mollie_general/profileid",
    "payment/paynl/apitoken",
    "payment/paynl/serviceid",
    "payment/paynl/tokencode",
    "payment/worldpay/auth_password",
    "payment/worldpay/response_password",
    "paypal/fetch_reports/ftp_password",
    "paypal/wpp/api_password",
    "paypal/wpp/api_signature",
    "paypal/wpp/api_username",
    "recaptcha_backend/type_for/user_forgot_password",
    "recaptcha_backend/type_for/user_login",
    "recaptcha_backend/type_invisible/private_key",
    "recaptcha_backend/type_invisible/public_key",
    "recaptcha_backend/type_recaptcha/private_key",
    "recaptcha_backend/type_recaptcha/public_key",
    "recaptcha_backend/type_recaptcha_v3/private_key",
    "recaptcha_backend/type_recaptcha_v3/public_key",
    "recaptcha_frontend/type_invisible/private_key",
    "recaptcha_frontend/type_invisible/public_key",
    "recaptcha_frontend/type_recaptcha/private_key",
    "recaptcha_frontend/type_recaptcha/public_key",
    "recaptcha_frontend/type_recaptcha_v3/private_key",
    "recaptcha_frontend/type_recaptcha_v3/public_key",
    "robinhq/api/key",
    "robinhq/api/secret",
    "robinhq/api/server_key",
    "robinhq/api/server_secret",
    "sales_email/creditmemo/copy_to",
    "sales_email/creditmemo_comment/copy_to",
    "sales_email/invoice/copy_to",
    "sales_email/invoice_comment/copy_to",
    "sales_email/magento_rma/copy_to",
    "sales_email/magento_rma_auth/copy_to",
    "sales_email/magento_rma_comment/copy_to",
    "sales_email/magento_rma_customer_comment/copy_to",
    "sales_email/order/copy_to",
    "sales_email/order_comment/copy_to",
    "sales_email/shipment/copy_to",
    "sales_email/shipment_comment/copy_to",
    "services_connector/services_connector_integration/production_api_key",
    "services_connector/services_connector_integration/production_private_key",
    "services_connector/services_connector_integration/sandbox_api_key",
    "services_connector/services_connector_integration/sandbox_private_key",
    "services_connector/services_id/environment_id",
    "services_connector/services_id/project_id",
    "sherpaconnect2/general/privatekey",
    "smtp/configuration_option/password",
    "smtp/configuration_option/username",
    "smtp/module/product_key",
    "swat/jwks",
    "swat/rsa_keypair",
    "system/cron_job_manager/email_recipients",
    "system/full_page_cache/fastly/fastly_api_key",
    "system/full_page_cache/fastly/fastly_blocking/block_by_acl",
    "system/full_page_cache/fastly/fastly_blocking/blocking_type",
    "system/full_page_cache/fastly/fastly_ga_cid",
    "system/full_page_cache/fastly/fastly_service_id",
    "system/smtp/host",
    "tig_buckaroo/account/merchant_key",
    "tig_buckaroo/account/secret_key",
    "tig_postnl/generalconfiguration_extension_status/api_key",
    "tig_postnl/generalconfiguration_extension_status/bls_code",
    "tig_postnl/generalconfiguration_extension_status/customer_code",
    "tig_postnl/generalconfiguration_extension_status/customer_number",
    "transactional_emails/ddg_transactional/password",
    "transactional_emails/ddg_transactional/username",
    "trustedshops_trustedshops/general/tsid",
    "tweakwise/export/api_import_url",
    "tweakwise/export/feed_key",
    "tweakwise/general/authentication_key",
    "twofactorauth/duo/application_key",
    "web/cookie/cookie_domain",
    "web/secure/base_link_url",
    "web/secure/base_media_url",
    "web/secure/base_static_url",
    "web/secure/base_url",
    "web/unsecure/base_link_url",
    "web/unsecure/base_media_url",
    "web/unsecure/base_static_url",
    "web/unsecure/base_url",
    "webforms/captcha/private_key",
    "webforms/captcha/public_key",
    "webforms/license/serial",
    "yotpo/settings/app_key",
    "yotpo/settings/secret",
    "youwe_erpconnector/api/environment",
    "youwe_erpconnector_environments/acceptance/password",
    "youwe_erpconnector_environments/acceptance/username",
    "youwe_erpconnector_environments/production/password",
    "youwe_erpconnector_environments/production/username",
    "youwe_erpconnector_environments/test/password",
    "youwe_erpconnector_environments/test/username",
    "youwe_onelogin/settings/enable",
    "youwe_onelogin/settings/issuer_url",
    "youwe_onelogin/settings/roles",
    "youwe_onelogin/settings/saml_acs_endpoint",
    "youwe_onelogin/settings/saml_audience",
    "youwe_onelogin/settings/x509_certificate_fingerprint"
);

This snapshot can cause trouble when a db is used on other systems

See magento/magento2#16171

DELETE FROM flag WHERE flag_code = 'system_config_snapshot';

Delete all encrypted values as those should likely not be copied to other systems.

This regex should match values encrypted by Magento\Framework\Encryption::encrypt().

DELETE FROM core_config_data WHERE value REGEXP '^[0-9]:[0-9]:[a-zA-Z0-9+/]+[=]{0,}$';

Syncronize media

rsync -apv -r --links --progress --timeout=0 --ignore-missing-args \
    --exclude=domains  \
    --exclude '.thumbswysiwyg'  \
    --exclude 'captcha'  \
    --exclude 'catalog/product/cache'  \
    --exclude 'catalog/tmp'  \
    --exclude 'import'  \
    --exclude 'customer'  \
    --exclude 'attachments'  \
    --exclude 'cache'  \
    --exclude 'tmp'  \
    --exclude 'order-attachments'  \
    --exclude 'static'  \
    --exclude 'sitemap'  \
    --exclude 'wysiwyg/.thumbs'  \
    --exclude '*.zip'  \
    --exclude '*.gzip'  \
    --exclude '*.tar.gz'  \
    --exclude '*.tgz'  \
    --exclude '.template-manager'  \
    --exclude '.thumbscatalog'  \
    --exclude '.thumbsemail'  \
    --exclude '.thumbsfavicon'  \
    --exclude '.thumbsmagefan_blog' \
    --update  \
    /data/web/current/pub/media/  \
    app@[SERVERNAME]:/data/web/current/pub/media/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment