Last active
April 4, 2022 12:09
-
-
Save nmattam/bcfbc8a4ebd9a520c2ac50ab0137e58f to your computer and use it in GitHub Desktop.
[Splunk] Automate "Apply Changes" for Monitoring Console
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Create DMC groups | |
# This is basically updating the system/local/distsearch.conf | |
# I got all the IPs used below from /servicesNS/admin/-/search/distributed/peers. I used DNS names for search head (I had only one SH) and cluster master. So it was easy to separate the indexer IPs. | |
# 8089 is my API PORT | |
curl -k -u <user>:<password> <Monitoring Console>/services/search/distributed/groups -d name=dmc_group_search_head -d member=<search_head_member1:8089> -d member=<search_head_member2:8089> -d default=false | |
curl -k -u <user>:<password> <Monitoring Console>/services/search/distributed/groups -d name=dmc_group_cluster_master -d member=<cluster_master:8089> -d default=false | |
curl -k -u <user>:<password> <Monitoring Console>/services/search/distributed/groups -d name=dmc_group_indexer -d member=<indexer1:8089> -d member=<indexer2:8089> -d default=true # not idea why default=true for indexers | |
curl -k -u <user>:<password> <Monitoring Console>/services/search/distributed/groups -d name=dmc_indexerclustergroup_<CLUSTER_LABEL> -d member=<peer1:8089> -d member=<peer2:8089> -d default=false # Configure all peers in the cluster. | |
# Second time you run the above commands, you will run into an ERROR that says that the group already exists. | |
# So if the group exists, hit the "edit" endpoint to update the list of peers in a group. | |
# curl -k -u <user>:<password> <Monitoring Console>/services/search/distributed/groups/<dmc_group_name>/edit -d member=<member1:8089> -d member=<member2:8089> -d default=<true/false> # true for indexers, false for everything else | |
# Now configure Monitoring Console. | |
# Update the splunk_monitoring_console_assets.conf | |
curl -k -u <user>:<password> <Monitoring Console>/servicesNS/nobody/splunk_monitoring_console/configs/conf-splunk_monitoring_console_assets/settings -d configuredPeers=<comma separated lists of all the peers> -d output_mode=json -d eai:appName=splunk_monitoring_console -d eai:userName=nobody | |
# Kicks off the search to build the assets. | |
curl -k -u <user>:<password> <Monitoring Console>/servicesNS/nobody/splunk_monitoring_console/saved/searches/DMC+Asset+-+Build+Full/dispatch -d output_mode=json -d trigger_actions=true -d dispatch.auto_cancel=30 -d dispatch.buckets=300 -d dispatch.enablePreview=true | |
# Finally update the app. | |
curl -k -u <user>:<password> <Monitoring Console>/servicesNS/nobody/system/apps/local/splunk_monitoring_console -d output_mode=json -d author=Splunk -d check_for_updates=1 -d configured=1 -d description=The+Splunk+Monitoring+Console+application+gives+you+insight+into+your+Splunk+deployment. -d label=Monitoring+Console -d version=<Splunk Version> -d visible=1 | |
# There was another endpoint that was hit to update the nav/default.xml to reveal new distributed dashboards. The request body | |
# had the entire xml, hence I landed it via Chef using the cerner_splunk cookbook apps hash (https://github.com/cerner/cerner_splunk/blob/stable/docs/databags.md#apps-hash). But this was the content | |
<nav color='#65A637'> | |
<view name='monitoringconsole_overview' default='true' /> | |
<view name='monitoringconsole_check' /> | |
<view name='monitoringconsole_instances' /> | |
<collection label='Indexing'> | |
<collection label='Performance'> | |
<view name='indexing_performance_instance' /> | |
<view name='indexing_performance_deployment' /> | |
</collection> | |
<collection label='Indexer Clustering'> | |
<!--<a href='Clustering'>Indexer Clustering:Status</a>--> | |
<view name='indexer_clustering_status' /> | |
<view name='indexer_clustering_service_activity' /> | |
</collection> | |
<collection label='Indexes and Volumes'> | |
<view name='indexes_and_volumes_instance' /> | |
<view name='indexes_and_volumes_deployment' /> | |
<view name='index_detail_instance' /> | |
<view name='index_detail_deployment' /> | |
<view name='volume_detail_instance' /> | |
<view name='volume_detail_deployment' /> | |
</collection> | |
<collection label='Inputs'> | |
<view name='http_event_collector_instance' /> | |
<view name='http_event_collector_deployment' /> | |
<view name='splunk_tcpin_performance_instance' /> | |
<view name='splunk_tcpin_performance_deployment' /> | |
<view name='data_quality' /> | |
</collection> | |
<collection label='License Usage'> | |
<view name='license_usage_today' /> | |
<view name='license_usage_30days' /> | |
</collection> | |
</collection> | |
<collection label='Search'> | |
<collection label='Activity'> | |
<view name='search_activity_instance' /> | |
<view name='search_activity_deployment' /> | |
<view name='search_usage_statistics_instance' /> | |
<view name='search_usage_statistics_deployment' /> | |
</collection> | |
<collection label='Distributed Search'> | |
<view name='distributed_search_instance' /> | |
<view name='distributed_search_deployment' /> | |
</collection> | |
<collection label='Search Head Clustering'> | |
<view name='shc_status_and_conf' /> | |
<view name='shc_conf_rep' /> | |
<view name='shc_artifact_replication' /> | |
<view name='shc_scheduler_delegation_statistics' /> | |
<view name='shc_app_deployment' /> | |
</collection> | |
<collection label='Scheduler Activity'> | |
<view name='scheduler_activity_instance' /> | |
<view name='scheduler_activity_deployment' /> | |
</collection> | |
<collection label='KV Store'> | |
<view name='kv_store_instance' /> | |
<view name='kv_store_deployment' /> | |
</collection> | |
</collection> | |
<collection label='Resource Usage'> | |
<view name='resource_usage_instance' /> | |
<view name='resource_usage_machine' /> | |
<view name='resource_usage_deployment' /> | |
</collection> | |
<collection label='Forwarders'> | |
<view name='forwarder_instance' /> | |
<view name='forwarder_deployment' /> | |
</collection> | |
<collection label='Settings'> | |
<view name='monitoringconsole_configure' /> | |
<view name='monitoringconsole_forwarder_setup' /> | |
<view name='monitoringconsole_alerts_setup' /> | |
<view name='monitoringconsole_overview_preferences'/> | |
<view name='monitoringconsole_check_list' /> | |
</collection> | |
<a href='search'>Run a Search</a> | |
</nav> |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It doesn't work anymore.