Monitoring SonarQube JVM/JMX metrics with Datadog

_README.md

Sonarqube

This was a quick hack to get Datadog Java/JMX monitoring working with Sonar.

Prerequisites

• Install docker

Run

• bash run.sh <your-datadog-api-key>
  • This will run two docker containers, one w/ sonarqube and one with the docker-dd-agent
• Connect to localhost:9000
• Download the CLI scanner at https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner
  • Update sonar-scanner.properties to point to localhost:9000
• Choose a project to scan or download the examples at https://github.com/SonarSource/sonar-examples/zipball/master
  • I suggest the projects/languages/generic-coverage/sonar-runner project
• Change into the directory where the code and configuration is
• Run <path-to-executable>/sonar-scanner
• You can find JMX metrics under the jmx.sonarqube.* and the standard java metrics under the jvm.* namespace!

docker_daemon.yaml

init_config:
  # Change the root directory to look at to get cgroup statistics. Useful when running inside a
  # container with host directories mounted on a different folder. Default: /.
  # Example for the docker-dd-agent container:
  # docker_root: /host

  # Timeout in seconds for the connection to the docker daemon
  # Default: 5 seconds
  #
  # timeout: 10

  # The version of the API the client will use. Specify 'auto' to use the API version provided by the server.
  # api_version: auto

  # Use TLS encryption while communicating with the Docker API
  #
  # tls: False
  # tls_client_cert: /path/to/client-cert.pem
  # tls_client_key: /path/to/client-key.pem
  # tls_cacert: /path/to/ca.pem
  # tls_verify: True

instances:
  - ## Daemon and system configuration
    ##

    # URL of the Docker daemon socket to reach the Docker API. HTTP/HTTPS also works.
    # Warning: if that's a non-local daemon, we won't be able to collect performance metrics.
    #
    url: "unix://var/run/docker.sock"

    ## Data collection
    ##

    # Create events whenever a container status change.
    # Defaults to true.
    #
    # collect_events: false

    # Collect disk usage per container with docker.container.size_rw and
    # docker.container.size_rootfs metrics.
    # Warning: This might take time for Docker daemon to generate,
    # ensure that `docker ps -a -q` run fast before enabling it.
    # Defaults to false.
    #
    # collect_container_size: false

    # Collect images stats
    # Number of available active images and intermediate images as gauges.
    # Defaults to false.
    #
    # collect_images_stats: false

    # Collect disk usage per image with docker.image.size and docker.image.virtual_size metrics.
    # The check gets this size with the `docker images` command.
    # Requires collect_images_stats to be enabled.
    # Defaults to false.
    #
    # collect_image_size: false

    # Collect disk metrics (total, used, free) through the docker info command for data and metadata.
    # This is useful when these values can't be obtained by the disk check.
    # Example: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html
    # Note that it only works when the storage driver is devicemapper.
    # Explanation of these metrics can be found here:
    # https://github.com/docker/docker/blob/v1.11.1/daemon/graphdriver/devmapper/README.md
    # Defaults to false.
    #
    # collect_disk_stats: true


    # Exclude containers based on their tags
    # An excluded container will be completely ignored. The rule is a regex on the tags.
    #
    # How it works: exclude first.
    # If a tag matches an exclude rule, it won't be included unless it also matches an include rule.
    # Examples:
    # exclude all, except ubuntu and debian.
    # exclude: [".*"]
    # include: ["docker_image:ubuntu", "docker_image:debian"]
    #
    # include all, except ubuntu and debian.
    # exclude: ["docker_image:ubuntu", "docker_image:debian"]
    # include: []
    #
    # Default: include all containers



    ## Tagging
    ##

    # You can add extra tags to your Docker metrics with the tags list option.
    # Example: ["extra_tag", "env:testing"]
    #
    tags: ["kelner-sonarqube"]

    # If the agent is running in an Amazon ECS task, tags container metrics with the ECS task name and version.
    # Default: true
    #
    # ecs_tags: false

    # Custom metrics tagging
    # Define which Docker tags to apply on metrics.
    # Since it impacts the aggregation, modify it carefully (only if you really need it).
    #
    # Tags for performance metrics.
    # Available:
    #   - image_name: Name of the image (example: "nginx")
    #   - image_tag: Tag of the image (example: "latest")
    #   - docker_image: LEGACY. The full image name:tag string (example: "nginx:latest")
    #   - container_name: Name of the container (example: "boring_euclid")
    #   - container_command: Command ran by the container (example: "echo 1")
    #   - container_id: Id of the container
    #
    # performance_tags: ["container_name", image_name", "image_tag", "docker_image"]

    # Tags for containers count metrics.
    # Available: ["image_name", "image_tag", "docker_image", "container_command"]
    #
    # container_tags: ["image_name", "image_tag", "docker_image"]

    # List of container label names that should be collected and sent as tags.
    # Default to None
    # Example:
    # collect_labels_as_tags: ["com.docker.compose.service", "com.docker.compose.project"]

jmx.yaml

init_config:
  # custom_jar_paths: # Optional, allows specifying custom jars that will be added to the classpath of the agent's JVM,
  # BREAKING CHANGE NOTICE : The agent currently supports a string if there is only one custom JAR. In future versions, this will be deprecated and MUST be a list in all cases.
  #   - /path/to/custom/jarfile.jar
  #   - /path/to/another/custom/jarfile2.jar

instances:
  - host: localhost
    port: 9999
    #user: username
    #password: password

    # If the agent needs to connect to a non-default JMX URL, specify it here instead of a host and a port
    # If you use this you need to specify a 'name' for the instance, below
    #jmx_url: "service:jmx:rmi:///jndi/rmi://myhost.host:9999/custompath"

    #process_name_regex: .*process_name.* # Instead of specifying a host and port or jmx_url, the agent can connect using the attach api.
                                          # This requires the JDK to be installed and the path to tools.jar to be set below.
    #tools_jar_path: /usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar # To be set when process_name_regex is set

    #name: jmx_instance
      #java_bin_path: /path/to/java # Optional, should be set if the agent cannot find your java executable
      #java_options: "-Xmx200m -Xms50m" # Optional, Java JVM options
      #trust_store_path: /path/to/trustStore.jks # Optional, should be set if ssl is enabled
      #trust_store_password: password
      #refresh_beans: 600 # Optional (in seconds), default is 600 seconds. Sets refresh period for refreshing matching MBeans list.
                          # Decreasing this value may result in increased CPU usage.
    tags:
      env: motorola
      poc: sonarqube

    # List of metrics to be collected by the integration
    # Read http://docs.datadoghq.com/integrations/java/ to learn how to customize it
    # https://docs.sonarqube.org/display/SONAR/Monitoring
    conf:
      - include:
          bean:
            - "SonarQube:name=ComputeEngineTasks"
            - "SonarQube:name=Database"
            - "SonarQube:name=ElasticSearch"
            - "SonarQube:name=Plugins"
            - "SonarQube:name=Settings"
            - "SonarQube:name=SonarQube"
            - "SonarQube:name=System"

run.sh

# !/bin/bash

if [ -z $1 ]; then
  echo "No API key passed, exiting"
  exit 1
fi

PWD=$(pwd)
J_OPTS="-Dcom.sun.management.jmxremote= -Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.rmi.port=9998 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false"

docker stop sonarqube-sandbox
docker rm sonarqube-sandbox

docker run -d --name sonarqube-sandbox \
  -p 9000:9000 -p 9999:9999 -p 9998:9998 \
  -e SONARQUBE_WEB_JVM_OPTS="${J_OPTS}" \
  -e JAVA_OPTIONS="${J_OPTS}" \
  sonarqube:latest

ADDR=$(docker exec sonarqube-sandbox hostname -i)
sed -i -e "s/localhost/${ADDR}/" ./jmx.yaml

docker stop dd-agent-sonar
docker rm dd-agent-sonar

docker run -d --name dd-agent-sonar \
   -v /var/run/docker.sock:/var/run/docker.sock:ro \
   -v /proc/:/host/proc/:ro \
   -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \
   -v $PWD:/conf.d:ro \
   -e API_KEY=$1 \
   datadog/docker-dd-agent:latest-jmx