Small From Scratch Demo

README.md

Small From Scratch Demo

This is a very brief demo to show how one could go from an environment with just a Python 3 runtime to pyhf installed and being useful.

Installation

To take advantage of the great tools that uproot provides us let's install pyhf with the xmlio options

pip3 install pyhf[xmlio]

Running with CLI

Now that we have pyhf installed let's ask what the CLI can do for us

$ pyhf --help
Usage: pyhf [OPTIONS] COMMAND [ARGS]...

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  cls
  inspect
  json2xml
  xml2json  Entrypoint XML: The top-level XML file for the PDF definition.

and then let's use pyhf inspect to check to see if our example.json is in a correct form for the schema

$ pyhf inspect example.json
          Summary       
    ------------------  
       channels  1
        samples  2
     parameters  2
      modifiers  2

       channels  nbins
     ----------  -----
  singlechannel    2  

        samples
     ----------
     background
         signal

     parameters  constraint              modifiers
     ----------  ----------              ----------
             mu  unconstrained           normfactor
uncorr_bkguncrt  constrained_by_poisson  shapesys

    measurement           poi            parameters
     ----------        ----------        ----------
(*) Measurement            mu            (none)

finally use pyhf cls to get the observed CL_s and the expected CL_s band values (the "Brazil Band")

$ pyhf cls example.json
{
    "CLs_exp": [
        0.07807427911686152,
        0.17472571775474582,
        0.35998495263681274,
        0.6343568235898907,
        0.8809947004472013
    ],
    "CLs_obs": 0.3599845631401913
}

Expanding the CLI with other tools

Additionally, you can also use JSON Patch to patch a new signal in and update the likelihood (think reinterpretation).

$ pyhf cls example.json --patch new_signal.json
{
    "CLs_exp": [
        0.14695763778146098,
        0.2738178123017487,
        0.47642647066827803,
        0.7289555883715226,
        0.922094011644031
    ],
    "CLs_obs": 0.4764263982925686
}

and as the return format from pyhf cls is JSON you can also use jq to filter results

$ pyhf cls example.json --patch new_signal.json | jq .CLs_obs
0.4764263982925686

and of course given the way the CLI is setup you can pipe in the JSON which means that you can grab remotely hosted files (think HEPData). Here we curl the example.json down from GitHub

$ export TARGET_URL="https://gist.githubusercontent.com/matthewfeickert/86d8a0034487e3f5fba1e1365254c627/raw/$(git rev-parse HEAD)/example.json"
$ curl -sL "${TARGET_URL}" | pyhf cls --patch new_signal.json
{
    "CLs_exp": [
        0.14695763778146098,
        0.2738178123017487,
        0.47642647066827803,
        0.7289555883715226,
        0.922094011644031
    ],
    "CLs_obs": 0.4764263982925686
}

Running example

You can also run this whole example in a Docker container to get a play sandbox

$ docker run --rm -it -v $PWD:/data/ -w /data python:3.7 /bin/bash
# bash example.sh

example.json

{
    "channels": [
        { "name": "singlechannel",
          "samples": [
            { "name": "signal",
              "data": [5.0, 10.0],
              "modifiers": [ { "name": "mu", "type": "normfactor", "data": null} ]
            },
            { "name": "background",
              "data": [50.0, 60.0],
              "modifiers": [ {"name": "uncorr_bkguncrt", "type": "shapesys", "data": [5.0,12.0]} ]
            }
          ]
        }
    ],
    "observations": [
        { "name": "singlechannel", "data": [50.0, 60.0] }
    ],
    "measurements": [
        { "name": "Measurement", "config": {"poi": "mu", "parameters": []} }
    ],
    "version": "1.0.0"
}

example.sh

#!/usr/bin/env bash

function print_and_run () {
    printf "\n# %s\n" "${1}"
    eval $(echo "$1")
}

print_and_run "pip3 install --upgrade --no-cache-dir pip setuptools wheel"
print_and_run "pip3 list"
print_and_run "pip3 install pyhf[xmlio]"

print_and_run "pyhf --help"
print_and_run "pyhf inspect example.json"
print_and_run "pyhf cls example.json"

print_and_run "pyhf cls example.json --patch new_signal.json"
print_and_run "apt-get update -qq && apt-get install -qq -y jq"
print_and_run "pyhf cls example.json --patch new_signal.json | jq .CLs_obs"
print_and_run "export TARGET_URL='https://gist.githubusercontent.com/matthewfeickert/86d8a0034487e3f5fba1e1365254c627/raw/$(git rev-parse HEAD)/example.json'"
print_and_run "curl -sL '${TARGET_URL}' | pyhf cls --patch new_signal.json"

new_signal.json

[{
    "op": "replace",
    "path": "/channels/0/samples/0/data",
    "value": [5.0, 6.0]
}]