[hydra]

readme.md

MLFlowと他ツールの組み合わせ - Retrieva TECH BLOG

ハイパラ管理のすすめ -ハイパーパラメータをHydra+MLflowで管理しよう- - やむやむもやむなし

Getting started | Hydra

confs

> tree .
.
├── conf
│   ├── config.yaml
│   ├── db
│   │   ├── mysql.yaml
│   │   └── postgresql.yaml
│   ├── schema
│   │   ├── school.yaml
│   │   ├── support.yaml
│   │   └── warehouse.yaml
│   └── ui
│       ├── ui.yaml
│       └── view.yaml
└─ my_app.py

config.yaml

defaults:
  - db: mysql

db/mysql.yaml

driver: mysql
user: omry
password: secret

run

$ python my_app.py db=postgresql

$ python my_app.py -m schema=warehouse,support,school

$ python my_app.py -m schema=warehouse,support,school db=mysql,postgresql

sample

# setup
pip install omegaconf
pip install hydra-core

my_app.py

import os
from loguru import logger

import hydra
from omegaconf import DictConfig, OmegaConf


logger.info(f"Working directory : {os.getcwd()}")

@hydra.main(config_path="conf", config_name="config")
def my_app(cfg: DictConfig) -> None:
    logger.info("..")
    logger.info(OmegaConf.to_yaml(cfg))
    logger.info(f"Org wor dir : {hydra.utils.get_original_cwd()}")
    logger.info(f"to_abs_path : {hydra.utils.to_absolute_path('foo')}")
    logger.info(f"to_abs_path : {hydra.utils.to_absolute_path('/foo')}")


if __name__ == "__main__":
    my_app()

working pathes

hydra.utils.get_original_cwd()

hydra.utils.to_absolute_path('foo')

logging

# ロギングのターゲットを指定
$ python my_app.py hydra.verbose=[__main__,hydra]

# ログを非表示
$ python my_app.py hydra/job_logging=disabled

info

$ python my_app.py --info

Shell complitation

help の確認

$ python my_app.py --hydra-help


Hydra (1.0.4)
See https://hydra.cc for more info.

== Flags ==
--help,-h : Application's help
--hydra-help : Hydra's help
--version : Show Hydra's version and exit
--cfg,-c : Show config instead of running [job|hydra|all]
--package,-p : Config package to show
--run,-r : Run a job
--multirun,-m : Run multiple jobs with the configured launcher and sweeper
--shell-completion,-sc : Install or Uninstall shell completion:
    Bash - Install:
    eval "$(python my_app.py -sc install=bash)"
    Bash - Uninstall:
    eval "$(python my_app.py -sc uninstall=bash)"

    Fish - Install:
    python my_app.py -sc install=fish | source
    Fish - Uninstall:
    python my_app.py -sc uninstall=fish | source

--config-path,-cp : Overrides the config_path specified in hydra.main().
                    The config_path is relative to the Python file declaring @hydra.main()
--config-name,-cn : Overrides the config_name specified in hydra.main()
--config-dir,-cd : Adds an additional config dir to the config search path
--info,-i : Print Hydra information
Overrides : Any key=value arguments to override config values (use dots for.nested=overrides)

== Configuration groups ==
Compose your configuration from those groups (For example, append hydra/job_logging=disabled to command line)

hydra/help: default
hydra/hydra_help: default
hydra/hydra_logging: default, disabled, hydra_debug
hydra/job_logging: default, disabled, stdout
hydra/launcher: basic
hydra/output: default
hydra/sweeper: basic


Use '--cfg hydra' to Show the Hydra config.

Shell complitation

Bash と Fish の場合のTab completion コマンドが記載されている

--shell-completion,-sc : Install or Uninstall shell completion:

    Bash - Install:
    eval "$(python my_app.py -sc install=bash)"
    Bash - Uninstall:
    eval "$(python my_app.py -sc uninstall=bash)"

    Fish - Install:
    python my_app.py -sc install=fish | source
    Fish - Uninstall:
    python my_app.py -sc uninstall=fish | source

これにより，configグループの対象が補完される. 便利.

> python my_app.py db=
db=mysql  db=postgresql