Candidate API changes for Chainer v2

chainer-v2-api-change.md

Candidate API changes for Chainer v2

In this gist, I list up the possible changes on existing APIs for Chainer v2. It also includes some ideas of adding new APIs related to existing ones. Note that this is not a complete list of future changes for v2.

Core features

Variable

• __len__ should return the length of the first axis.
  • See chainer/chainer#1792
• Remove volatile flag?
  • It can be replaced by no_backprop_mode, which is easier to use.
  • We can also remove Flag
• Remove zerograd
• Support uninitialized Variable
  • It can be used for the better support of uninitialized parameters (used for "parameter shape placeholder")
• Support optimizer for Variable
  • It enables us to choose an optimizer for each parameter variable.
  • We need a design document and discussions on it.

Function

• type_check_enable: Make it a global/thread-local flag
• Make type checking enabled only in debug mode by default

Link

• add_param and add_persistent does not work well with PyCharm
  • An added attribute is not recognized by the IDE
  • It is better to design a new API to avoid this issue
• add_uninitialized_param should be redesigned to work with uninitialized Variable (see above)

Chain/ChainList

• add_link: see the above discussions on add_param
• to_gpu should be applied to links added in the future
• We want to support duplicated parents of a link (it is currently prohibited)

Optimizer

• Remove deprecated methods (most of which can be replaced by optimizer hooks)
• Support per-parameter configurations (see notes on Variable)

Serializer/Deserializer

• Stop using Abstract Base Class
• Support non-strict mode that allows the parameter set mismatch to the set of loaded parameters

Updater/StandardUpdater

• The interface should be updated to support the updated optimizer APIs.

Reporter

• Support non-scalar observations.

FunctionSet

• Remove it.

cuda module

• Remove deprecated APIs.
• Deprecate get_device() and add alternatives: get_device_from_id, get_device_from_object, etc.
• to_cpu / to_gpu: support Variable as an input.

Collection of functions, links, and extensions

extensions

• snapshot/snapshot_object: Remove the trigger option, which is redundant.
• LogReport: Think a better name of the trigger option.

functions

• Flags: make them global/thread-local and remove them from arguments
  • use_cudnn
  • train/test
  • deterministic
• batch_normalization: Think about better interface
• softmax_cross_entropy: Rename normalize option
• softmax_cross_entropy: Allow ignore_label configurable by init argument
• split_axis: Make force_tuple True by default

links

• initialW, initial_bias, ...: Unify the naming convention of the arguments.
• input size, input channels, ...: Make them optional (we may need to change the overall APIs)
• wscale: Remove it.
• set_state / reset_state of RNN units: Unify the interface.
• BatchNormalization: Think about better interface
• ConvolutionND: Make the bias enabled by default
• Linear: Make the number of axes for batch dimensions configurable

(sorry for late replay, I came down with a cold around the last weekend) Thanks for writing up the use case of grad accumulation to intermediate nodes. It now makes sense to me. I think it's better to provide a new backprop interface to allow multiple starting points (i.e., the "root nodes" of the backward path) instead of accumulating gradients to intermediate nodes, since the latter approach will break codes which actually runs the backprop through the same subgraph multiple times (I don't think such a code is reasonable in terms of performance, but it should work in some try-and-error for coding in which the performance does not matter). The current Variable.backward() can be seen as a special case of this method for which only one root node is passed as the starting point of the backward path.