HParams

class texar.torch.HParams(hparams, default_hparams, allow_new_hparam=False)

A class that maintains hyperparameters for configuring Texar modules. The class has several useful features:

• Auto-completion of missing values. Users can specify only a subset of hyperparameters they care about. Other hyperparameters will automatically take the default values. The auto-completion performs recursively so that hyperparameters taking dict values will also be auto-completed All Texar modules provide a default_hparams() containing allowed hyperparameters and their default values. For example:

  ## Recursive auto-completion
  default_hparams = {"a": 1, "b": {"c": 2, "d": 3}}
  hparams = {"b": {"c": 22}}
  hparams_ = HParams(hparams, default_hparams)
  hparams_.todict() == {"a": 1, "b": {"c": 22, "d": 3}}
      # "a" and "d" are auto-completed
  
  ## All Texar modules have built-in `default_hparams`
  hparams = {"dropout_rate": 0.1}
  emb = tx.modules.WordEmbedder(hparams=hparams, ...)
  emb.hparams.todict() == {
      "dropout_rate": 0.1,  # provided value
      "dim": 100            # default value
      ...
  }
  

• Automatic type-check. For most hyperparameters, provided value must have the same or compatible dtype with the default value. HParams does necessary type-check, and raises Error if improper dtype is provided. Also, hyperparameters not listed in default_hparams are not allowed, except for “kwargs” as detailed below.

• Flexible dtype for specified hyperparameters. Some hyperparameters may allow different dtypes of values.

  • Hyperparameters named “type” are not type-checked. For example, in get_rnn_cell(), hyperparameter “type” can take value of an RNNCell class, its string name of module path, or an RNNCell class instance. (String name or module path is allowed so that users can specify the value in YAML configuration files.)

  • For other hyperparameters, list them in the “@no_typecheck” field in default_hparams() to skip type-check. For example, in Conv1DNetwork, hyperparameter “kernel_size” can be set to either a list of ints or simply an int.

• Special flexibility of keyword argument hyperparameters. Hyperparameters named "kwargs" are used as keyword arguments for a class constructor or a function call. Such hyperparameters take a dict, and users can add arbitrary valid keyword arguments to the dict. For example:

  default_rnn_cell_hparams = {
      "type": "LSTMCell",
      "kwargs": {"num_units": 256}
      # Other hyperparameters
      ...
  }
  my_hparams = {
      "kwargs" {
          "num_units": 123,
          # Other valid keyword arguments for LSTMCell constructor
          "forget_bias": 0.0
          "activation": "torch.nn.functional.relu"
      }
  }
  _ = HParams(my_hparams, default_rnn_cell_hparams)
  

• Rich interfaces. An HParams instance provides rich interfaces for accessing, updating, or adding hyperparameters.

  hparams = HParams(my_hparams, default_hparams)
  # Access
  hparams.type == hparams["type"]
  # Update
  hparams.type = "GRUCell"
  hparams.kwargs = { "num_units": 100 }
  hparams.kwargs.num_units == 100
  # Add new
  hparams.add_hparam("index", 1)
  hparams.index == 1
  
  # Convert to `dict` (recursively)
  type(hparams.todic()) == dict
  
  # I/O
  pickle.dump(hparams, "hparams.dump")
  with open("hparams.dump", 'rb') as f:
      hparams_loaded = pickle.load(f)
  

Parameters

• hparams – A dict or an HParams instance containing hyperparameters. If None, all hyperparameters are set to default values.

• default_hparams (dict) – Hyperparameters with default values. If None, Hyperparameters are fully defined by hparams.

• allow_new_hparam (bool) – If False (default), hparams cannot contain hyperparameters that are not included in default_hparams, except for the case of "kwargs" as above.

items()

Returns the list of hyperparameter (name, value) pairs.

keys()

Returns the list of hyperparameter names.

get(name, default=None)

Returns the hyperparameter value for the given name. If name is not available then returns default.

Parameters

• name (str) – the name of hyperparameter.

• default – the value to be returned in case name does not exist.

add_hparam(name, value)

Adds a new hyperparameter.

todict()

Returns a copy of hyperparameters as a dictionary.