matchzoo.data_generator.callbacks package

Submodules

matchzoo.data_generator.callbacks.callback module

class matchzoo.data_generator.callbacks.callback.Callback

Bases: object

DataGenerator callback base class.

To build your own callbacks, inherit mz.data_generator.callbacks.Callback and overrides corresponding methods.

A batch is processed in the following way:

  • slice data pack based on batch index
  • handle on_batch_data_pack callbacks
  • unpack data pack into x, y
  • handle on_batch_x_y callbacks
  • return x, y
on_batch_data_pack(data_pack)

on_batch_data_pack.

Parameters:data_pack (DataPack) – a sliced DataPack before unpacking.
on_batch_unpacked(x, y)

on_batch_unpacked.

Parameters:
  • x (dict) – unpacked x.
  • y (ndarray) – unpacked y.

matchzoo.data_generator.callbacks.dynamic_pooling module

class matchzoo.data_generator.callbacks.dynamic_pooling.DynamicPooling(fixed_length_left, fixed_length_right, compress_ratio_left=1, compress_ratio_right=1)

Bases: matchzoo.data_generator.callbacks.callback.Callback

DPoolPairDataGenerator constructor.

Parameters:
  • fixed_length_left (int) – max length of left text.
  • fixed_length_right (int) – max length of right text.
  • compress_ratio_left (float) – the length change ratio, especially after normal pooling layers.
  • compress_ratio_right (float) – the length change ratio, especially after normal pooling layers.
on_batch_unpacked(x, y)

Insert dpool_index into x.

Parameters:
  • x – unpacked x.
  • y – unpacked y.

matchzoo.data_generator.callbacks.histogram module

class matchzoo.data_generator.callbacks.histogram.Histogram(embedding_matrix, bin_size=30, hist_mode='CH')

Bases: matchzoo.data_generator.callbacks.callback.Callback

Generate data with matching histogram.

Parameters:
  • embedding_matrix (ndarray) – The embedding matrix used to generator match histogram.
  • bin_size (int) – The number of bin size of the histogram.
  • hist_mode (str) – The mode of the MatchingHistogramUnit, one of CH, NH, and LCH.
on_batch_unpacked(x, y)

Insert match_histogram to x.

matchzoo.data_generator.callbacks.lambda_callback module

class matchzoo.data_generator.callbacks.lambda_callback.LambdaCallback(on_batch_data_pack=None, on_batch_unpacked=None)

Bases: matchzoo.data_generator.callbacks.callback.Callback

LambdaCallback. Just a shorthand for creating a callback class.

See matchzoo.data_generator.callbacks.Callback for more details.

Example

>>> import matchzoo as mz
>>> from matchzoo.data_generator.callbacks import LambdaCallback
>>> data = mz.datasets.toy.load_data()
>>> batch_func = lambda x: print(type(x))
>>> unpack_func = lambda x, y: print(type(x), type(y))
>>> callback = LambdaCallback(on_batch_data_pack=batch_func,
...                           on_batch_unpacked=unpack_func)
>>> data_gen = mz.DataGenerator(
...     data, batch_size=len(data), callbacks=[callback])
>>> _ = data_gen[0]
<class 'matchzoo.data_pack.data_pack.DataPack'>
<class 'dict'> <class 'numpy.ndarray'>
on_batch_data_pack(data_pack)

on_batch_data_pack.

on_batch_unpacked(x, y)

on_batch_unpacked.

Module contents