Mixture

The mixturelib.mixture contains classes:

class mixturelib.mixture.Mixture[source]

Base class for all mixtures.

fit(X=None, Y=None, epoch=10, progress=None)[source]

A method that fit a hyper model and local models in one procedure.

Parameters:
  • X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature.
  • Y – The tensor of shape num_elements \(\times\) num_answers.
  • epoch (function) – The number of epoch of training.
  • progress – The yield function for printing progress, like a tqdm. The function must take an iterator at the input and return the same data.
predict(X)[source]

A method that predict value for given input data.

Parameters:X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature.
Returns:The prediction of shape num_elements \(\times\) num_answers.
Return type:FloatTensor
class mixturelib.mixture.MixtureEM(HyperParameters={}, HyperModel=None, ListOfModels=None, ListOfRegularizeModel=None, model_type='default', device='cpu')[source]

The implementation of EM-algorithm for solving the two stage optimisation problem.

Warning

All Hyper Parameters should be additive to models, when you wanna optimize them.

Parameters:
  • HyperParameters – The dictionary of all hyper parametrs. Where key is string and value is float or FloatTensor.
  • HyperModel (mixturelib.hyper_models.HyperModel) – The hyper model which are weighted all local models.
  • ListOfModels (list) – The list of models with E_step and M_step methods.
  • ListOfRegularizeModel (list) – The list of regulizers with E_step and M_step methods.
  • model_type (string) – Type os EM algorithm. Can be default or sample. In default EM model all objects uses in each local models with weights. In sample EM model all objects are sampled during to their weights and just sampled samples uses in local models.
  • device (string) – The device for pytorch. Can be ‘cpu’ or ‘gpu’. Default ‘cpu’.

Example:

>>> _ = torch.random.manual_seed(42) # Set random seed for repeatability
>>>
>>> first_w = torch.randn(2, 1) # Generate first real parameter vector
>>> second_w = torch.randn(2, 1) # Generate second real parameter vector
>>> X = torch.randn(102, 2) # Generate features data
>>> Y = torch.cat(
...         [
...             X[:50]@first_w, 
...             X[50:10]@second_w, 
...             X[100:101]@first_w, 
...             X[101:]@second_w
...         ])
...     + 0.01 * torch.randn(102, 1) # Generate target data with noise 0.1
>>>
>>> first_model = EachModelLinear(
...     input_dim=2, 
...     A=torch.tensor([1., 1.]),
...     w=torch.tensor([0., 0.])) # Init first local model
>>> second_model = EachModelLinear(
...     input_dim=2,
...     A=torch.tensor([1., 1.]),
...     w=torch.tensor([[1.], [1.]])) # Init second local model
>>> hyper_model = HyperExpertNN(
...     input_dim=2, 
...     output_dim=2) # Init hyper model with Diriclet weighting
>>> hyper_parameters = {'beta': 1.} # Withor hyper parameters
>>>
>>> mixture = MixtureEM(
...     HyperModel=hyper_model, 
...     HyperParameters=hyper_parameters, 
...     ListOfModels=[first_model, second_model],
...     model_type='sample') # Init hyper model
>>> mixture.fit(X[:100], Y[:100]) # Optimise model parameter
>>>
>>> mixture.predict(X[100:])[0].view(-1)
tensor([-0.1245, -0.4357])
>>> Y[100:].view(-1)
tensor([-0.0936, -0.4177])
E_step(X, Y)[source]

Doing E-step of EM-algorigthm. This method call E_step for all local models, for hyper model and for all regularizations step by step.

Parameters:
  • X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature.
  • Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers.
M_step(X, Y)[source]

Doing M-step of EM-algorigthm. This method call M_step for all local models, for hyper model and for all regularizations step by step.

Parameters:
  • X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature.
  • Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers.
fit(X=None, Y=None, epoch=10, progress=None)[source]

A method that fit a hyper model and local models in one procedure.

Call E-step and M-step in each epoch.

Parameters:
  • X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature.
  • Y – The tensor of shape num_elements \(\times\) num_answers.
  • epoch (function) – The number of epoch of training.
  • progress – The yield function for printing progress, like a tqdm. The function must take an iterator at the input and return the same data.
predict(X)[source]

A method that predict value for given input data.

For each x from X predicts \(answer = \sum_{k=1}^{K}\pi_k\bigr(x\bigr)g_k\bigr(x\bigr)\), where \(g_k\) is a local model.

Parameters:X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature.
Returns:The prediction of shape num_elements \(\times\) num_answers.

The probability of shape num_elements \(\times\) num_models.
Return type:FloatTensor, FloatTensor