Mixture¶

The mixturelib.mixture contains classes:

mixturelib.mixture.Mixture
mixturelib.mixture.MixtureEmSample
mixturelib.mixture.MixtureEM

class mixturelib.mixture.Mixture[source]¶

Base class for all mixtures.

fit(X=None, Y=None, epoch=10, progress=None)[source]¶

A method that fit a hyper model and local models in one procedure.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y – The tensor of shape num_elements \(\times\) num_answers. epoch (function) – The number of epoch of training. progress – The yield function for printing progress, like a tqdm. The function must take an iterator at the input and return the same data.

predict(X)[source]¶

A method that predict value for given input data.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature.
Returns:	The prediction of shape num_elements \(\times\) num_answers.
Return type:	FloatTensor

class mixturelib.mixture.MixtureEM(input_dim=10, K=2, HyperParameters={}, HyperModel=None, ListOfModels=None, ListOfRegularizeModel=None, device='cpu')[source]¶

The implementation of EM-algorithm for solving the two stage optimisation problem. Unlike mixturelib.mixture.MixtureEM, this class uses analytical solution when fit local models.

Warning

All Hyper Parameters should be additive to models, when you wanna optimize them.

Warning

It’s necessary! The parameter input_dim will be depricated.

It’s necessary! The parameter K will be depricated.

Parameters:

input_dim (int) – The number of features.
K (int) – The number of local models.
HyperParameters – The dictionary of all hyper parametrs. Where key is string and value is float or FloatTensor.
HyperModel (mixturelib.hyper_models.HyperModel) – The hyper model which are weighted all local models.
ListOfModels (list) – The list of models with E_step and M_step methods.
ListOfRegularizeModel (list) – The list of regulizers with E_step and M_step methods.
device (string) – The device for pytorch. Can be ‘cpu’ or ‘gpu’. Default ‘cpu’.

Example:

>>> _ = torch.random.manual_seed(42) # Set random seed for repeatability
>>>
>>> w = torch.randn(2, 1) # Generate real parameter vector
>>> X = torch.randn(12, 2) # Generate features data
>>> Y = torch.cat(
...         [
...             X[:5]@first_w, 
...             X[5:10]@second_w, 
...             X[10:11]@first_w, 
...             X[11:]@second_w
...         ])
...     + 0.1 * torch.randn(12, 1) # Generate target data with noise 0.1
>>>
>>> first_model = EachModelLinear(
...     input_dim=2, 
...     A=torch.tensor([1., 1.]),
...     w=torch.tensor([[0.], [0.]])) # Init first local model
>>> second_model = EachModelLinear(
...     input_dim=2,
...     A=torch.tensor([1., 1.]),
...     w=torch.tensor([[1.], [1.]])) # Init second local model
>>> hyper_model = HyperExpertNN(
...     input_dim=2, 
...     output_dim=2) # Init hyper model with Diriclet weighting
>>> hyper_parameters = {'beta': 1.} # Withor hyper parameters
>>>
>>> mixture = MixtureEmSample(
...     input_dim=2, K=2, HyperModel=hyper_model, 
...     HyperParameters=hyper_parameters, 
...     ListOfModels=[first_model, second_model]) # Init hyper model
>>> mixture.fit(X[:10], Y[:10]) # Optimise model parameter
>>>
>>> mixture.predict(X[10:])[0].view(-1)
tensor([ 0.0878, -0.4212])
>>> Y[10:].view(-1)
tensor([-0.2571, -0.4907])

E_step(X, Y)[source]¶

Doing E-step of EM-algorigthm. This method call E_step for all local models, for hyper model and for all regularizations step by step.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers.

M_step(X, Y)[source]¶

Doing M-step of EM-algorigthm. This method call M_step for all local models, for hyper model and for all regularizations step by step.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers.

fit(X=None, Y=None, epoch=10, progress=None)[source]¶

A method that fit a hyper model and local models in one procedure.

Call E-step and M-step in each epoch.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y – The tensor of shape num_elements \(\times\) num_answers. epoch (function) – The number of epoch of training. progress – The yield function for printing progress, like a tqdm. The function must take an iterator at the input and return the same data.

predict(X)[source]¶

A method that predict value for given input data.

For each x from X predicts \(answer = \sum_{k=1}^{K}\pi_k\bigr(x\bigr)g_k\bigr(x\bigr)\), where \(g_k\) is a local model.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature.
Returns:	The prediction of shape num_elements \(\times\) num_answers. The probability of shape num_elements \(\times\) num_models.
Return type:	FloatTensor, FloatTensor

class mixturelib.mixture.MixtureEmSample(input_dim=10, K=2, HyperParameters={}, HyperModel=None, ListOfModels=None, ListOfRegularizeModel=None, device='cpu')[source]¶

The implementation of EM-algorithm for solving the two stage optimisation problem. Unlike mixturelib.mixture.MixtureEM, this class samples data when fit local models.

Warning

All Hyper Parameters should be additive to models, when you wanna optimize them.

Warning

It’s necessary! The parameter input_dim will be depricated.

It’s necessary! The parameter K will be depricated.

Parameters:

input_dim (int) – The number of features.
K (int) – The number of local models.
HyperParameters – The dictionary of all hyper parametrs. Where key is string and value is float or FloatTensor.
HyperModel (mixturelib.hyper_models.HyperModel) – The hyper model which are weighted all local models.
ListOfModels (list) – The list of models with E_step and M_step methods.
ListOfRegularizeModel (list) – The list of regulizers with E_step and M_step methods.
device (string) – The device for pytorch. Can be ‘cpu’ or ‘gpu’. Default ‘cpu’.

Example:

>>> _ = torch.random.manual_seed(42) # Set random seed for repeatability
>>>
>>> w = torch.randn(2, 1) # Generate real parameter vector
>>> X = torch.randn(12, 2) # Generate features data
>>> Y = torch.cat(
...         [
...             X[:5]@first_w, 
...             X[5:10]@second_w, 
...             X[10:11]@first_w, 
...             X[11:]@second_w
...         ])
...     + 0.1 * torch.randn(12, 1) # Generate target data with noise 0.1
>>>
>>> first_model = EachModelLinear(
...     input_dim=2, 
...     A=torch.tensor([1., 1.]),
...     w=torch.tensor([[0.], [0.]])) # Init first local model
>>> second_model = EachModelLinear(
...     input_dim=2,
...     A=torch.tensor([1., 1.]),
...     w=torch.tensor([[1.], [1.]])) # Init second local model
>>> hyper_model = HyperExpertNN(
...     input_dim=2, 
...     output_dim=2) # Init hyper model with Diriclet weighting
>>> hyper_parameters = {'beta': 1.} # Withor hyper parameters
>>>
>>> mixture = MixtureEmSample(
...     input_dim=2, K=2, HyperModel=hyper_model, 
...     HyperParameters=hyper_parameters, 
...     ListOfModels=[first_model, second_model]) # Init hyper model
>>> mixture.fit(X[:10], Y[:10]) # Optimise model parameter
>>>
>>> mixture.predict(X[10:])[0].view(-1)
tensor([-0.0129, -0.4518])
>>> Y[10:].view(-1)
tensor([-0.2571, -0.4907])

E_step(X, Y)[source]¶

Doing E-step of EM-algorigthm. This method call E_step for all local models, for hyper model and for all regularizations step by step.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers.

M_step(X, Y)[source]¶

Doing M-step of EM-algorigthm. This method call M_step for all local models, for hyper model and for all regularizations step by step.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers.

fit(X=None, Y=None, epoch=10, progress=None)[source]¶

A method that fit a hyper model and local models in one procedure.

Call E-step and M-step in each epoch.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y – The tensor of shape num_elements \(\times\) num_answers. epoch (function) – The number of epoch of training. progress – The yield function for printing progress, like a tqdm. The function must take an iterator at the input and return the same data.

predict(X)[source]¶

A method that predict value and probability for each local models for given input data.

For each x from X predicts \(answer = \sum_{k=1}^{K}\pi_k\bigr(x\bigr)g_k\bigr(x\bigr)\), where \(g_k\) is a local model.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature.
Returns:	The prediction of shape num_elements \(\times\) num_answers. The probability of shape num_elements \(\times\) num_models.
Return type:	FloatTensor, FloatTensor