Local Models¶

The mixturelib.local_models contains classes:

mixturelib.local_models.EachModel
mixturelib.local_models.EachModelLinear

class mixturelib.local_models.EachModel[source]¶

Base class for all local models.

E_step(X, Y, Z, HyperParameters)[source]¶

Doing E-step of EM-algorithm. Finds variational probability q of model parameters.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers. Z (FloatTensor) – The tensor of shape num_elements \(\times\) 1. HyperParameters (dict) – The dictionary of all hyper parametrs. Where key is string and value is FloatTensor.

LogLikeLihoodExpectation(X, Y, HyperParameters)[source]¶

Returns expected log-likelihod of a given vector of answers for given data seperated. The expectation is taken according to the model parameters \(\mathsf{E}p\bigr(Y|X\bigr)\).

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers. HyperParameters (dict) – The dictionary of all hyper parametrs. Where key is string and value is FloatTensor.
Returns:	The tensor of shape num_elements \(\times\) 1.
Return type:	FloatTensor

M_step(X, Y, Z, HyperParameters)[source]¶

Doing M-step of EM-algorithm. Finds model hyper parameters.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers. Z (FloatTensor) – The tensor of shape num_elements \(\times\) 1. HyperParameters (dict) – The dictionary of all hyper parametrs. Where key is string and value is FloatTensor.

OptimizeHyperParameters(X, Y, Z, HyperParameters, Parameter)[source]¶

Returns the local part of new Parameter.

Warning

Returned local part must be aditive to Parameter, because new value of Parameter is sum of local part from all local model.

Warning

The number num_answers can be just 1.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers. Z (FloatTensor) – The tensor of shape num_elements \(\times\) 1. HyperParameters (dict) – The dictionary of all hyper parametrs. Where key is string and value is FloatTensor. Parameter (string) – The key from HyperParameters dictionary. The name of the hyperparameter to be optimized.
Returns:	A local part of new HyperParameters[Parameter] value.
Return type:	FloatTensor

forward(input)[source]¶

Returns model prediction for the given input data.

Parameters:	input (FloatTensor.) – The tensor of shape num_elements \(\times\) num_feature.
Returns:	The tensor of shape num_elements \(\times\) num_answers. Model answers for the given input data
Return type:	FloatTensor

class mixturelib.local_models.EachModelLinear(input_dim=20, device='cpu', A=None, w=None, OptimizedHyper={'A', 'beta', 'w_0'})[source]¶

A model for solving the linear regression problem \(\textbf{y} = \textbf{x}^{\mathsf{T}}\textbf{w}\). The model uses an analytical solution for estimation \(\textbf{w}\). Also model finds a distribution of model parameters \(\textbf{w}\).

Distribution is \(\textbf{w} \sim \mathcal{N}\bigr(\hat{\textbf{w}}, \textbf{A}\bigr)\)

Warning

The priors A and w must be set or not together.

Parameters:

input_dim (int) – The number of feature in each object. Must be positive.
device (string) – The device for pytorch. Can be ‘cpu’ or ‘gpu’. Default ‘cpu’.
A (FloatTensor) – The tensor of shape input_dim \(\times\) input_dim. It is a prior covariance matrix for model parameters. Also can be the tensor of shape input_dim. In this case, it is a diagonal of prior covariance matrix for model parameters, and all nondiagonal values are zerous.
w (FloatTensor) – The tensor of shape input_dim. It is a prior mean for model parameters
OptimizedHyper (set) – The set of hyperparameters that will be optimized. Default all hyperparameters will be optimized.

Example:

>>> _ = torch.random.manual_seed(42) # Set random seed for repeatability
>>>
>>> w = torch.randn(2, 1) # Generate real parameter vector
>>> X = torch.randn(10, 2) # Generate features data
>>> Z = torch.ones(10, 1) # Set that all data correspond to this model
>>> Y = X@w + 0.1*torch.randn(10, 1) # Generate target data with noise 0.1
>>>
>>> linear_model = EachModelLinear(
...     input_dim=2) # Init linear model withot any priors
>>> hyper_parameters = {
...     'beta': torch.tensor(1.)} # Init noise level beta
>>>
>>> linear_model.E_step(
...     X[:8], Y[:8], Z[:8], hyper_parameters) # Optimise model parameter
>>>
>>> linear_model(X[8:]).view(-1) # Model prediction for the test part
tensor([-0.1975, -0.2427])
>>> Y[8:].view(-1) # Real target for test part
tensor([-0.2124, -0.1837])

B = None¶: Object \(\textbf{B}\) is a covariance matrix for variational distribution

E_step(X, Y, Z, HyperParameters)[source]¶

Doing E-step of EM-algorithm. Finds variational probability q of model parameters.

Calculate analytical solution for estimate q in the class of normal distributions \(q = \mathcal{N}\bigr(m, B\bigr)\), where \(B = (A^{-1}+\beta\sum_{i=1}X_iX_i^{\mathsf{T}}\mathsf{E}z_{i})\) and \(m = B(A^{-1}w_0+\beta\sum_{i=1}X_iY_i\mathsf{E}z_{i})\)

Warning

HyperParameters must contain beta hyperparameter.

Warning

The number num_answers can be just 1.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers. Z (FloatTensor) – The tensor of shape num_elements \(\times\) 1. HyperParameters (dict) – The dictionary of all hyper parametrs. Where key is string and value is FloatTensor.

LogLikeLihoodExpectation(X, Y, HyperParameters)[source]¶

Returns expected log-likelihod of a given vector of answers for given data seperated. The expectation is taken according to the model parameters \(\mathsf{E}p\bigr(Y|X\bigr)\).

Warning

HyperParameters must contain beta hyperparameter.

Warning

The number num_answers can be just 1.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers. HyperParameters (dict) – The dictionary of all hyper parametrs. Where key is string and value is FloatTensor.
Returns:	The tensor of shape num_elements \(\times\) 1.
Return type:	FloatTensor

M_step(X, Y, Z, HyperParameters)[source]¶

Doing M-step of EM-algorithm. Finds model hyper parameters.

Warning

HyperParameters must contain beta hyperparameter.

Warning

The number num_answers can be just 1.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers. Z (FloatTensor) – The tensor of shape num_elements \(\times\) 1. HyperParameters (dict) – The dictionary of all hyper parametrs. Where key is string and value is FloatTensor.

OptimizeHyperParameters(X, Y, Z, HyperParameters, Parameter)[source]¶

Returns the local part of new Parameter.

In this case local_part is inverse beta: \(\frac{1}{\beta} = \frac{1}{num\_elements} \sum_{i=1}^{num\_elements}[Y_i^2-2Y_iX_i^{\mathsf{T}}\hat{\textbf{w}} + X_i^{\mathsf{T}}\mathsf{E}[ww^{\mathsf{T}}]X_i]\)

Warning

Return local part must be aditive to Parameter, because new value of Parameter is sum of local part from all local model.

Warning

HyperParameters must contain beta hyperparameter.

Warning

The number num_answers can be just 1.

Parameters:	X (FloatTensor) – The tensor of shape num_elements \(\times\) num_feature. Y (FloatTensor) – The tensor of shape num_elements \(\times\) num_answers. Z (FloatTensor) – The tensor of shape num_elements \(\times\) 1. HyperParameters (dict) – The dictionary of all hyper parametrs. Where key is string and value is FloatTensor. Parameter (string) – The key from HyperParameters dictionary. The name of the hyperparameter to be optimized.
Returns:	A local part of new HyperParameters[Parameter] value.
Return type:	FloatTensor

W = None¶: Object \(\textbf{W}\) is a model parameters

forward(input)[source]¶

Returns model prediction for the given input data. Linear prediction is \(input@w\).

Warning

The number num_answers can be just 1.

Parameters:	input (FloatTensor.) – The tensor of shape num_elements \(\times\) num_feature.
Returns:	The tensor of shape num_elements \(\times\) num_answers. Model answers for the given input data
Return type:	FloatTensor