SVI

Stochastic variational inference algorithms.

svi

Stochastic Variational Inference (SVI) training loop.

SVI optimizes the ELBO by taking gradient steps on the guide and model parameters. Each call to step performs one optimization step and returns the loss value.

SVI 

SVI(model: MonadicProgram, guide: Guide, optim: Optimizer, loss: ELBO)

Stochastic Variational Inference optimizer.

PARAMETER DESCRIPTION
model

The generative model.

TYPE: MonadicProgram

guide

The variational guide.

TYPE: Guide

optim

Optimizer for both model and guide parameters.

TYPE: Optimizer

loss

The ELBO loss module.

TYPE: ELBO

Source code in src/quivers/inference/svi.py 
32
33
34
35
36
37
38
39
40
41
42
def __init__(
    self,
    model: MonadicProgram,
    guide: Guide,
    optim: torch.optim.Optimizer,
    loss: ELBO,
) -> None:
    self.model = model
    self.guide = guide
    self.optim = optim
    self.loss = loss

step 

step(x: Tensor, observations: dict[str, Tensor]) -> float

Perform a single SVI optimization step.

PARAMETER DESCRIPTION
x

Program input. Shape (batch, ...).

TYPE: Tensor

observations

Observed variable values.

TYPE: dict[str, Tensor]

RETURNS DESCRIPTION
float

The loss value for this step.
Source code in src/quivers/inference/svi.py 
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
def step(
    self,
    x: torch.Tensor,
    observations: dict[str, torch.Tensor],
) -> float:
    """Perform a single SVI optimization step.

    Parameters
    ----------
    x : torch.Tensor
        Program input. Shape (batch, ...).
    observations : dict[str, torch.Tensor]
        Observed variable values.

    Returns
    -------
    float
        The loss value for this step.
    """
    self.optim.zero_grad()
    loss_val = self.loss(self.model, self.guide, x, observations)
    loss_val.backward()
    self.optim.step()
    return loss_val.item()