🌊 Tractable Density Estimation: The Power of Invertible Transformations and Exact Likelihoods
While GANs and VAEs offer powerful generative capabilities, they rely on approximations (adversarial minimax games or Evidence Lower Bounds / ELBOs) and cannot compute the exact probability density of an input data point. Normalizing Flows bridge this limitation by chaining a sequence of invertible, differentiable mappings ($f = f_1 \circ f_2 \circ \dots \circ f_K$) to transform a complex data distribution $x \sim p_x(x)$ into a simple base latent distribution $z \sim p_z(z)$ (such as standard Gaussian noise). By rigorously applying the multivariate **Change of Variables Theorem**, normalizing flows enable exact maximum likelihood training and instantaneous bidirectional sampling without information loss or intractable approximations.
import torch
import torch.nn as nn
class RealNVPCouplingLayer(nn.Module):
def __init__(self, dim: int, hidden_dim: int = 64, mask_even: bool = True):
super().__init__()
self.dim = dim
self.mask_even = mask_even
# Scale (s) and Translation (t) neural networks
self.s_net = nn.Sequential(nn.Linear(dim // 2, hidden_dim), nn.ReLU(), nn.Linear(hidden_dim, dim // 2))
self.t_net = nn.Sequential(nn.Linear(dim // 2, hidden_dim), nn.ReLU(), nn.Linear(hidden_dim, dim // 2))
def forward(self, x):
# Split input vector along dimensions
d = self.dim // 2
x1, x2 = x[:, :d], x[:, d:] if self.mask_even else (x[:, :d], x[:, d:])
# For simplicity in syntax demonstration
x1, x2 = x[:, :d], x[:, d:]
s = self.s_net(x1)
# Apply stable exponential scaling
s = torch.tanh(s)
t = self.t_net(x1)
y2 = x2 * torch.exp(s) + t
y = torch.cat([x1, y2], dim=1)
# Log absolute Jacobian determinant sum
log_det = torch.sum(s, dim=1)
return y, log_det
def inverse(self, y):
d = self.dim // 2
y1, y2 = y[:, :d], y[:, d:]
s = torch.tanh(self.s_net(y1))
t = self.t_net(y1)
x2 = (y2 - t) * torch.exp(-s)
x = torch.cat([y1, x2], dim=1)
return x
# Glow replaces fixed permutations with learned invertible 1x1 convolutions.
# Weight matrix W is parameterized via LU decomposition: W = P * L * (U + diag(s))
# ensuring rapid determinant computation: log|det W| = sum(log abs(s)).
def compute_exact_log_likelihood(flow_model, seismic_data, conditioning_context):
z, log_det = flow_model(seismic_data, context=conditioning_context)
base_dist_log_prob = base_normal.log_prob(z).sum(dim=1)
exact_log_likelihood = base_dist_log_prob + log_det
return exact_log_likelihood.mean()
# Maps non-Gaussian asset returns into standard normal latent space via invertible polynomial flows,
# computing exact tail probabilities for regulatory stress testing.
# Packets yielding extremely low exact log-likelihood under normal traffic flow model
# are immediately flagged as zero-day data exfiltration attempts.
# Combines Squeeze operations and invertible 1x1 convolutions with affine coupling layers
# to map Gaussian noise directly to high-fidelity 22.05 kHz speech waveforms.
# Samples multiple plausible tumor boundary masks from the latent space conditioned on raw MRI voxels.
# Maps observed light distortions back to initial dark matter gravitational potential fields.
# Bypasses slow molecular dynamics simulation bottlenecks by sampling directly from target thermal equilibrium states.
# If rolling average log-likelihood drops below threshold Z-score,
# Prometheus triggers an automated MLOps alert for data drift retraining.