updating

Author: mhjensen

doc/pub/week13/html/week13-bs.html

@@ -132,14 +132,6 @@
               ('Diffusion models', 2, None, 'diffusion-models'),
               ('Original idea', 2, None, 'original-idea'),
               ('Diffusion learning', 2, None, 'diffusion-learning'),
-              ('Diffusion models, basics', 2, None, 'diffusion-models-basics'),
-              ('Problems with probabilistic models',
-               2,
-               None,
-               'problems-with-probabilistic-models'),
-              ('Diffusion models', 2, None, 'diffusion-models'),
-              ('Original idea', 2, None, 'original-idea'),
-              ('Diffusion learning', 2, None, 'diffusion-learning'),
               ('Mathematics of diffusion models',
                2,
                None,
@@ -273,11 +265,6 @@
      <!-- navigation toc: --> <li><a href="#diffusion-models" style="font-size: 80%;">Diffusion models</a></li>
      <!-- navigation toc: --> <li><a href="#original-idea" style="font-size: 80%;">Original idea</a></li>
      <!-- navigation toc: --> <li><a href="#diffusion-learning" style="font-size: 80%;">Diffusion learning</a></li>
-     <!-- navigation toc: --> <li><a href="#diffusion-models-basics" style="font-size: 80%;">Diffusion models, basics</a></li>
-     <!-- navigation toc: --> <li><a href="#problems-with-probabilistic-models" style="font-size: 80%;">Problems with probabilistic models</a></li>
-     <!-- navigation toc: --> <li><a href="#diffusion-models" style="font-size: 80%;">Diffusion models</a></li>
-     <!-- navigation toc: --> <li><a href="#original-idea" style="font-size: 80%;">Original idea</a></li>
-     <!-- navigation toc: --> <li><a href="#diffusion-learning" style="font-size: 80%;">Diffusion learning</a></li>
      <!-- navigation toc: --> <li><a href="#mathematics-of-diffusion-models" style="font-size: 80%;">Mathematics of diffusion models</a></li>
      <!-- navigation toc: --> <li><a href="#chains-of-vaes" style="font-size: 80%;">Chains of VAEs</a></li>
      <!-- navigation toc: --> <li><a href="#mathematical-representation" style="font-size: 80%;">Mathematical representation</a></li>
@@ -1316,72 +1303,6 @@ <h2 id="code-in-pytorch-for-vaes" class="anchor">Code in PyTorch for VAEs </h2>
 </div>
 
 
-<!-- !split -->
-<h2 id="diffusion-models-basics" class="anchor">Diffusion models, basics </h2>
-
-<p>Diffusion models are inspired by non-equilibrium thermodynamics. They
-define a Markov chain of diffusion steps to slowly add random noise to
-data and then learn to reverse the diffusion process to construct
-desired data samples from the noise. Unlike VAE or flow models,
-diffusion models are learned with a fixed procedure and the latent
-variable has high dimensionality (same as the original data).
-</p>
-
-<!-- !split -->
-<h2 id="problems-with-probabilistic-models" class="anchor">Problems with probabilistic models </h2>
-
-<p>Historically, probabilistic models suffer from a tradeoff between two
-conflicting objectives: \textit{tractability} and
-\textit{flexibility}. Models that are \textit{tractable} can be
-analytically evaluated and easily fit to data (e.g. a Gaussian or
-Laplace). However, these models are unable to aptly describe structure
-in rich datasets. On the other hand, models that are \textit{flexible}
-can be molded to fit structure in arbitrary data. For example, we can
-define models in terms of any (non-negative) function \( \phi(\boldsymbol{x}) \)
-yielding the flexible distribution \( p\left(\boldsymbol{x}\right) =
-\frac{\phi\left(\boldsymbol{x} \right)}{Z} \), where \( Z \) is a normalization
-constant. However, computing this normalization constant is generally
-intractable. Evaluating, training, or drawing samples from such
-flexible models typically requires a very expensive Monte Carlo
-process.
-</p>
-
-<!-- !split -->
-<h2 id="diffusion-models" class="anchor">Diffusion models </h2>
-<p>Diffusion models have several interesting features</p>
-<ul>
-<li> extreme flexibility in model structure,</li>
-<li> exact sampling,</li>
-<li> easy multiplication with other distributions, e.g. in order to compute a posterior, and</li>
-<li> the model log likelihood, and the probability of individual states, to be cheaply evaluated.</li>
-</ul>
-<!-- !split -->
-<h2 id="original-idea" class="anchor">Original idea </h2>
-
-<p>In the original formulation, one uses a Markov chain to gradually
-convert one distribution into another, an idea used in non-equilibrium
-statistical physics and sequential Monte Carlo. Diffusion models build
-a generative Markov chain which converts a simple known distribution
-(e.g. a Gaussian) into a target (data) distribution using a diffusion
-process. Rather than use this Markov chain to approximately evaluate a
-model which has been otherwise defined, one can  explicitly define the
-probabilistic model as the endpoint of the Markov chain. Since each
-step in the diffusion chain has an analytically evaluable probability,
-the full chain can also be analytically evaluated.
-</p>
-
-<!-- !split -->
-<h2 id="diffusion-learning" class="anchor">Diffusion learning </h2>
-
-<p>Learning in this framework involves estimating small perturbations to
-a diffusion process. Estimating small, analytically tractable,
-perturbations is more tractable than explicitly describing the full
-distribution with a single, non-analytically-normalizable, potential
-function.  Furthermore, since a diffusion process exists for any
-smooth target distribution, this method can capture data distributions
-of arbitrary form.
-</p>
-
 <!-- !split -->
 <h2 id="diffusion-models-basics" class="anchor">Diffusion models, basics </h2>
 

doc/pub/week13/html/week13-reveal.html

@@ -1348,78 +1348,6 @@ <h2 id="diffusion-learning">Diffusion learning </h2>
 </p>
 </section>
 
-<section>
-<h2 id="diffusion-models-basics">Diffusion models, basics </h2>
-
-<p>Diffusion models are inspired by non-equilibrium thermodynamics. They
-define a Markov chain of diffusion steps to slowly add random noise to
-data and then learn to reverse the diffusion process to construct
-desired data samples from the noise. Unlike VAE or flow models,
-diffusion models are learned with a fixed procedure and the latent
-variable has high dimensionality (same as the original data).
-</p>
-</section>
-
-<section>
-<h2 id="problems-with-probabilistic-models">Problems with probabilistic models </h2>
-
-<p>Historically, probabilistic models suffer from a tradeoff between two
-conflicting objectives: \textit{tractability} and
-\textit{flexibility}. Models that are \textit{tractable} can be
-analytically evaluated and easily fit to data (e.g. a Gaussian or
-Laplace). However, these models are unable to aptly describe structure
-in rich datasets. On the other hand, models that are \textit{flexible}
-can be molded to fit structure in arbitrary data. For example, we can
-define models in terms of any (non-negative) function \( \phi(\boldsymbol{x}) \)
-yielding the flexible distribution \( p\left(\boldsymbol{x}\right) =
-\frac{\phi\left(\boldsymbol{x} \right)}{Z} \), where \( Z \) is a normalization
-constant. However, computing this normalization constant is generally
-intractable. Evaluating, training, or drawing samples from such
-flexible models typically requires a very expensive Monte Carlo
-process.
-</p>
-</section>
-
-<section>
-<h2 id="diffusion-models">Diffusion models </h2>
-<p>Diffusion models have several interesting features</p>
-<ul>
-<p><li> extreme flexibility in model structure,</li>
-<p><li> exact sampling,</li>
-<p><li> easy multiplication with other distributions, e.g. in order to compute a posterior, and</li>
-<p><li> the model log likelihood, and the probability of individual states, to be cheaply evaluated.</li>
-</ul>
-</section>
-
-<section>
-<h2 id="original-idea">Original idea </h2>
-
-<p>In the original formulation, one uses a Markov chain to gradually
-convert one distribution into another, an idea used in non-equilibrium
-statistical physics and sequential Monte Carlo. Diffusion models build
-a generative Markov chain which converts a simple known distribution
-(e.g. a Gaussian) into a target (data) distribution using a diffusion
-process. Rather than use this Markov chain to approximately evaluate a
-model which has been otherwise defined, one can  explicitly define the
-probabilistic model as the endpoint of the Markov chain. Since each
-step in the diffusion chain has an analytically evaluable probability,
-the full chain can also be analytically evaluated.
-</p>
-</section>
-
-<section>
-<h2 id="diffusion-learning">Diffusion learning </h2>
-
-<p>Learning in this framework involves estimating small perturbations to
-a diffusion process. Estimating small, analytically tractable,
-perturbations is more tractable than explicitly describing the full
-distribution with a single, non-analytically-normalizable, potential
-function.  Furthermore, since a diffusion process exists for any
-smooth target distribution, this method can capture data distributions
-of arbitrary form.
-</p>
-</section>
-
 <section>
 <h2 id="mathematics-of-diffusion-models">Mathematics of diffusion models </h2>
 

doc/pub/week13/html/week13-solarized.html

@@ -159,14 +159,6 @@
               ('Diffusion models', 2, None, 'diffusion-models'),
               ('Original idea', 2, None, 'original-idea'),
               ('Diffusion learning', 2, None, 'diffusion-learning'),
-              ('Diffusion models, basics', 2, None, 'diffusion-models-basics'),
-              ('Problems with probabilistic models',
-               2,
-               None,
-               'problems-with-probabilistic-models'),
-              ('Diffusion models', 2, None, 'diffusion-models'),
-              ('Original idea', 2, None, 'original-idea'),
-              ('Diffusion learning', 2, None, 'diffusion-learning'),
               ('Mathematics of diffusion models',
                2,
                None,
@@ -1242,72 +1234,6 @@ <h2 id="code-in-pytorch-for-vaes">Code in PyTorch for VAEs </h2>
 </div>
 
 
-<!-- !split --><br><br><br><br><br><br><br><br><br><br>
-<h2 id="diffusion-models-basics">Diffusion models, basics </h2>
-
-<p>Diffusion models are inspired by non-equilibrium thermodynamics. They
-define a Markov chain of diffusion steps to slowly add random noise to
-data and then learn to reverse the diffusion process to construct
-desired data samples from the noise. Unlike VAE or flow models,
-diffusion models are learned with a fixed procedure and the latent
-variable has high dimensionality (same as the original data).
-</p>
-
-<!-- !split --><br><br><br><br><br><br><br><br><br><br>
-<h2 id="problems-with-probabilistic-models">Problems with probabilistic models </h2>
-
-<p>Historically, probabilistic models suffer from a tradeoff between two
-conflicting objectives: \textit{tractability} and
-\textit{flexibility}. Models that are \textit{tractable} can be
-analytically evaluated and easily fit to data (e.g. a Gaussian or
-Laplace). However, these models are unable to aptly describe structure
-in rich datasets. On the other hand, models that are \textit{flexible}
-can be molded to fit structure in arbitrary data. For example, we can
-define models in terms of any (non-negative) function \( \phi(\boldsymbol{x}) \)
-yielding the flexible distribution \( p\left(\boldsymbol{x}\right) =
-\frac{\phi\left(\boldsymbol{x} \right)}{Z} \), where \( Z \) is a normalization
-constant. However, computing this normalization constant is generally
-intractable. Evaluating, training, or drawing samples from such
-flexible models typically requires a very expensive Monte Carlo
-process.
-</p>
-
-<!-- !split --><br><br><br><br><br><br><br><br><br><br>
-<h2 id="diffusion-models">Diffusion models </h2>
-<p>Diffusion models have several interesting features</p>
-<ul>
-<li> extreme flexibility in model structure,</li>
-<li> exact sampling,</li>
-<li> easy multiplication with other distributions, e.g. in order to compute a posterior, and</li>
-<li> the model log likelihood, and the probability of individual states, to be cheaply evaluated.</li>
-</ul>
-<!-- !split --><br><br><br><br><br><br><br><br><br><br>
-<h2 id="original-idea">Original idea </h2>
-
-<p>In the original formulation, one uses a Markov chain to gradually
-convert one distribution into another, an idea used in non-equilibrium
-statistical physics and sequential Monte Carlo. Diffusion models build
-a generative Markov chain which converts a simple known distribution
-(e.g. a Gaussian) into a target (data) distribution using a diffusion
-process. Rather than use this Markov chain to approximately evaluate a
-model which has been otherwise defined, one can  explicitly define the
-probabilistic model as the endpoint of the Markov chain. Since each
-step in the diffusion chain has an analytically evaluable probability,
-the full chain can also be analytically evaluated.
-</p>
-
-<!-- !split --><br><br><br><br><br><br><br><br><br><br>
-<h2 id="diffusion-learning">Diffusion learning </h2>
-
-<p>Learning in this framework involves estimating small perturbations to
-a diffusion process. Estimating small, analytically tractable,
-perturbations is more tractable than explicitly describing the full
-distribution with a single, non-analytically-normalizable, potential
-function.  Furthermore, since a diffusion process exists for any
-smooth target distribution, this method can capture data distributions
-of arbitrary form.
-</p>
-
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
 <h2 id="diffusion-models-basics">Diffusion models, basics </h2>
 

doc/pub/week13/html/week13.html

@@ -236,14 +236,6 @@
               ('Diffusion models', 2, None, 'diffusion-models'),
               ('Original idea', 2, None, 'original-idea'),
               ('Diffusion learning', 2, None, 'diffusion-learning'),
-              ('Diffusion models, basics', 2, None, 'diffusion-models-basics'),
-              ('Problems with probabilistic models',
-               2,
-               None,
-               'problems-with-probabilistic-models'),
-              ('Diffusion models', 2, None, 'diffusion-models'),
-              ('Original idea', 2, None, 'original-idea'),
-              ('Diffusion learning', 2, None, 'diffusion-learning'),
               ('Mathematics of diffusion models',
                2,
                None,
@@ -1319,72 +1311,6 @@ <h2 id="code-in-pytorch-for-vaes">Code in PyTorch for VAEs </h2>
 </div>
 
 
-<!-- !split --><br><br><br><br><br><br><br><br><br><br>
-<h2 id="diffusion-models-basics">Diffusion models, basics </h2>
-
-<p>Diffusion models are inspired by non-equilibrium thermodynamics. They
-define a Markov chain of diffusion steps to slowly add random noise to
-data and then learn to reverse the diffusion process to construct
-desired data samples from the noise. Unlike VAE or flow models,
-diffusion models are learned with a fixed procedure and the latent
-variable has high dimensionality (same as the original data).
-</p>
-
-<!-- !split --><br><br><br><br><br><br><br><br><br><br>
-<h2 id="problems-with-probabilistic-models">Problems with probabilistic models </h2>
-
-<p>Historically, probabilistic models suffer from a tradeoff between two
-conflicting objectives: \textit{tractability} and
-\textit{flexibility}. Models that are \textit{tractable} can be
-analytically evaluated and easily fit to data (e.g. a Gaussian or
-Laplace). However, these models are unable to aptly describe structure
-in rich datasets. On the other hand, models that are \textit{flexible}
-can be molded to fit structure in arbitrary data. For example, we can
-define models in terms of any (non-negative) function \( \phi(\boldsymbol{x}) \)
-yielding the flexible distribution \( p\left(\boldsymbol{x}\right) =
-\frac{\phi\left(\boldsymbol{x} \right)}{Z} \), where \( Z \) is a normalization
-constant. However, computing this normalization constant is generally
-intractable. Evaluating, training, or drawing samples from such
-flexible models typically requires a very expensive Monte Carlo
-process.
-</p>
-
-<!-- !split --><br><br><br><br><br><br><br><br><br><br>
-<h2 id="diffusion-models">Diffusion models </h2>
-<p>Diffusion models have several interesting features</p>
-<ul>
-<li> extreme flexibility in model structure,</li>
-<li> exact sampling,</li>
-<li> easy multiplication with other distributions, e.g. in order to compute a posterior, and</li>
-<li> the model log likelihood, and the probability of individual states, to be cheaply evaluated.</li>
-</ul>
-<!-- !split --><br><br><br><br><br><br><br><br><br><br>
-<h2 id="original-idea">Original idea </h2>
-
-<p>In the original formulation, one uses a Markov chain to gradually
-convert one distribution into another, an idea used in non-equilibrium
-statistical physics and sequential Monte Carlo. Diffusion models build
-a generative Markov chain which converts a simple known distribution
-(e.g. a Gaussian) into a target (data) distribution using a diffusion
-process. Rather than use this Markov chain to approximately evaluate a
-model which has been otherwise defined, one can  explicitly define the
-probabilistic model as the endpoint of the Markov chain. Since each
-step in the diffusion chain has an analytically evaluable probability,
-the full chain can also be analytically evaluated.
-</p>
-
-<!-- !split --><br><br><br><br><br><br><br><br><br><br>
-<h2 id="diffusion-learning">Diffusion learning </h2>
-
-<p>Learning in this framework involves estimating small perturbations to
-a diffusion process. Estimating small, analytically tractable,
-perturbations is more tractable than explicitly describing the full
-distribution with a single, non-analytically-normalizable, potential
-function.  Furthermore, since a diffusion process exists for any
-smooth target distribution, this method can capture data distributions
-of arbitrary form.
-</p>
-
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
 <h2 id="diffusion-models-basics">Diffusion models, basics </h2>