gp function reference docs (Issue #185) #272

bbbales2 · 2020-10-08T23:11:08Z

Submission Checklist

Builds locally
Declare copyright holder and open-source license: see below

Summary

I copy-pasted the formulas and such from the math docs and the function signatures themselves come from what the stanc3 compiler says it exposes. I moved cov_exp_quad to the deprecated functions.

Copyright and Licensing

Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company): Columbia University

By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:

Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

drezap

See comments. Thank you :)

drezap · 2020-10-09T00:39:09Z

src/functions-reference/matrix_operations.Rmd

-<!-- matrix; cov_exp_quad; (row_vectors x, real alpha, real rho); -->
-\index{{\tt \bfseries cov\_exp\_quad }!{\tt (row\_vectors x, real alpha, real rho): matrix}|hyperpage}
+$$
+k(x_i, x_j) = \alpha^2 \exp \left( -\dfrac{1}{2\rho^2} |x_i - x_j|^2 \right)


The parameters for this formula and the inputs of the math library implementation don't match, and it's a bit misleading.

Can we agree on parameter names? sigma for magnitude and l for length scale are fine, as these are both used in references, and these match the math library implementation.

I've been using tau for the magnitude parameter and l for the length-scale parameter to match BDA3 on this branch: https://github.com/stan-dev/docs/tree/issue-185, but sigma and l are fine.

Ooo, yeah, I just copy pasted that from cov_exp_quad. I'll go \sigma and l with names magnitude and length scale.

drezap · 2020-10-09T00:42:32Z

src/functions-reference/matrix_operations.Rmd

+With scale $\sigma$ and length scale $l$, the exponential kernel is:
+
+$$
+k(x_i, x_j) = \sigma^2 exp(-\frac{|x_i - x_j|}{2 l^2})


Can you use \left ( for parenthesis so size of the parenthesis is cleaner? Same for kernels below.

drezap · 2020-10-09T00:46:43Z

src/functions-reference/deprecated_functions.Rmd

+}
+```
+
+With scale $\alpha$ and length scale $l$, the exponentiated quadratic kernel is:


alpha isn't a scale parameter. It's a magnitude or marginal SD parameter. Scale and length scale are synonymous.

I'll go magnitude here. Also I mixed notations again with the l vs rho thing.

drezap · 2020-10-09T00:50:31Z

src/functions-reference/matrix_operations.Rmd

+set of points and itself, $K_{ij} = k(x_i, x_j)$, or the covariance between
+two different sets of points, $K_{ij} = k(x_i, x_j)$. $x$ can be an array of
+scalars for a one dimensional kernel or an array of vectors for a
+multidimensional kernel.


This is not clear. Something like: Gaussian process covariance functions compute the covariance between each observation in the input data set, or the cross-covariance between two input data sets. And the notation for covariance and cross covariance should be unique, in some way. Right now it's the same.

May be vector of doubles, instead of scalars. May be make x boldface so we know it's a vector?

Can you also indicate somewhere that x_i is observation (row) i of x?

Also, instead of the kernel being one-dimensional, the input data is multi-dimensional. Or the design matrix has more than one covariate. Something like that. We say the GP is 1-D, not the kernel.

drezap · 2020-10-09T00:52:50Z

src/functions-reference/matrix_operations.Rmd

+
+### Dot product kernel
+
+With scale $\sigma$ the dot product kernel is:


this isn't a scale parameter. This is an intercept or bias parameter.

Also, may be a subscript 0 for the sigma parameter to differentiate it from the magnitude parameter.

bias and \sigma_0 sound good.

drezap · 2020-10-09T00:54:17Z

src/functions-reference/matrix_operations.Rmd

+
+### Exponential kernel
+
+With scale $\sigma$ and length scale $l$, the exponential kernel is:


same here, sigma is not a scale parameter. This is the same for all matern kernels (including exponential).

Oh I see it's magnitude in the code. I'll go with that.

drezap · 2020-10-09T00:56:33Z

src/functions-reference/matrix_operations.Rmd

+With scale $\sigma$ and length scale $l$, the Matern 3/2 kernel is:
+
+$$
+k(x_i, x_j) = \sigma^2(1 + \frac{\sqrt{3}|x_i - x_j|}{l}) \exp(-\frac{\sqrt{3}|x_i - x_j|)}{l})


you're missing an opening ( in the exponential. Either remove the closing one or add an opening one.

drezap · 2020-10-09T01:09:00Z

src/functions-reference/matrix_operations.Rmd

-`matrix` **`cov_exp_quad`**`(vectors x1, vectors x2, real alpha, real rho)`<br>\newline
-The covariance matrix with an exponentiated quadratic kernel of x1 and
-x2.
+Gaussian process with squared exponential kernel in multiple dimensions.


This is picky, and optional, but I'd be explicit about the dimension of D being greater than 1.

I left it at multiple dimensions. The code accepts 1D, so it'd work if someone did it.

drezap · 2020-10-09T01:09:34Z

src/functions-reference/matrix_operations.Rmd

-x2.
+`matrix` **`gp_exp_quad_cov`**`(vector[] x, real sigma, real[] length_scale)`<br>\newline
+
+Gaussian process with squared exponential kernel in multiple dimensions with a


something like: with a separate length scale parameter for each dimension.

drezap · 2020-10-09T01:10:37Z

src/functions-reference/matrix_operations.Rmd

+
+`matrix` **`gp_exp_quad_cov`**`(vector[] x1, vector[] x2, real sigma, real length_scale)`<br>\newline
+
+Gaussian process with squared exponential kernel in multiple dimensions between


Something like: computes the cross covariance between two sets of data points x1, x2. Number of dimensions must be the same, as well.

bbbales2 · 2020-10-18T21:23:21Z

@drezap I went through and did the updates.

Here's the rendered file for your convenience: gaussian-process-covariance-functions.html.zip

Edit: I went with magnitude for the \sigmas on everything, and switched most of the xs to look like vectors. Intro I copied from what you posted and added some text to differentiate the cross-covariances from just the regular covariances.

avehtari · 2020-10-19T11:55:50Z

Quick comment: If the covariance function is named gp_exp_quad_cov, we should write in the documentation always "exponentiated quadratic" instead of "squared exponential"

bbbales2 · 2020-10-19T12:41:58Z

"exponentiated quadratic" instead of "squared exponential"

Oh yeah I changed it to squared exponential I can change it back.

bbbales2 · 2020-10-19T12:42:51Z

Oh I see. Squared exponential seems like exp(2x) where as exponentiated quadratic is like exp(x^2).

bbbales2 · 2020-10-19T13:40:50Z

Updated (and new rendered html here: gaussian-process-covariance-functions.html.zip)

drezap · 2020-10-19T21:12:36Z

thanks for the doc, I'll take a look within the next few days Best, Andre Zapico

…

On Mon, Oct 19, 2020 at 9:41 PM Ben Bales ***@***.***> wrote: Updated (and new rendered html here: gaussian-process-covariance-functions.html.zip <https://github.com/stan-dev/docs/files/5402147/gaussian-process-covariance-functions.html.zip> ) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#272 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACY543GPWHNMRTWZYQPPAGTSLQ6XHANCNFSM4SJLXYVA> .

dirmeier · 2021-06-26T20:02:58Z

Hey @bbbales2 , can I give you a hand with anything here? It would be nice if this docu were available in the official reference.
Cheers, Simon

avehtari

looks good to me (and sorry for missing the review request)

WardBrian · 2021-11-22T21:05:02Z

@bbbales2 - do you mind if I update the merge on this for you and add the 'Since version x' text to the functions? I'd like to merge this before #439

bbbales2 · 2021-11-23T04:15:16Z

Sounds good, take it away!

rok-cesnovar

The changes since Aki’s approval look good to me! Thanks Ben and Brian.

Initial attempt at documenting the gp functions (Issue #185)

579b790

bbbales2 requested a review from avehtari October 8, 2020 23:11

Updated link to cov_exp_quad docs (Issue #185)

7f1c2f3

drezap reviewed Oct 9, 2020

View reviewed changes

Updated gp section for review comments (#185)

9bf4340

Changed squared exponential to exponentiated quadratic (Issue #185)

49d0d15

avehtari approved these changes Jul 2, 2021

View reviewed changes

Merge branch 'master' into feature/gp_function_reference_docs

f934f2b

WardBrian linked an issue Nov 23, 2021 that may be closed by this pull request

update manual to include new GP covariance functions #185

Closed

WardBrian requested a review from rok-cesnovar November 23, 2021 14:15

rok-cesnovar approved these changes Nov 23, 2021

View reviewed changes

rok-cesnovar merged commit e0564d3 into master Nov 23, 2021

rok-cesnovar deleted the feature/gp_function_reference_docs branch November 23, 2021 14:18

WardBrian mentioned this pull request Dec 10, 2021

add cov_exp_quad doc #52

Closed

WardBrian mentioned this pull request Jan 29, 2022

gp_ functions exposed but not documented #478

Closed


		### Dot product kernel

		With scale $\sigma$ the dot product kernel is:


		### Exponential kernel

		With scale $\sigma$ and length scale $l$, the exponential kernel is:


		`matrix` `gp_exp_quad_cov``(vector[] x1, vector[] x2, real sigma, real length_scale)`<br>\newline

		Gaussian process with squared exponential kernel in multiple dimensions between

Uh oh!

gp function reference docs (Issue #185) #272

gp function reference docs (Issue #185) #272

Uh oh!

Conversation

bbbales2 commented Oct 8, 2020

Submission Checklist

Summary

Copyright and Licensing

Uh oh!

drezap left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bbbales2 commented Oct 18, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

avehtari commented Oct 19, 2020

Uh oh!

bbbales2 commented Oct 19, 2020

Uh oh!

bbbales2 commented Oct 19, 2020

Uh oh!

bbbales2 commented Oct 19, 2020

Uh oh!

drezap commented Oct 19, 2020 via email

Uh oh!

dirmeier commented Jun 26, 2021

Uh oh!

avehtari left a comment

Choose a reason for hiding this comment

Uh oh!

WardBrian commented Nov 22, 2021

Uh oh!

bbbales2 commented Nov 23, 2021

Uh oh!

rok-cesnovar left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

bbbales2 commented Oct 18, 2020 •

edited

Loading