use litedown for rendering vignettes

[ben-schwen]

Closes #6394
Towards #6584

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.97%. Comparing base (6f8bb9e) to head (cda54e5).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6583      +/-   ##
==========================================
- Coverage   98.97%   98.97%   -0.01%     
==========================================
  Files          87       87              
  Lines       16741    16739       -2     
==========================================
- Hits        16569    16567       -2     
  Misses        172      172              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ben-schwen]

This PR is on hold until current GH version of xfun makes its way to CRAN.

[MichaelChirico]

Earmark: after #6589, if {knitr} is not in Suggests it should be in Enhances.

[MichaelChirico]

NB: After #6618, we'll also want to check this issue whether it should be included, or left as a TODO: yihui/litedown#67

[yihui]

NB: After #6618, we'll also want to check this issue whether it should be included, or left as a TODO: yihui/litedown#67

It's safe to use litedown:::.env$input with the current CRAN version of litedown. The next CRAN release should take place in a few weeks, and you'll be able to use litedown::get_context('input').

[rikivillalba]

Sorry I meant to push a sub-branch to merge but instead forget to switch and pushed the commit directly.
The commit updates the translation script with litedown and changes instances of `r ...` to `{r} ...` as is recognized by litedown

[jangorecki]

xfun is fixed on CRAN so we could proceed with this issue, isn't it? @ben-schwen

[ben-schwen]

xfun is fixed on CRAN so we could proceed with this issue, isn't it? @ben-schwen

Yes. Only blocker, I'm aware of now is that := is printing on litedown and not printing on knitr

[aitap]

This can be solved similarly to #6589, except that methods for xfun::record_print must return the printed output, not print it:

library(data.table)
library(xfun)
x <- data.table(a = 1)
xfun::record('x[,a:=1]') |> print(browse = FALSE)
# x[,a:=1]
# 
# #>        a
# #>    <num>
# #> 1:     1
#
.S3method(
  'record_print', 'data.table',
  \(x, ...) if (!shouldPrint(x)) character() else NextMethod()
)
xfun::record('x[,a:=1]') |> print(browse = FALSE)
# x[,a:=1]
# 
xfun::record('x[,a:=1][]') |> print(browse = FALSE)
# x[,a:=1][]
# 
# #>        a
# #>    <num>
# #> 1:     1
# 

Additionally, inline expressions of the form `r ... ` aren't parsed the same way. For example, data.table-intro.Rmd uses

In (1), for each group, a vector is returned, with length = 6,4,2 here. However, (2) returns a list of length 1 for each group, with its first element holding vectors of length 6,4,2. Therefore, (1) results in a length of ` 6+4+2 = `r 6+4+2``, whereas (2) returns `1+1+1=`r 1+1+1``.

...and with litedown, the inline expressions inside code blocks are not evaluated.

At some point we'll need to transition the translated vignettes too, right?

[aitap]

In datatable-intro.Rmd, by-group j-expressions that call print() render like:

DT[, print(.SD), by = ID]

#        a     b     c
#    <int> <int> <int>
# 1:     1     7    13
# 2:     2     8    14
# 3:     3     9    15
#        a     b     c
#    <int> <int> <int>
# 1:     4    10    16
# 2:     5    11    17
#        a     b     c
#    <int> <int> <int>
# 1:     6    12    18

ID

I wonder where is the last ID coming from.

---

I couldn't make the `some text `r expression(...)`` syntax work in Litedown and wrote a replacement function for datatable-intro.Rmd.

---

In datatable-reference-semantics.Rmd, the CSS class syntax #### {.bs-callout .bs-callout-info} does not work; it produces a level-4 heading titled {.bs-callout .bs-callout-info} instead.

[MichaelChirico]

NB: please Update branch before merging & possibly include #7151.

[aitap]

Unfortunately, pkgdown::build_site does not understand the new `{r} expression...` syntax in the vignettes, so the translation marks are lost, and so is other dynamic text.

[MichaelChirico]

Per the {litedown} README, {litedown} itself is supposed to do the site rendering, right?

Otherwise, we should ask the {pkgdown} maintainers to support {litedown} rendering; IINM, the issue is that {pkgdown} relies on {rmarkdown} to render:

https://github.com/r-lib/pkgdown/blob/824ab4a4f20ecd895282b434d8652120e1f62ba4/R/build-article.R#L294
https://github.com/r-lib/pkgdown/blob/824ab4a4f20ecd895282b434d8652120e1f62ba4/R/build-article.R#L114

[jangorecki]

It does support basic website.
https://github.com/jangorecki/pkgup/blob/a1ec87d02209423a28c432c92b7269a086cd3e7b/.github/workflows/pkgup.yaml#L56
https://jangorecki.github.io/pkgup/

pkgdown could support it anyway, but not sure if and how fast that will happen.
pkgdown did not support well markdown vignettes in the past, only rmarkdown.

[ChristianWia]

comparing the results of html vignette generation with knitr and litedown on "data-table-joins.Rmd" :
litedownKnitrCompared.txt

knitr/litedown diff screenshot section 7.2:

litedown resulting file to follow the comments:
litedownJointuresavecdatatable.pdf

My Windows environment :

R version 4.0.2 (2020-06-22) -- "Taking Off Again"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

installation from CRAN source packages : ‘commonmark’, ‘xfun’, ‘litedown’

URL 'https://pbil.univ-lyon1.fr/CRAN/src/contrib/commonmark_2.0.0.tar.gz'
URL 'https://pbil.univ-lyon1.fr/CRAN/src/contrib/xfun_0.52.tar.gz'
URL 'https://pbil.univ-lyon1.fr/CRAN/src/contrib/litedown_0.7.tar.gz'

[jangorecki]

comparing the litedownKnitrCompared.txt results of html vignette generation with knitr and litedown on "data-table-joins.Rmd"

This may be as well interesting for @yihui

[ben-schwen]

I have exchanged the knitr mock with an litedown mock, however, currently it only prints with the comment char via record_print.data.table when called with print.

[ChristianWia]

I have exchanged the knitr mock with an litedown mock, however, currently it only prints with the comment char via record_print.data.table when called with print.

hi, @ben-schwen can you please attach the generated .html to follow evolution. Thanks.

[ChristianWia]

Some requirements when retrying with litedown :

(1) Rstudio and knit button generates :
==> rmarkdown::render('E:/Downloads/datatable-benchmarking.Rmd', encoding = 'UTF-8');
Erreur : It appears that you clicked the 'Knit' button in RStudio to render the document. You are recommended to use litedown::roam() to preview or render documents instead. Alternatively, you can add a top-level field 'knit: litedown:::knit' to the YAML metadata, so the document can be rendered by litedown::fuse() instead of rmarkdown::render().
Exécution arrêtée

(2) so needed to add a line knit: litedown:::knit in the source yaml document to be compatible with knit
Then ('E:/git/datatable/vignettes/datatable-benchmarking.Rmd') is generated ok with button

(3) the links to the translated vignettes FR RU .... are still missing in the HTML document due to line :
{r} .write.translation.links("Translations of this document are available in: %s")

(4) once html was generated, a new working directory is created each time in ./vignettes and remains empty needing its manual delete :
E:\git\datatable\vignettes\datatable-benchmarking__files

(5) loaded document:
in R console litedown::fuse('E:/git/datatable/vignettes/datatable-benchmarking.Rmd') is ok

[ChristianWia]

my review completed.