chore(website): migrate MDX heading ids to comment syntax + upgrade Crowdin parser version (facebook#11779)

Author: slorber

crowdin-v2.yaml

@@ -1,15 +1,9 @@
-#
-# Your Crowdin credentials
-#
+# Crowdin credentials
 project_id: '428890'
 api_token_env: CROWDIN_PERSONAL_TOKEN
 # base_path: '.'
 # base_url: https://api.crowdin.com
 
-#
-# Choose file structure in Crowdin
-# e.g. true or false
-#
 preserve_hierarchy: true
 
 # We generally want to use the "two-letters-code" of a locale (ie the language)
@@ -20,18 +14,24 @@ languages_mapping: &languages_mapping
   two_letters_code:
     pt-BR: pt-BR
 
-# Crowdin regularly update their MDX parser
-# Unfortunately, their v2 parser is more "MDX compliant" and thus can't parse
-# Docusaurus MDX files correctly due to our custom {#headingId} syntax.
-# Adding this type param permits using their older v1.2 parser.
-# Note: you can find the version of a file using browser DevTools
-# The source file icons will have a class such as "file_type_mdx_v1_2"
-#
-# TODO fix our headingId syntax
-# providing an explicit type is annoying and not future-proof
-# there's a risk that when adding an image in /docs, it will be parsed as mdx
-# and duplicating source file configs for various extensions is not great either
-mdx_file_type: &mdx_file_type mdx_v1_2
+# Crowdin regularly update their MDX parser, leading to subtle breakage.
+# Freezing the parser to a specific version is helpful to ensure things keep
+# working when they upgrade their MDX parser.
+# See https://github.com/facebook/docusaurus/pull/11432
+#
+# Note: you can find the parser version of a Crowdin file using browser DevTools
+# and inspecting the MDX file icon, it should have a class ".file_type_mdx_v1_2"
+# Find the latest version: manually upload a new .mdx file, then inspect it
+#
+# How to upgrade the parser version?
+# - Rename /docs to /docs_backup on Crowdin UI (same for other mdx folders)
+# - Update the parser version of this config file
+# - Use the CLI to re-upload the MDX source files: they should now have the new parser version
+# - Use the CLI to download the translations
+# - Build the full i18n site to ensure it still works (the new parser might break things)
+# - Make sure the site content remains translated (normally /docs & /docs_backup share the translation strings)
+# - If things work well, you can merge the config update and delete the mdx backup folders on Crowdin UI
+mdx_file_type: &mdx_file_type mdx_v2_4
 
 #
 # Files configuration
@@ -71,100 +71,3 @@ files:
     translation: /website/i18n/%two_letters_code%/docusaurus-plugin-content-pages/**/%original_file_name%
     ignore: [/**/*.js, /**/*.jsx, /**/*.ts, /**/*.tsx, /**/*.css]
     languages_mapping: *languages_mapping
-#
-# Source files filter
-# e.g. "/resources/en/*.json"
-#
-#"source" : "/website/docs/**/*.md",
-#
-# Where translations will be placed
-# e.g. "/resources/docs/%two_letters_code%/%original_file_name%"
-#
-#"translation" : "/website/i18n/%language%/docs/current/%original_file_name%",
-#
-# Files or directories for ignore
-# e.g. ["/**/?.txt", "/**/[0-9].txt", "/**/*\?*.txt"]
-#
-#"ignore" : [],
-#
-# The dest allows you to specify a file name in Crowdin
-# e.g. "/messages.json"
-#
-#"dest" : "",
-#
-# File type
-# e.g. "json"
-#
-#"type" : "",
-#
-# The parameter "update_option" is optional. If it is not set, after the files update the translations for changed strings will be removed. Use to fix typos and for minor changes in the source strings
-# e.g. "update_as_unapproved" or "update_without_changes"
-#
-#"update_option" : "",
-#
-# Start block (for XML only)
-#
-#
-# Defines whether to translate tags attributes.
-# e.g. 0 or 1  (Default is 1)
-#
-# "translate_attributes" : 1,
-#
-# Defines whether to translate texts placed inside the tags.
-# e.g. 0 or 1 (Default is 1)
-#
-# "translate_content" : 1,
-#
-# This is an array of strings, where each item is the XPaths to DOM element that should be imported
-# e.g. ["/content/text", "/content/text[@value]"]
-#
-# "translatable_elements" : [],
-#
-# Defines whether to split long texts into smaller text segments
-# e.g. 0 or 1 (Default is 1)
-#
-# "content_segmentation" : 1,
-#
-# End block (for XML only)
-#
-#
-# Start .properties block
-#
-#
-# Defines whether single quote should be escaped by another single quote or backslash in exported translations
-# e.g. 0 or 1 or 2 or 3 (Default is 3)
-# 0 - do not escape single quote;
-# 1 - escape single quote by another single quote;
-# 2 - escape single quote by backslash;
-# 3 - escape single quote by another single quote only in strings containing variables ( {0} ).
-#
-# "escape_quotes" : 3,
-#
-# Defines whether any special characters (=, :, ! and #) should be escaped by backslash in exported translations.
-# e.g. 0 or 1 (Default is 0)
-# 0 - do not escape special characters
-# 1 - escape special characters by a backslash
-#
-# "escape_special_characters": 0
-#
-#
-# End .properties block
-#
-#
-# Often software projects have custom names for the directories where translations are placed. crowdin-cli allows you to map your own languages to be understandable by Crowdin.
-#
-#"languages_mapping" : {
-#  "two_letters_code" : {
-#    "crowdin_language_code" : "local_name"
-#   }
-#},
-#
-# Does the first line contain header?
-# e.g. true or false
-#
-#"first_line_contains_header" : true,
-#
-# for spreadsheets
-# e.g. "identifier,source_phrase,context,uk,ru,fr"
-#
-# "scheme" : "",

packages/docusaurus/src/commands/writeHeadingIds.ts

@@ -109,6 +109,8 @@ export async function writeHeadingIds(
   } else {
     logger.warn`number=${
       markdownFiles.length
-    } Markdown files already have explicit heading IDs. If you intend to overwrite the existing heading IDs, use the code=${'--overwrite'} option.`;
+    } Markdown files already have explicit heading IDs.
+If you intend to overwrite the existing heading IDs, use the code=${'--overwrite'} option.
+If you intend to change their heading ID syntax, use the code=${'--migrate'} option.`;
   }
 }

website/_dogfooding/_docs tests/tests/links/target.mdx

@@ -6,4 +6,4 @@ slug: target-doc-slug
 
 This is just a doc meant to be linked to by other docs.
 
-## Target heading {#target-heading}
+## Target heading \{#target-heading}

website/_dogfooding/_pages tests/markdown-tests-mdx.mdx

@@ -188,17 +188,19 @@ sequenceDiagram
     Bob-->>John: Jolly good!
 ```
 
-## Custom heading ID {#custom}
+## Classic heading ID syntax \{#classic-heading-id-syntax}
 
-### Weird heading {#你好}
+Dogfood that the classic heading ID syntax `{#id}` remains supported on MDX docs. It requires manual escaping with `\{#id}` since we disabled the MDX v1 compat layer (`markdown.mdx1Compat.headingIds`).
 
-### Weird heading {#2022.1.1}
+### Weird heading \{#你好}
 
-### Weird heading {#a#b}
+### Weird heading \{#2022.1.1}
 
-### Weird heading {#a b}
+### Weird heading \{#a#b}
 
-### Weird heading {#a\{b}
+### Weird heading \{#a b}
+
+### Weird heading \{#a\{b}
 
 ## Pipe
 

website/blog/2017/12-14-introducing-docusaurus.mdx

@@ -15,13 +15,13 @@ We created [Docusaurus](https://docusaurus.io) for the following reasons:
 1. To make it easy to push updates, new features, and bug fixes to everyone all at once.
 1. And, finally, to provide a consistent look and feel across all of our open source projects.
 
-<!--truncate-->
+{/* truncate */}
 
 Docusaurus is a tool designed to make it easy for teams to publish documentation websites without having to worry about the infrastructure and design details. At its core, all a user has to provide are documentation files written in Markdown, customization of a provided home page written in React, and a few configuration modifications. Docusaurus handles the rest by providing default styles, site formatting, and simple document navigation. Getting started is easy, as users can [install](https://v1.docusaurus.io/docs/en/installation.html) it using `npm` or `yarn` via a simple initialization script that [creates a working example website out of the box](https://v1.docusaurus.io/docs/en/site-preparation.html).
 
 Docusaurus also provides core website and documentation features out-of-the-box including [blog support](https://v1.docusaurus.io/docs/en/blog.html), [internationalization](https://v1.docusaurus.io/docs/en/translation.html), [search](https://v1.docusaurus.io/docs/en/search.html), and [versioning](https://v1.docusaurus.io/docs/en/versioning.html). While some projects may not require any of these features, enabling them is generally a matter of updating configuration options instead of having to add the infrastructure from the ground up. As more features get added to Docusaurus, users just can easily update to the latest version. This can be done by simply running npm or yarn update and updating configuration options. Users or teams will no longer need to manually rework their entire website infrastructure each time a new feature gets added.
 
-## The Birth of docusaurus
+## The Birth of docusaurus {/* #the-birth-of-docusaurus */}
 
 ![](/img/slash-birth.png)
 
@@ -37,7 +37,7 @@ Docusaurus was born!
 
 At Facebook, Docusaurus allows us to quickly get different projects up and running with documentation websites, especially for teams who don't have much experience with web development or primarily want a basic site to showcase their project. Docusaurus already supports sites needing more advanced features like internationalization for Jest and versioning for React Native. As different projects request new features for their sites, they are added to Docusaurus and simultaneously provided to all projects! All together, this ends up greatly reducing the work needed to maintain different sites for different projects. Our teams are able to focus on keeping their projects healthier by spending more time adding features, fixing bugs, and writing documentation.
 
-## Getting Up and Running
+## Getting Up and Running {/* #getting-up-and-running */}
 
 ![](/img/slash-up-and-running.png)
 
@@ -69,7 +69,7 @@ root-of-repo
 
 With the exception of node_modules and package.json, all the directories and files you see are where you customize and add content to your Docusaurus-based website. The docs folder is where you add your Markdown that represents your documentation; the blog folder is where you add your Markdown for your [blog posts](https://v1.docusaurus.io/docs/en/blog.html); `siteConfig.js` is where you make most of the [customizations](https://v1.docusaurus.io/docs/en/site-config.html) for your site; `sidebars.json` is where you maintain the layout and content of the [sidebar](https://v1.docusaurus.io/docs/en/navigation.html) for your documentation; the `pages` folder is where you add [custom](https://v1.docusaurus.io/docs/en/custom-pages.html) pages for your site; the `static` folder is where all of your static assets go (e.g., CSS stylesheets and images); and the `core` folder is where you can customize core components of the site, in this case the footer.
 
-## How does Docusaurus work?
+## How does Docusaurus work? {/* #how-does-docusaurus-work */}
 
 Docusaurus is written primarily in JavaScript and [React](https://facebook.github.io/react), replacing Jekyll which we used in the old template. We use [Remarkable](https://github.com/jonschlinkert/remarkable) for our Markdown rendering and [highlight.js](https://highlightjs.org/) for our code block syntax highlighting. The core of Docusaurus' functionality is in the [lib directory](https://github.com/facebookexperimental/Docusaurus/tree/master/lib) of the [Docusaurus repo](https://github.com/facebook/docusaurus/). The general structure looks like:
 
@@ -125,13 +125,13 @@ build
 │   └── users.html # custom page
 ```
 
-## Community
+## Community {/* #community */}
 
 ![](/img/docusaurus.svg)
 
 We welcome your [contributions](https://github.com/facebook/docusaurus/blob/master/CONTRIBUTING.md) to Docusaurus, whether you want to use it for your own site, you want to [contribute](https://github.com/facebook/docusaurus/blob/master/CONTRIBUTING.md) to the Docusaurus core or just have questions. Follow us on [GitHub](https://github.com/facebook/docusaurus) and [X)](https://x.com/docusaurus).
 
-## Acknowledgements
+## Acknowledgements {/* #acknowledgements */}
 
 Docusaurus wouldn't exist without the work of the rest of the core Docusaurus team: [Eric Nakagawa](http://x.com/ericnakagawa), [Hector Ramos](https://x.com/hectorramos), [Eric Vicenti](https://x.com/EricVicenti) and [Frank Li](https://github.com/deltice) — a former intern at Facebook who implemented the core technology and features.
 
@@ -148,7 +148,7 @@ Special thanks also goes out to our earliest [adopters](https://v1.docusaurus.io
 
 Without their dedication to creating or migrating their websites over to the platform, we would have not have been in the position where we are today.
 
-## Resources
+## Resources {/* #resources */}
 
 - [Read our documentation](https://v1.docusaurus.io)
 - [Follow our X feed](https://x.com/docusaurus)

website/blog/2018/04-30-How-I-Converted-Profilo-To-Docusaurus.mdx

@@ -12,7 +12,7 @@ This is the story of the rather short journey it took to create the [Profilo](ht
 
 Profilo, an Android library for collecting performance traces from production, [was announced](https://code.fb.com/android/profilo-understanding-app-performance-in-the-wild/) earlier this year. The project was [published on GitHub](https://github.com/facebookincubator/profilo/tree/802042f90f990998a272387e371b893af52465b8) with a less than [a handful or Markdown files](https://github.com/facebookincubator/profilo/tree/802042f90f990998a272387e371b893af52465b8/docs) to describe its functionality and no website to showcase any branding and highlight the logo. The task at hand was to turn these existing docs and logo into a website.
 
-<!--truncate-->
+{/* truncate */}
 
 In general, when creating a website with Docusaurus you do the following:
 
@@ -26,7 +26,7 @@ In general, when creating a website with Docusaurus you do the following:
 
 Given I had pre-existing Markdown files, I didn't have to generate the core content but simply make sure that Docusaurus could process the files by adding the expected metadata to them. Most of the work would therefore consist of customizing the defaults provided by Docusaurus.
 
-## Overview of Steps Taken
+## Overview of Steps Taken {/* #overview-of-steps-taken */}
 
 Here's an overview of the steps taken to convert to a website. I'll discuss some of the design aspects in a later section.
 
@@ -83,7 +83,7 @@ Here's an overview of the steps taken to convert to a website. I'll discuss some
 
 The final website was published on https://facebookincubator.github.io/profilo/. It had taken 1.5 hours to get to the initial PR stage and another half an hour or so to respond to review feedback and publish the website.
 
-## Design
+## Design {/* #design */}
 
 Here's what the initial website looked like when the first pull request was sent out:
 
@@ -115,7 +115,7 @@ Lastly, I didn't have to worry about handling responsive design. You get this ou
 
 ![Mobile screenshots of the front page and sample doc page. The layout is automatically adjusted to make it appear more natural. The doc sidebar is hidden behind a button.](/img/profilo_blog_post_android_ios.png)
 
-## Final Thoughts
+## Final Thoughts {/* #final-thoughts */}
 
 The Profilo engineers were happy to see that they didn't have to change their workflow to update existing content. They were able to continue working with Markdown files. This will still be true in the future if new docs are added, although there may be some config changes needed if the sidebar navigation needs to be updated.