Theme does not apply immediately when loading via extension

[dhrubomoy]

Syntax highlighting does not apply after the editor is rendered. But if I interact with the editor e.g. type something or scroll, only then it gets applied. I am applying custom theme that I am loading via extension in vscodeApiConfig (code here). I am loading the worker that I bundled using webpack like its mentioned in this doc.

The issue does not happen when I set 'editor.experimental.asyncTokenization': false

You can reproduce the issue in my branch here

npm install
npm run build
npm run bundle:workers --workspace packages/examples
npm run dev

Then open the statemachine react example in browser

[kaisalmen]

Hi @dhrubomoy I am able to reproduce this behaviour as well with our webpack verification example, but I can't explain it, yet. When 'editor.experimental.asyncTokenization': false is configured textmate is running in main thread. So, webpack's worker init is somewhat different. I don't observe this behaviour with vite/rollup (dev or production).

[dhrubomoy]

Thanks @kaisalmen for the reply. I had to change the entry file for the textmate worker due to the error I got that I mentioned in this thread. Here is the full webpack config

export default {
    entry: {
        editor:"../../node_modules/@codingame/monaco-vscode-editor-api/esm/vs/editor/editor.worker.js",
        // Changed following. 
        // Original path mentioned in doc: './node_modules/@codingame/monaco-vscode-textmate-service-override/worker.js'
        textmate: '../../node_modules/@codingame/monaco-vscode-textmate-service-override/vscode/src/vs/workbench/services/textMate/browser/backgroundTokenization/worker/textMateTokenizationWorker.worker.js'
    },
    output: {
        filename: "[name].js",
        path: resolve(__dirname, "./src/assets/monaco-workers"),
    },
    mode: "development",
    performance: {
        hints: false,
    },
};

[dhrubomoy]

Another solution that worked is to call forceTokenization after the editor is initialized. Code here. But it feels a bit hacky :(

<MonacoEditorReactComp
      style={{ 'height': '50vh' }}
      vscodeApiConfig={appConfig.vscodeApiConfig}
      editorAppConfig={appConfig.editorAppConfig}
      languageClientConfig={appConfig.languageClientConfig}
      onTextChanged={onTextChanged}
      onEditorStartDone={(editorApp) => {
          if (editorApp) {
              const standaloneEditor = editorApp.getEditor();
              if (standaloneEditor) {
                  setTimeout(() => {
                      const model = standaloneEditor.getModel() as unknown as {
                          tokenization: {
                              forceTokenization: () => void;
                          };
                      };
                      model.tokenization.forceTokenization();
                  }, 500);
              }
          }
      }}
  />

[zerocewl]

Same here, issue comes up with latest update (from 20.10.25). Using webpack too.

Checked that 'editor.experimental.asyncTokenization': false can be used as workaround.

[dhrubomoy]

Unfortunately for us its not an option :(
Setting 'editor.experimental.asyncTokenization': false leads to performance issues as mentioned here.