我好奇这个损失图的意义。I am curious about the significance of this loss plot.

[DDDMUC]

这个图的浅线是engram的成绩吗？
这个图的深线是lora的成绩吗？

请问对于这个图，你们实验里engram模块的参数是多少m？
请问对于这个图，你们实验里lora模块的参数是多少m？

Does the light-colored line in this graph represent the performance of Engram?
Does the dark-colored line represent the performance of LoRA?
Additionally, regarding this specific plot:
What was the parameter count (in millions/M) for the Engram module used in your experiment?
What was the parameter count (in millions/M) for the LoRA module used in your experiment?"

[Alichua]

Hi, DDMUC,

Thanks for asking.

In Key Finding 1, we do not compare Engram with LoRA. Therefore, the graph you're referring to only shows the convergence curve of Engram. The light- and dark-colored lines are generated by TensorBoard's "smoothing" toggle.

As stated in our documentation, this loss plot—paired with the training loss figure—is intended to demonstrate the robust convergence behavior of Engram as an SFT method.

The param design correspond to this graph can be found in the description of the Anti-catastrophic Forgetting experiment, since the model configuration is shared. The layers of injection are L5/L7/L13/L17 with Qwen3-0.6B as backbone.

While in TinyEngram experiments we typically focus more on the Engram vocabulary size and the number of layers into which it is injected—rather than the total flattened parameter count—you can use the formula provided in this issue comment to estimate the approximate parameter count for any given configuration.