Data validation and sources

[therealnb]

We need to check further sources of good and bad commands

e.g. https://github.com/TellinaTool/nl2bash/blob/master/data/bash/all.cm, https://gitlab.ics.muni.cz/muni-kypo-trainings/datasets/commands/-/tree/master?ref_type=heads

and see if we can include them successfully.

[poppysec]

I've been using Kiln as an alternative to Promptwright.
It allows the manual creation of topic trees based on an initial prompt. The manual and GUI-based nature of the tool has meant it is slower to set up than Promptwright, but the resulting data can be pruned and reviewed as you go on, meaning overall it is better quality.

The generated data can be reviewed one by one and the model will repair any inadequate results, which I've found useful.

From Kiln I have:

• Safe Windows commands
• Malicious Windows commands
• Safe Linux commands
• Malicious Linux commands