[Feature Request] Cut back gratuitous regulation

[mbuechse]

Many recommendations are just being ignored by CSPs, because they can:

• It doesn't hurt certification if recommended testcases are failed.
• Arguably, many of these testcase have little to no benefit.

This state of affairs means that recommendations in general will be ignored and those recommendations that actually matter will get overlooked!

Therefore, I think we should use (or at least formally test) recommendations more sparingly.

Some recommendations that I find gratuitous:

• Flavor name with uneven RAM size (such as 3.5 GiB) for RAM > 10 GiB. Who even does that? (Besides, our test is flawed, because it looks at the actual size instead of the advertised size.)
• Disk size in flavor name should be in (5, 10, 20, 50, 100, 200, 500, 1000). Who even cares about ephemeral disks these days? And many providers seem to like 1, 2, or 25. Who are we to stop them?
• Entropy standard: recommendations regarding flavor extra_spec and image property -- only interesting if a real hw rng is to be used. It doesn't seem to be relevant to anyone! Same for the requirement(!) about rngd. I would drop it (at least from the tests).

[mbuechse]

@garloff I intend to put this on the agenda for SIG Std/Cert.

[garloff]

Recommendations are always a compromise between aligning providers without constraining them too much.
If we can build (almost) universal support, recommendations can later become mandatory. If not, they may still help to reduce fragmentation.
If we require something that everybody does anyhow, that's good, because this way a followed best practice becomes a guarantee. A recommendation may still have a little deterring effect ...
Overall I do see that our providers tend to not align as closely with us as they used to.
I guess one reason is that we have less bandwidth to invest into the interaction.
We could advertise a percentage rating of recommended fulfillment to increase the motivation a bit for doing the recommended things.

[garloff]

Some recommendations that I find gratuitous:

* Flavor name with uneven RAM size (such as 3.5 GiB) for RAM > 10 GiB. Who even does that? (Besides, our test is flawed, because it looks at the actual size instead of the advertised size.)

• If the test is flawed, we should open a bug against it and fix.
• If the recommendation is followed universally, we can make it mandatory. But even if we don't, there is some deterrence value IMVHO.
• The reason we allow it for small flavors is to allow for a tiny SCS-1L-0.5.

* Disk size in flavor name should be in (5, 10, 20, 50, 100, 200, 500, 1000). Who even cares about ephemeral disks these days? And many providers seem to like 1, 2, or 25. Who are we to stop them?

• The disk size is reflected in the flavor name and it help portability to not having to adjust the names.
• The recommendation is thus meant to reduce the variability / fragmentation, well admitting that provider may have specific reasons to diverge. But if they don't, maybe they still follow our recommendation ...
• I agree that ephemeral disks have their main use for the standardized -20s and -100s.

* Entropy standard: recommendations regarding flavor extra_spec and image property -- only interesting if a real hw rng is to be used. It doesn't seem to be relevant to anyone! Same for the requirement(!) about rngd. I would drop it (at least from the tests).

• Lack of entropy used to be one of the traps that caused VMs that had to generate keys to unexpectedly fail. I guess that newer kernels have largely addressed this. So the whole standard may be retired long-term.
• If a specific setting is no longer useful, let's remove it.

[fkr]

I guess one reason is that we have less bandwidth to invest into the interaction

I'd like to disagree here: Over the past 13 months I've been chasing the CSPs (together with @mbuechse) over and over.

[mbuechse]

If we can build (almost) universal support, recommendations can later become mandatory. If not, they may still help to reduce fragmentation.

I disagree. CSPs have an allegiance to their business models and paying customers first and to SCS second. They will absolutely ignore these recommendations.

If we require something that everybody does anyhow, that's good, because this way a followed best practice becomes a guarantee.

I disagree again. If everybody is doing it, no one is asking for regulation, and then regulation will be seen as a burden. SCS already has the reputation of regulating lots of irrelevant stuff (which may or may not be true in general, but in case of 10.5 GiB RAM, I'm inclined to agree).

A recommendation may still have a little deterring effect ... Overall I do see that our providers tend to not align as closely with us as they used to. I guess one reason is that we have less bandwidth to invest into the interaction.

Like Felix said, I don't think this is the case. I would put it like so: a bandwidth bottleneck may exist, but it's not on our end!

We could advertise a percentage rating of recommended fulfillment to increase the motivation a bit for doing the recommended things.

We have discussed this:
#950

And we decided to table it for the time being, because there is no demand:
https://github.com/SovereignCloudStack/minutes/blob/main/sig-standardization/20250612.md#compliance-score-and-aggregated-compliance-score

[mbuechse]

I want to stress this again: if we outlaw RAM sizes that no one intends to use anyway (or some people intend to use no matter what), this means we have

• a standard text that is longer than necessary
• a list of testcases that is longer than necessary
• potentially, log output that is longer than neessary

And this is a real cost for adoption! I don't think it's worth this cost!

[mbuechse]

Maybe, when SCS has a sizeable market share, maybe then we can (re-) introduce this kind of regulation (provided the members allow it)?

[fkr]

Looking at this, I think the most important arguement for me is:

Many recommendations are just being ignored by CSPs, because they can:

• It doesn't hurt certification if recommended testcases are failed.
• Arguably, many of these testcase have little to no benefit.

This state of affairs means that recommendations in general will be ignored and those recommendations that actually matter will get overlooked!

Therefore, I think we should use (or at least formally test) recommendations more sparingly.

Currently our recommendations are meaningless, since it can't easily be differentiated between importan (read: relevant) and other good, but not so relevant recommendations. This is for me the primary point I see in improvements here.

[fkr]

If we require something that everybody does anyhow, that's good, because this way a followed best practice becomes a guarantee.
I disagree again. If everybody is doing it, no one is asking for regulation, and then regulation will be seen as a burden. SCS already has the reputation of regulating lots of irrelevant stuff (which may or may not be true in general, but in case of 10.5 GiB RAM, I'm inclined to agree).

I concur and I will bring this to the workshop on the 24th of February, since this is exactly the feedback we got (over regulating things that don't add real value for the enduser) but these voice are missing in the ongoing standardization effort but instead reach us through other feedback loops.

[mbuechse]

[certain voices] instead reach us through other feedback loops

In my experience, usability and other issues with SCS aren't voiced at all unless people are (a) pressed for it or (b) talking to you anyway about some other topic.

[mbuechse]

BTW another interesting perspective would be that of the much-heralded independent software vendors, but right now I feel they mostly care about the K8s layer.