Voting for new tasks/challenges before they get online

[natrius]

Is your feature request related to a problem? Please describe.

There is currently a very lifely discussion in the OSM forum about the quality of some MapRoulette challenges and how it could be possible to improve them before they go live. Some challenges are poorly designed, leading to incorrect edits or frustration among contributors.

Describe the solution you'd like

A potential solution would be to require new tasks/challenges to receive positive feedback before they are published. However, there are some challenges to consider:

• What happens if a challenge is well-designed but does not attract enough reviewers, leading to a lack of votes?
• Should MapRoulette have an internal voting/review system, or could an external system, such as OpenStreetMap Discourse voting, be leveraged for this purpose?
• This could be integrated with Mandatory link to discourse-thread before being able to start a challenge #2563, where new challenges must be discussed in a forum thread. The process could require a simple yes/no poll to be open for X days, with the challenge only going live if negative votes stay below a certain percentage (e.g., Y%). Not sure if that even possible to request from the discourse forum.
• How should cases be handled where a challenge initially receives significant negative feedback but is later improved based on discussions and changes?

Describe alternatives you've considered
There could be some ranking system, but i personally don't like this because OSM itself does not have any kind of "ranking". There could also be a checklist that needs to be checked, but i'm not sure

Additional context

• Require certain number of tasks solved to gain "reputation" before you can create challenges #1853
• https://community.openstreetmap.org/t/maproulette-critique/116922/139

[whyteks]

* What happens if a challenge is well-designed but does not attract enough reviewers, leading to a lack of votes?

Something similar as to what happens when a patch is sent to code review in a collaborative project and does not attract reviewers - the author would reach out to other mappers and invite review. This extra step and bar to entry can only be helpful in avoiding badly designed challenges.

And maybe, just maybe, some kind of CI/CR (equivalent of unit and compilation tests) could be implemented with language analysis of the challenge instructions. I don't want to and am not advocating for the introduction of such technology, nor do I know much about it, but these kinds of tools seem to be the way things are going for this type of problem.

[mvexel]

Hi @natrius and @whyteks - thanks for bringing this up and sorry it took me a while to respond.

We are currently thinking about ways to add more community input opportunities on challenges. We are considering a voting mechanism. It is more likely we will implement something that is available after a challenge goes live, but we will take your arguments into account.

Thanks again for your feedback on this, it is very much appreciated.

[natrius]

I personally would see it BEFORE the challenge goes live. If we mean the same thing, my understanding is "live = people can start solving challenges" and i think something like voting should be done before its live. There could/should be three stages:

• Draft (For changing and trying around, with checkmark could even be publically visible)
• Voting (publically available for voting)
• Live(or "Active" when challenges can be solved)