Indicators of plugin quality (stars, updated, downloads, travis ci)

[pdclark]

Submitted via email by @kurtpayne

The problem we were discussing over twitter is how to represent plugin quality. I see that WordPress has 3 controls for this:

• Last updated (can get this from github)
• Total downloads (not available from github)
• Star rating (not available from github)
• Reviews (not available from github)

Github does promote unit testing, though, so if a plugin has a travis badge, that could be displayed in the results. Github also has reviews, activity, and number of watchers.

As for the missing info (reviews, ratings) .... it wouldn't be difficult to create a third party site for this. Enter a review + rating attached to a github URL. It could be one of the GitHub service hooks.

So, in summary .... take what you have here, add in a way to do ratings/reviews (possibly using a github service hook) ... and you can get quality measure ... and possibly explore other things like travis badges, # of followers, etc.

What do you think?

Overly complicated?

[pdclark]

I think this is a critical consideration. I was experimenting with the plugin as a quick-and-dirty tool for developers, but with a method for communicating quality, installation via Github could be much more mainstream!

Ratings

My first, and main, considerations where using Star/Watch/Fork Count (in place of star rating) and some kind of centralized list of verified authors; perhaps just a Git ropository listing approved authors that pull requests could be submitted to. I think homebrew uses a system like this with their forumlas: https://github.com/mxcl/homebrew .

Travis CL

I am less familiar with how to tie-in with Travis CL. Does the badge display in the readme, or is it loaded by Github querying the Travis API?

Reviews

Reviews get a lot more complicated, but you know that. ;) It would certainly require a centralized database somewhere. I don't know if it'd be worth going into that level of infrastructure unless quite a few users began with this method.

Comparison to WP.org

Github certainly could completely replace what's available on the WP.org, but I'm not sure that it should. My main concern would be maintaining simplicity for the end user. Even if the target is developers, I'd like the front-end interface to be as simple and straightforward as possible.

Additional server / infrastructure

On the other hand, a central server would be very helpful in overcoming some limitations of the Github API. The search methods I'm using are still in preview, but at a minimum, I'm already having to make some compromises to get plugin identification to be semi-reliable.

Again, remember that this is just a few hours into dev. ;)

[kurtpayne]

Travis has API documentation here:
https://api.travis-ci.org/docs/

And the status images documentation is here:
http://about.travis-ci.org/docs/user/status-images/

[afragen]

GitHub also has some score field but I really don't understand what it is. Just for fun I've added ratings (to afragen/github-updater) very arbitrarily based on adding score and stargazers. Anything over 100 stays there. I'm just making it up. 😉

In this manner it's more a popularity indicator and not an indicator of quality.