Add Date Modified and Date Created sorting for repositories

[fabiovincenzi]

Context

The repo list UI has a sort dropdown that previously included "Date Modified" and "Date Created" options. These are being removed as part of the server-side pagination in #1464 because the fields lastModified and dateCreated don't exist on the Repo model, sorting fell back silently to alphabetical order.

What needs to be done

• Add lastModified and dateCreated fields to the Repo model
• Populate them on repo creation and update
• Add them to the sortBy allowlist in the repo route
• Re-add the options to the Filtering dropdown in the UI

Migration considerations

• Existing DB records won't have these fields
• Need a migration strategy for both MongoDB and NeDB
• Should consider backwards compatibility / ability to roll back
• kriswest noted: dates could potentially be extracted from existing logs

[jescalada]

As you mentioned, backward compatibility is a must here - we probably shouldn't be breaking the API when adding a new endpoint might do the trick. Likewise with the DB, filling in the blanks like what we did with repo URLs in #1043 might be the best: just automatically fixing those up on startup so users don't need to worry about migrations.

This ought to be added in 2.1 (or 2.2) rather than being forced to cut a 3.0 just for this single, potentially breaking change.

Pinging @kriswest for thoughts on this 🤔

[kriswest]

If we can migrate forwards and backwards to/from the next/previous version, and we're not breaking any existing API calls, then I think this probably non-breaking and can go in a minor release. We may need to add something to track and perform (DB) migrations in order to achieve it (i.e. add a DB table that tracks whether a migration has been run) - that would be a good facility for us to have anyway and may help us to implement a variety of other planned features that will affect the data model and prevent those also forcing a major release. There are various libs out there that implement migration like behaviour, but its also easy enough to roll your own.

[andypols]

Out of curiosity, what use case is this answering - why is it interesting to the user? We have new version of the ui (coming to a pr soon) and this has not come up as something anyone wanted to do - we can sort by alpha, relevance (ie repos I'm a contributor/reviewer), and latest push.

[kriswest]

Is latest push not the same thing as date modified @andypols ? How are you doing that without a date field? (I haven't a chance to check how we're doing it now). What field are you sorting by for the alpha sort? Repo name?

Thinking about it, performing a review would affect the modified date. So latest push and modified date are not necessarily the same thing.

Regarding the use case, we probably want to be able to sort by the date first pushed (default sort, created date) and date updated (modified date, updated either by push, review etc.).

[andypols]

@kriswest - I thought this was about the sorting of repositories - they are only modified when you add/remove users? Or am I being too literal? We are optionally sorting the repos based on the git pushes for that repo by using a mongo pipeline query. Perhaps this is what @fabiovincenzi is asking for.

[kriswest]

My bad, I see your point on modified date for a repo. I imagine we might pick up more editable properties in future that make this more interesting.

You can likely still predict a date for creation based on the oldest push, but I guess it's less necessary than I had thought. Sorting by organisation+name or name seems more useful. Perhaps also a computed field for number of contributions. But there is still some small utility to sorting by most recent created/edited if the list of repos is long.

[kriswest]

Quick bit of feedback from #1524, there is a clear use case for the created date. Last modified date is less useful (no clear use case), but it doesn't hurt to include. @andypols suggested doing this change in 2.2.0 alongside the revised UI - so lets go for it.

If you've already raised a PR its not linked here btw