feature: Add option to control log level for retrying jobs

[nikiepaydk]

Hi,

We have some type of jobs, where we expect a high error rate. When these jobs fail they can be quite noisy in our system logs, and we currently use a filter on the logs to not show these lines based on the job kind.

River seems to always log a warn "Job errored; retrying" error message whenever a job fails. And we would like to be able to either disable or lower the log level to debug for jobs like this.

We could solve it using "Snoozing jobs" and logging ourselves. But this would require us to track attempts elsewhere, since snoozing has no impact on the main retry logic otherwise.

If there are any other recommended solutions, please let me know!

[brandur]

Yeah, those statements should be info. I'm surprised no one's reported this before. Added in a fix in #1190.