The design problem associated with this is that we must be able to look ahead without consuming.
As the current version makes no guarantees about tokens that do not fit in a single input buffer, we can simply examine the buffer itself. In the future, however, it will be necessary to read multiple buffers of information without consuming them.
However, BufRead.fill_buf() will happily return the same buffer over and over again if you do not consume in between.