Reading a huge data flow

[maxim133]

Hello! I have found strange behavior when I was reading a long data flow. If the data size more than readBufferSize_B_ (default value is 256) it will not receive other part of the data.
There are two ways to resolve it:

1. Using a cycle. For example:

    void SerialPort::Read(std::string& data) {
        PortIsOpened(__PRETTY_FUNCTION__);

        // Read from file
        // We provide the underlying raw array from the readBuffer_ vector to this C api.
        // This will work because we do not delete/resize the vector while this method
        // is called
        while (ssize_t n = read(fileDesc_, &readBuffer_[0], readBufferSize_B_)){
                // Error Handling
            if(n < 0) {
                // Read was unsuccessful
                throw std::system_error(EFAULT, std::system_category());
            }
            else if(n == 0) {
                // n == 0 means EOS, but also returned on device disconnection. We try to get termios2 to distinguish two these two states
                struct termios2 term2;
                int rv = ioctl(fileDesc_, TCGETS2, &term2);

                if(rv != 0) {
                    throw std::system_error(EFAULT, std::system_category());
                }
            }
            else if(n > 0) {
                data += std::string(&readBuffer_[0], n);
            }
        }

        // If code reaches here, read must of been successful
    }

2. Return 'n'. User will be responsible for cycling reading.

What do you think about it?

[gbmhunter]

Hi @maxim133 ,

I'm trying to figure out how this should work with SetTimeout(). The user can use this to set if the call to Read() is non-blocking (set it to 0), or wait for a certain time for data before returning. This manipulates the VMIN and VTIME settings provided by the Linux API. If the timeout is set to 0, this loop you're describing would work well. It would be a bit strange if the timeout was say, 1s. This would result in the call to Read() potentially taking many seconds to return, or perhaps never return at all (imagine if there was continuous data such that every second it received some more, in this case every call to read() would return an n > 0 and then go back to waiting another second for more data. Does that make sense?

One option is to just loop inside the Read() if the timeout is set to 0. This doesn't feel like a clean way of handling it.

Another option is to leave it up to the user. I don't think we need to provide n, as they can easily work this out based on if the length of the std::string or std::vector has changed size.

Another idea is that the user could set n before Read() is called, e.g. a new method called SetRxBufferSize(). Then the user could check if the returned bytes is equal to this size. If so, there might be a good chance there is more data available immediately. I sort of like this idea.

Another idea is we could expose VTIME and VMIN directly to the user. SetTimeout() was meant to be a simplification to make it easier to understand, but it does limit the functionality compared to having control over VTIME/VMIN directly. See http://www.unixwiz.net/techtips/termios-vmin-vtime.html for a good explanation on what they do!

Thoughts?