A93: xDS ExtProc Support

[markdroth]

No description provided.

[eshitachandwani]

Curious about the example here, from what I understand, sending and getting the modified headers should be a blocking operation and only after modifying the headers should we use them to create the dataplane stream. The Send and Receive functions should come after the stream is created. The example seems to suggest that Send can happen before getting the header modification. Just want to understand the sequence of events.

[markdroth]

In the C-core API, the application does not need for sending the headers to complete before it sends the first message on the stream, which is why this can happen.

I am guessing from your comment that the grpc-go API does not allow the application to send the first message until it gets the stream object, which is not returned until the headers have gone out on the wire? If so, then I think there are two possible ways to deal with it:

1. Keep things the way they are, which means this specific case can't happen in Go.
2. Tell the application that the stream has been created as soon as headers are sent to the ext_proc server, without waiting for the headers to be sent to the data plane server. This would increase the amount of memory used for buffering headers, but it would allow better pipelining.

I think either option is probably fine. @easwars and/or @dfawley may have thoughts on which one makes more sense for Go.

Note that even if this specific example can't happen in Go, there are other cases that will happen. For example, in the server-to-client direction, the ext_proc filter does not need to wait until it receives the response headers from the ext_proc server before it sends the first response message.

@ejona86 Not sure if this is an issue for Java.

[ejona86]

IIRC, in Go the client headers block until a stream is allocated. So it could make sense to wait for those to be processed by ext_proc before unblocking the client, to still wait for the stream to be allocated. That doesn't sound required though. And it would add significant latency, especially for unary RPCs.

Java doesn't have flow control for unary RPCs, so it doesn't impact that. We do typically delay saying the RPC is ready for data until we allocate the stream, but we can trivially implement this either way. (I think it'd only impact a single if.)

[easwars]

From what I can see, when an application attempts to create a stream:

• the grpc layer creates an instance of the client stream interface and as part of doing this, it creates an attempt
• the attempt retrieves the transport by doing a Pick and creates a stream on the transport
• this results in an instance of the client stream interface being created at the transport layer
  • this allocates the stream ID and checks for write quota (which should exist, the default is 64K)
  • creates the header frame and queues it in the control buf
  • returns

At this point, the application should have a stream and should be able to send messages on it. They won't get sent out on the wire until the headers frame is sent out. I don't see the transport code even waiting to get a headers frame before sending out data (as long as there is write quota).

@eshitachandwani : If you think otherwise, let's talk and go over the code together.

[markdroth]

@easwars The ext_proc filter would run between the first and second bullet in your list -- i.e., after the client stream instance is created but before the LB pick. This means that the workflow will go like this:

1. gRPC layer creates client stream interface, which creates an attempt.
2. Headers are sent to the ext_proc filter. The ext_proc filter sends them to the ext_proc server and waits for the response.
3. The ext_proc filter receives the headers from the ext_proc server. Now the ext_proc filter sends the headers on.
4. We do an LB pick, resulting in creating the client stream at the transport layer. This gets returned to the application, which can then send the first message.

The problem here is that there is a round-trip to the ext_proc server in steps 2 and 3, and the application will need to wait for that to finish before it can send the first message on the stream. This will hurt latency.

The alternative is to do something like this:

1. gRPC layer creates client stream interface, which creates an attempt.
2. Headers are sent to the ext_proc filter. The ext_proc filter sends them to the ext_proc server and waits for the response.
3. While waiting for the ext_proc response, return the client stream interface to the application, so that it can start sending messages on the stream.
4. The client application sends a message on a stream. This message gets down to the ext_proc filter, which sends the message to the ext_proc server.
5. The ext_proc filter receives the headers from the ext_proc server. Now the ext_proc filter sends the headers on.
6. We do an LB pick, resulting in creating the client stream at the transport layer.

This approach would improve latency by allowing the application to start sending messages on the stream without waiting to get the headers back from the ext_proc server.

[eshitachandwani]

Understood. I think the alternative should be doable in Go too.

[eshitachandwani]

@markdroth I realised there might possibly also be a downside here. In the case where the ext proc stream ends with OK status before it can deliver the initial header modifications, we would want to use the original headers to create the dataplane stream. But it could so happen that in the meantime , we have sent some request body messages to the proc server, so they will be lost. Or does the guaruntee that that proc server must emit all the messages it has received before ending stream hold here as well and and we will either get the header response and all the sent messages back?

[markdroth]

If the ext_proc filter is configured to send request or response messages to the ext_proc server, then the ext_proc server must use the drain procedure described in the next section before closing the stream. I've updated the text in the next section to attempt to clarify this.

[markdroth]

@kannanjgithub @eshitachandwani @rishesh007 FYI, it looks like we don't actually have a use-case for the mode override feature, and the logic for that is a little ugly, so I've removed it from the design. We can consider adding it in the future if/when we encounter a use-case for it.

[rishesh007]

Let’s say the gRPC server has already started sending S2C messages, and those messages are being forwarded to the proc server. However, before any response is received from the proc server for those S2C messages, the server trailers arrive from the upstream server.

In this scenario, when the trailer ProcessingMode is set to SKIP, what should be the expected behavior of the filter? Should the filter immediately close the processing stream and continue forwarding the trailers downstream, or is there another recommended way to handle this case?

Edit: Same scenario for ServerInitialMetadata instead of S2C messages

[markdroth]

Thanks for calling this out!

This made me notice that if the response body send mode is GRPC, then the response trailer send mode cannot be SKIP, because otherwise the ext_proc server cannot indicate when it's finished sending messages. (Note that Envoy has this same requirement for FULL_DUPLEX_STREAMED mode, which is the mode that GRPC mode is based on.) I've noted this restriction in the configuration section.

I don't think this is a problem for server initial metadata, because (as @kannanjgithub pointed out in a comment above), the ext_proc protocol represents Trailers-Only as headers rather than trailers, and that's the only case where we'd see trailers before headers.

[eshitachandwani]

The gRFC now says that : Note that if response_body_mode is set to GRPC, then response_header_mode must be set to SEND. but @markdroth your comment and the FULL_DUPLEX_STREAMED mode says that response trailer send mode cannot be SKIP ? Am I missing something?

[rishesh007]

If a header named 'xyz' is marked as immutable (header mutation is not allowed), but its HeaderValueOption.AppendAction is set to kAddIfAbsent, should the RPC fail if the header is already present in the metadata? Since the header is already there, no actual mutation would take place.