org.apache.kafka.common.errors.TimeoutException on UI only

[pmeisen]

Hi,

we are using Kafka 2.2.0 and observing org.apache.kafka.common.errors.TimeoutException when selecting a specific consumer. The issue seems to occur only in the UI, when we use the shell script kafka-consumer-groups.sh on the servers (on each broker), everything works just fine and the results for the group are returned immediately. This does not happen for all groups just for some.

bin/kafka-consumer-groups.sh --describe --bootstrap-server <server>:<port> --group <group>

The exact error in the UI:

Error Failed to get offsets by times in 15000ms (show more)
org.apache.kafka.common.errors.TimeoutException thrown at -> Failed to get offsets by times in 15000ms

Any idea on how we could narrow down the problem or what could cause this issue?

Thanks a lot!

[pmeisen]

Just to follow up on the issue, we also noticed that in the UI some consumers are supposed to be lagging, i.e., the consumer-offset is way lower than the tail offset. Nevertheless, checking the cluster and getting the information via the shell script (bin/kafka-consumer-groups.sh) reveals that there is no lag and that the shown tail offset in the UI is just incorrect. Any idea how that can happen? Does the UI cache any information or is maybe the version not compatible?

PS: I added this as a comment since I'm sure that these two observations are somehow related to each other.

[pmeisen]

Just another follow up, while we further investigated the issue. It looks like that this only happens for Consumer Groups which have multiple Topics assigned to the group. When I mentioned in the initial post, that this is only happening for some, it actually only happens for groups which have multiple topics assigned. Maybe that helps to determine the issue.

[Crim]

Thanks for reporting. Can you send me which URL specifically you're getting the timeout on along with a stack trace?

Thanks!

[pmeisen]

Hi Crim,

thanks for following up:

url for website: http://server/cluster/1/consumer/group-name
GET ajax: http://server/api/cluster/1/consumer/group-name/offsetsAndTailPositions

Response from GET:

{
	"error": true,
	"message": "Failed to get offsets by times in 15000ms",
	"requestType": "ClusterNodes",
	"causes": [{
		"type": "org.apache.kafka.common.errors.TimeoutException",
		"message": "Failed to get offsets by times in 15000ms",
		"file": "",
		"method": "",
		"line": 0
	}]
}

Stack-Trace:
I do not see any stack-trace printed to the logs for this error (just the output on the UI). If you can point me into any direction on how to configure the logging so that it would print it, let me know and I'll get it for you.