GQL Integration Rework

[mpforce1]

Description

Full rework of the GQL layer. There is now a specific parser for each response type to ensure that we are working with the correct data structure. There is now error handling with retry logic for all GQL requests, this is an attempt to reduce error logs for the user.

If there is an error parsing the json we will now see an error log telling us exactly where the issue was:

GQL Operation 'Test Name' returned errors: [Error({'message': 'test message 1', 'path': ['test path 1', 'test path 2']})]
GQL Operation 'VideoPlayerStreamInfoOverlayChannel' returned errors: [Error({'message': 'service timeout', 'path': ['user']})]
JSON at ["extensions"] has an invalid shape: value should not be None
JSON at [] has an invalid shape: response was empty

Fixes #739

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Tested today locally, no additional errors encountered.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented on my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation (README.md)
• My changes generate no new warnings
• Any dependent changes have been updated in requirements.txt

[mpforce1]

@Armi1014 If possible, I would please like your feedback on my 2nd commit on this branch. I've attempted to make a proper response parser and retry system for the GQL layer. Nothing's hooked up yet but the parsing and request handling should all be in place in the new gql module.

@rdavydov I also wouldn't mind you having a look but I know you're busy 😄

[Armi1014]

@Armi1014 If possible, I would please like your feedback on my 2nd commit on this branch. I've attempted to make a proper response parser and retry system for the GQL layer. Nothing's hooked up yet but the parsing and request handling should all be in place in the new gql module.

@rdavydov I also wouldn't mind you having a look but I know you're busy 😄

Thanks for pinging me, I had a look at your second commit. Overall the new GQL layer looks really nice, feels like a good step towards a properly typed API instead of random JSON poking 👍

A few things I noticed while reading:

• ClientSession vs headers: ClientSession stores the version as self.version, but the GQL headers use self.client_session.client_version. That will raise an AttributeError at runtime. I would either rename the field to client_version or switch the header to self.client_session.version.

• Generic method syntax: def post_gql_request[T](...) uses the new PEP 695 syntax, which only works on Python 3.13+. The repo still targets older Python versions, so this will be a syntax error there. I would go back to T = TypeVar("T") plus def post_gql_request(self, ..., parse: Callable[[Any], T]) -> T | None:.

• channel_follows pagination: With the new Paginated type, parsed_response.follows is a Paginated[Follow], not an iterable of Follow. The cursor also lives on the Edge and the property is page_info.has_next_page, not pageInfo.has_next. I think the loop should iterate paginated.edges and use edge.node.login / edge.cursor, plus paginated.page_info.has_next_page.

• get_drop_campaign_details parameter: The function signature says campaign_ids: list[str] but the body uses campaign["id"]. If it is really a list of IDs, that will blow up. Either accept a list of objects with an id field or drop the index and just use dropID: campaign.

• Error handling in post_gql_request:

  • Right now HTTP status codes are not checked; response.json() will be called even on 500s and non-JSON bodies.
  • logger.error("Error handling GQL response.", e) does not log the exception the way you expect; logger.exception(...) would probably be better.
  • Catching GQLError here means methods like join_raid will not actually ever raise it even though the docstring says they do. It might be worth letting GQLError bubble and only retry/catch RequestException.

• Error name clash: In response/__init__.py you re-export Error from both Error.py and Predictions.py, so the predictions one wins. It might be clearer to give one of them a more specific name or avoid re-exporting both under the same name.

Overall I really like the structure (ClientSession + GQL + typed response models), just a few rough edges that should be easy to clean up before wiring it into the rest of the miner.

[mpforce1]

Thanks for the quick feedback! I'll look into those rough edges soon.

Generic method syntax

This was actually added in 3.12 and I think that's the version we're targeting moving forward, although I can't remember if that's on this branch or on a different one (might be the Hermes API branch). I'll make it explicit in this branch.

Catching GQLError

Yeah, I considered letting them fall through but I wanted the automatic retries. I was being a bit greedy, I think what'll be best is having a specific Exception type for recoverable errors at the parsing stage (i.e. service timeout) and retrying on those and RequestException.

[Armi1014]

Thanks for the quick feedback! I'll look into those rough edges soon.

Generic method syntax

This was actually added in 3.12 and I think that's the version we're targeting moving forward, although I can't remember if that's on this branch or on a different one (might be the Hermes API branch). I'll make it explicit in this branch.

Catching GQLError

Yeah, I considered letting them fall through but I wanted the automatic retries. I was being a bit greedy, I think what'll be best is having a specific Exception type for recoverable errors at the parsing stage (i.e. service timeout) and retrying on those and RequestException.

Thanks for the quick reply and clarifications!

If we are officially targeting 3.12+ then I am totally fine with the PEP 695 syntax. Making that explicit in this branch (README / pyproject / CI / Docker image) sounds good so people on 3.10/3.11 aren’t surprised when it doesn’t even parse.

On the error handling side, a dedicated “recoverable” GQL exception sounds like a good split to me. My main concern was just that hard failures (auth issues, invalid operations, schema changes, etc.) don’t silently turn into None everywhere. Retrying on RequestException + a specific recoverable GQL error, and letting the rest bubble up, would solve that nicely.

Happy to test this once you’ve wired it into the rest of the miner!

[Gamerns10s]

Heyyyy bro
Any updates about this ?

[mpforce1]

Heyyyy bro Any updates about this ?

Sorry, I've had less time to work on this than I thought. I've pushed up a commit that hopefully handles the last feedback I got. Just gotta wire it all through and run some tests. I've got some time tomorrow to get it done 🤞

[mpforce1]

Just to update, I've got the implementation finished and wired into the miner, just running a live test to see if everything is working as expected. I'll push up my changes once I'm satisfied it's all working.

[mpforce1]

Alright, the new integration is now wired into the miner. The main difference should be that when we get errors from the GQL API you should get more useful errors and there are also automatic retries for some types of errors.

This is a fairly large rework that touches a lot of the miner's functionality so there's almost certainly something I've missed. If possible, I'd like some people to test this version and let me know if you have any issues.

[Gamerns10s]

I was just updating my code according to this pr but I don't have the websocketpool file as my code mainly works according to your Hermes version. So is there any point of me updating to this pr or will the not work without the current updates you did to that file
Also the updates in get_stream_info method in twitch.py collides with the #753 changes 🥲

[mpforce1]

I was just updating my code according to this pr but I don't have the websocketpool file as my code mainly works according to your Hermes version. So is there any point of me updating to this pr or will the not work without the current updates you did to that file Also the updates in get_stream_info method in twitch.py collides with the #753 changes 🥲

Yeah, those would be messy merges for sure. WebSocketPool is new to the hermes pr so this pr doesn't have it, you'd have to do it manually and I haven't attempted that myself. Even messier would be a merge with the #753 pr branch. We both have extensive changes to the Twitch class. For now, I would suggest just testing this pr without including any other changes.

[Gamerns10s]

Oh ngl but I already did all of the changes now 😭 lemme revert everything and make a new branch for it 🤦‍♂️
Update: it's a lot of work I'll take around a day to get that reversed now (I got exams going on 🥲)

[mpforce1]

Finally caught a service timeout response in this version. The miner reacted as I wanted, it retried and got a successful result:

30/11/25 18:54:12 - DEBUG - TwitchChannelPointsMiner.classes.gql.Integration - [__handle_result]: VideoPlayerStreamInfoOverlayChannel succeeded after 2 attempts. Errors: [GQL Operation 'VideoPlayerStreamInfoOverlayChannel' returned errors: [Error({'recoverable': True, 'message': 'service timeout', 'path': ['user']})]]

I've made this a DEBUG log since people don't really need to know if this gets retried unless it fails all attempts.

[mpforce1]

I've been trying to catch some errors in the wild so I can see if some changes I've made to the error logging are working as expected. Unfortunately (or fortunately?) I've not seen any errors with the GQL API since I made those changes last week. So I've pushed them up, I'd be grateful anyone could test this branch.

In particular, if you get GQL errors you should only get a stack trace if it's something like a connection error, you should not get them for things like persistent query errors, service timeouts, or JSON parsing issues. In those cases you'll get an ERROR log that plainly states what happened, the stack trace is omitted here for being useless to the end user and debuggers.

[brunoshure]

So I've pushed them up, I'd be grateful anyone could test this branch.

I've been testing this today and I've been able to get a few errors. Nothing major.

One instance like this:

07/12 21:20:38 - Error while trying to load channel points context: GQL Operation 'ChannelPointsContext' failed all 1 attempts, errors:
[GQL Operation 'ChannelPointsContext' returned errors: [Error({'recoverable': False, 'message': 'service error', 'path': ['community', 'channel', 'communityPointsSettings', 'emoteVariants']})]]

And a few of these:

07/12 21:42:46 - #6 - WebSocket error: fin=1 opcode=8 data=b'\x10\x04ping pong failed'
07/12 21:42:46 - fin=1 opcode=8 data=b'\x10\x04ping pong failed' - goodbye
07/12 21:42:46 - #6 - WebSocket closed
07/12 21:42:46 - #6 - Reconnecting to Twitch PubSub server in ~60 seconds
07/12 21:42:47 - #8 - WebSocket error: fin=1 opcode=8 data=b'\x10\x04ping pong failed'
07/12 21:42:47 - fin=1 opcode=8 data=b'\x10\x04ping pong failed' - goodbye
07/12 21:42:47 - #8 - WebSocket closed
07/12 21:42:47 - #8 - Reconnecting to Twitch PubSub server in ~60 seconds
07/12 21:42:47 - #57 - WebSocket error: [Errno 32] Broken pipe
07/12 21:42:47 - [Errno 32] Broken pipe - goodbye
07/12 21:42:47 - #57 - WebSocket closed
07/12 21:42:47 - #57 - Reconnecting to Twitch PubSub server in ~60 seconds
07/12 21:42:47 - #44 - WebSocket error: fin=1 opcode=8 data=b'\x10\x04ping pong failed'
07/12 21:42:47 - fin=1 opcode=8 data=b'\x10\x04ping pong failed' - goodbye
07/12 21:42:47 - #44 - WebSocket closed
07/12 21:42:47 - #44 - Reconnecting to Twitch PubSub server in ~60 seconds

Other than that, everything seems to be working fine.

[mpforce1]

One instance like this:

07/12 21:20:38 - Error while trying to load channel points context: GQL Operation 'ChannelPointsContext' failed all 1 attempts, errors:
[GQL Operation 'ChannelPointsContext' returned errors: [Error({'recoverable': False, 'message': 'service error', 'path': ['community', 'channel', 'communityPointsSettings', 'emoteVariants']})]]

That's interesting, the formatting is exactly what I want so that's great. However, I'm not familiar with service error for the message. If it only happened once then it's probably not a problem, and can probably be retried. Any chance you could post the line before this in your DEBUG logs, it should show the full request and response containing that error text?

[mpforce1]

I did a bit more searching on the service error message and found examples of this happening in the drops miner project. The consensus over there seems to be that this is usually a Twitch issue and Twitch will recover (likely by restarting affected servers) in time. The time to recovery is of the order minutes to hours so there's no point retrying them in the short term as Twitch likely won't have recovered by then. Compare this to service timeout errors which can often be successfully retried immediately.

Something we could do is specifically ignore service error/service timeouts that occur on parts of the data structure we don't need. I don't know how easy that would be to implement, it might require duplicating the parsing logic so that the error parser and the data parsers both know what the structure should be, and ideally we shouldn't duplicate that. There might be a solution in having the expected structure for required values encoded in a way that both parts of the codebase can query. I think we'll have a look into that at a later date though, for now we should focus on getting these changes tested and merged.

[mpforce1]

Finally got another live error myself, formatting looks good:

09/12/25 10:37:01 - ERROR - [__get_campaign_ids_from_streamer]: Error while trying to get drops campaign ids for streamer: GQL Operation 'DropsHighlightService_AvailableDrops' failed all 1 attempts, errors: [GQL Operation 'DropsHighlightService_AvailableDrops' returned errors: [Error({'recoverable': False, 'message': 'service unavailable', 'path': ['channel']})]]

Twitch recovered from this by the next attempt 2 minutes later, so it's possible this could be immediately retried. I'll just leave it noted here for now.