feat(integration-twitch): Twitch EventSub webhook ingestion (data lake) #272
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| # Integration Twitch Context | ||
|
|
||
| Transport and integration layer for the Twitch platform. Owns all HTTP communication with Twitch APIs (via Saloon), OAuth flows, EventSub webhook ingestion, and ETL for transforming raw event data into domain entities. | ||
|
|
||
| ## Glossary | ||
|
|
||
| | Term | Definition | Not to be confused with | | ||
| | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | | ||
| | **Transport** | The Saloon-based HTTP layer (`Transport/`) that sends requests to Twitch's Helix API and OAuth endpoints. All outbound HTTP goes here. | Inbound webhooks (which are received in `Http/`) | | ||
| | **TwitchHelixConnector** | Saloon connector authenticated with App Access Token + Client-Id. Used for Helix API calls (EventSub, Users). | `TwitchOAuthConnector` (which handles token exchange, no auth) | | ||
| | **TwitchOAuthConnector** | Saloon connector for OAuth2 token exchange and app access token retrieval. Base URL: `id.twitch.tv/oauth2`. | — | | ||
| | **App Access Token** | Server-to-server token obtained via client_credentials grant. Cached. Used as default auth on `TwitchHelixConnector`. | User Access Token (obtained via authorization_code, per-user) | | ||
| | **EventSub** | Twitch's unified event notification system. We receive events via webhook transport (HTTP POST to our endpoint). | PubSub (deprecated, shut down April 2025) | | ||
| | **TwitchEventLog** | Raw event record in `twitch_event_logs`. Stores the full EventSub payload as JSONB. Data lake — no processing on write. | Processed domain entities (future ETL output) | | ||
| | **ETL** | Layer that transforms external data (raw payloads) into domain entities. Covers both historical imports and real-time event processing. | Transport (which is outbound HTTP) or Http (which is ingestion) | | ||
|
|
||
| ## Structure | ||
|
|
||
| ``` | ||
| src/ | ||
| ├── Console/ | ||
| │ ├── LinkTwitchChannelCommand.php ← Links channel to tenant via ExternalIdentity | ||
| │ └── SubscribeTwitchEventsCommand.php ← Creates EventSub subscriptions via Helix API | ||
| ├── Enums/ | ||
| │ └── TwitchEventSubType.php ← All EventSub subscription types with version/condition | ||
| ├── ETL/ ← Empty in MVP, ready for processing | ||
| │ ├── Actions/ | ||
| │ ├── Console/ | ||
| │ └── DTOs/ | ||
| ├── Http/ | ||
| │ ├── Controllers/ | ||
| │ │ └── TwitchWebhookController.php ← Receives EventSub webhooks, persists to TwitchEventLog | ||
| │ └── Middleware/ | ||
| │ └── VerifyTwitchSignature.php ← HMAC-SHA256 signature verification | ||
| ├── Models/ | ||
| │ └── TwitchEventLog.php ← Raw event data lake | ||
| ├── OAuth/ | ||
| │ ├── TwitchOAuthClient.php ← Implements OAuthClientContract (uses Transport) | ||
| │ ├── TwitchAppTokenService.php ← Client credentials flow, cached app token | ||
| │ └── DTO/ | ||
| │ ├── TwitchOAuthAccessDTO.php | ||
| │ └── TwitchOAuthDTO.php | ||
| └── Transport/ | ||
| ├── TwitchHelixConnector.php ← App token auth, base URL: api.twitch.tv/helix | ||
| ├── TwitchOAuthConnector.php ← No default auth, base URL: id.twitch.tv/oauth2 | ||
| └── Requests/ | ||
| ├── OAuth/ ← ExchangeCodeForToken, GetAppAccessToken | ||
| ├── Users/ ← GetCurrentUser, GetUsers | ||
| └── EventSub/ ← CreateSubscription, ListSubscriptions, DeleteSubscription | ||
| ``` | ||
|
|
||
| ## Module Boundaries | ||
|
|
||
| ### This module owns: | ||
|
|
||
| - All HTTP requests to Twitch APIs (Helix + OAuth) via Saloon | ||
| - OAuth token exchange and user profile retrieval | ||
| - App access token management (client credentials, cached) | ||
| - EventSub webhook reception and raw payload storage | ||
| - EventSub subscription lifecycle (create, list, delete) | ||
| - ETL processing of Twitch events into domain entities (future) | ||
|
|
||
| ### This module does NOT own: | ||
|
|
||
| - User/tenant identity management (belongs to `identity`) | ||
| - Gamification or XP from Twitch events (belongs to `character` / `ranking`) | ||
| - Discord notifications about Twitch events (belongs to `bot-discord`) | ||
|
|
||
| ## Dependencies | ||
|
|
||
| - **Identity** — OAuth user resolution (`OAuthClientContract`, `ExternalIdentity`) | ||
| - **Saloon** — HTTP transport layer (`saloon/saloon ^4.0`) | ||
| - **No dependency on** Moderation, Bot Discord, or Integration Discord | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,28 @@ | ||||||
| <?php | ||||||
|
|
||||||
| declare(strict_types=1); | ||||||
|
|
||||||
| use Illuminate\Database\Migrations\Migration; | ||||||
| use Illuminate\Database\Schema\Blueprint; | ||||||
| use Illuminate\Support\Facades\Schema; | ||||||
|
|
||||||
| return new class extends Migration | ||||||
| { | ||||||
| public function up(): void | ||||||
| { | ||||||
| Schema::create('twitch_event_logs', function (Blueprint $table): void { | ||||||
| $table->id(); | ||||||
| $table->string('event_type')->index(); | ||||||
| $table->string('broadcaster_user_id')->nullable()->index(); | ||||||
| $table->string('user_id')->nullable(); | ||||||
| $table->string('twitch_message_id')->nullable()->unique(); | ||||||
|
There was a problem hiding this comment. Enforce non-null Line 18 makes Suggested schema tweak- $table->string('twitch_message_id')->nullable()->unique();
+ $table->string('twitch_message_id')->unique();📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||
| $table->jsonb('payload'); | ||||||
| $table->timestamps(); | ||||||
| }); | ||||||
| } | ||||||
|
|
||||||
| public function down(): void | ||||||
| { | ||||||
| Schema::dropIfExists('twitch_event_logs'); | ||||||
| } | ||||||
| }; | ||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| <?php | ||
|
|
||
| declare(strict_types=1); | ||
|
|
||
| use He4rt\IntegrationTwitch\Http\Controllers\TwitchWebhookController; | ||
| use He4rt\IntegrationTwitch\Http\Middleware\VerifyTwitchSignature; | ||
| use Illuminate\Support\Facades\Route; | ||
|
|
||
| Route::prefix('api/webhooks/twitch') | ||
| ->middleware(VerifyTwitchSignature::class) | ||
| ->group(function (): void { | ||
| Route::post('/eventsub', TwitchWebhookController::class) | ||
| ->name('twitch.eventsub.webhook'); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| <?php | ||
|
|
||
| declare(strict_types=1); | ||
|
|
||
| namespace He4rt\IntegrationTwitch\Console; | ||
|
|
||
| use He4rt\Identity\ExternalIdentity\Data\ClientAccessManager; | ||
| use He4rt\Identity\ExternalIdentity\Enums\CredentialsType; | ||
| use He4rt\Identity\ExternalIdentity\Enums\IdentityProvider; | ||
| use He4rt\Identity\ExternalIdentity\Enums\IdentityType; | ||
| use He4rt\Identity\Tenant\Models\Tenant; | ||
| use He4rt\IntegrationTwitch\Transport\Requests\Users\GetUsers; | ||
| use He4rt\IntegrationTwitch\Transport\TwitchHelixConnector; | ||
| use Illuminate\Console\Command; | ||
|
|
||
| final class LinkTwitchChannelCommand extends Command | ||
| { | ||
| protected $signature = 'twitch:link-channel {login : Twitch channel login name} {--tenant= : Tenant slug or ID}'; | ||
|
|
||
| protected $description = 'Link a Twitch channel to a tenant via ExternalIdentity'; | ||
|
|
||
| public function handle(TwitchHelixConnector $helix): int | ||
| { | ||
| $login = $this->argument('login'); | ||
| $tenantOption = $this->option('tenant'); | ||
|
|
||
| $response = $helix->send(new GetUsers(login: $login)); | ||
| $users = $response->json('data', []); | ||
|
|
||
| if (blank($users)) { | ||
| $this->error(sprintf("Twitch user '%s' not found.", $login)); | ||
|
|
||
| return self::FAILURE; | ||
| } | ||
|
|
||
| $twitchUser = $users[0]; | ||
| $broadcasterId = $twitchUser['id']; | ||
| $displayName = $twitchUser['display_name']; | ||
|
|
||
| $query = Tenant::query()->where('slug', $tenantOption); | ||
|
|
||
| if (is_numeric($tenantOption)) { | ||
| $query->orWhere('id', $tenantOption); | ||
| } | ||
|
|
||
| $tenant = $query->first(); | ||
|
|
||
| if (!$tenant) { | ||
| $this->error(sprintf("Tenant '%s' not found.", $tenantOption)); | ||
|
|
||
| return self::FAILURE; | ||
| } | ||
|
|
||
| $existing = $tenant->providers() | ||
| ->where('provider', IdentityProvider::Twitch) | ||
| ->where('external_account_id', $broadcasterId) | ||
| ->first(); | ||
|
|
||
| if ($existing) { | ||
| $this->warn(sprintf("Channel '%s' (ID: %s) is already linked to tenant '%s'.", $login, $broadcasterId, $tenant->name)); | ||
|
|
||
| return self::SUCCESS; | ||
| } | ||
|
|
||
| $tenant->providers()->create([ | ||
| 'tenant_id' => $tenant->getKey(), | ||
| 'type' => IdentityType::External, | ||
| 'provider' => IdentityProvider::Twitch, | ||
| 'credentials_type' => CredentialsType::OAuth2, | ||
| 'credentials' => ClientAccessManager::make(), | ||
| 'external_account_id' => $broadcasterId, | ||
| 'connected_at' => now(), | ||
| 'metadata' => [ | ||
| 'login' => $login, | ||
| 'display_name' => $displayName, | ||
| ], | ||
| ]); | ||
|
|
||
| $this->info(sprintf("Linked Twitch channel '%s' (ID: %s) to tenant '%s'.", $displayName, $broadcasterId, $tenant->name)); | ||
|
|
||
| return self::SUCCESS; | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a language identifier to the fenced code block.
This currently trips markdownlint MD040 and can fail docs linting in CI.
Proposed fix
📝 Committable suggestion
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 19-19: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents