Include the token usage for every conversation and workspace (#788)

* Include the token usage for every conversation and workspace

Related: #418

This PR does introduces the changes necessary to track the
used tokens per request and then process them to return them
in the API.

Specific changes:
- Make sure we process all the stream and record at the very end
- Include the flag `"stream_options": {"include_usage": True},` so
the providers respond with the tokens
- Added the necessary processing for the API
- Modified the initial API models to display correctly the tokens and its price

* Moved token recording to DB

* Changed token usage code to get info from file and added GHA to get file periodically

* formatting changes

* Move model cost to dedicated folder

* Fix problems with copilot streaming

Author: aponcedeleonch

.github/workflows/update-model-costs.yml

@@ -0,0 +1,59 @@
+# This workflow updates the file in /model_cost_data/model_prices_and_context_window.json
+name: Update model prices and context window JSON file
+
+on:
+  workflow_call:
+  schedule:
+    - cron: '0 2 * * 0'   # Run every Sunday at 2:00 AM
+
+jobs:
+  update_model_prices:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      pull-requests: write    
+
+    steps:
+    - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
+
+    - name: Get the latest file
+      run: |
+        curl -Ss 'https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json' > model_cost_data/model_prices_and_context_window.json
+
+    - name: Check if file changed
+      id: check-model-prices
+      run: |
+          if ! git diff --quiet model_cost_data/model_prices_and_context_window.json ; then
+              echo "changed=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "changed=false" >> "$GITHUB_OUTPUT"
+          fi      
+
+    - name: Set git config
+      run: |
+        git config --local user.email "github-actions[bot]@users.noreply.github.com"
+        git config --local user.name "github-actions[bot]"
+    
+    - name: Get current date
+      id: date
+      run: |
+        echo "CURRENT_DATETIME=$(date +%Y-%m-%d)" >> $GITHUB_ENV
+    
+    - name: Generate PR if needed
+      if: steps.check-model-prices.outputs.changed == 'true'
+      run: |
+        git checkout -b update-model-prices-$GITHUB_SHA
+
+        git add model_cost_data/model_prices_and_context_window.json
+        git commit -m "Update model_prices_and_context_window.json to version generated on ${{ env.CURRENT_DATETIME }}"
+
+        echo "Pushing branch so we can create a PR..."
+        git push --set-upstream origin update-model-prices-$GITHUB_SHA
+
+        gh pr create --title "Update model_prices_and_context_window.json" \
+          --body "This PR updates the model_prices_and_context_window.json definition to the version generated on ${{ env.CURRENT_DATETIME }}" \
+          --repo "$GITHUB_REPOSITORY" \
+          --base main \
+          --head update-model-prices-$GITHUB_SHA
+      env:
+        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}        

migrations/versions/2025_01_28_0915-0c3539f66339_add_token_usage_columns.py

@@ -0,0 +1,47 @@
+"""add token usage columns
+
+Revision ID: 0c3539f66339
+Revises: 0f9b8edc8e46
+Create Date: 2025-01-28 09:15:54.767311+00:00
+
+"""
+
+from typing import Sequence, Union
+
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "0c3539f66339"
+down_revision: Union[str, None] = "0f9b8edc8e46"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # Begin transaction
+    op.execute("BEGIN TRANSACTION;")
+
+    # We add the columns to the outputs table
+    # Add the columns with default values to avoid issues with the existing data
+    # The prices of the tokens may change in the future,
+    # so we need to store the cost of the tokens at the time of the request
+    op.execute("ALTER TABLE outputs ADD COLUMN input_tokens INT DEFAULT NULL;")
+    op.execute("ALTER TABLE outputs ADD COLUMN output_tokens INT DEFAULT NULL;")
+    op.execute("ALTER TABLE outputs ADD COLUMN input_cost FLOAT DEFAULT NULL;")
+    op.execute("ALTER TABLE outputs ADD COLUMN output_cost FLOAT DEFAULT NULL;")
+
+    # Finish transaction
+    op.execute("COMMIT;")
+
+
+def downgrade() -> None:
+    # Begin transaction
+    op.execute("BEGIN TRANSACTION;")
+
+    op.execute("ALTER TABLE outputs DROP COLUMN input_tokens;")
+    op.execute("ALTER TABLE outputs DROP COLUMN output_tokens;")
+    op.execute("ALTER TABLE outputs DROP COLUMN input_cost;")
+    op.execute("ALTER TABLE outputs DROP COLUMN output_cost;")
+
+    # Finish transaction
+    op.execute("COMMIT;")