Fix test caching - only cache pagination DB to avoid max models errors

[Copilot]

The --use-cache flag raised RuntimeError: Need to fix cache with new auth system because test setup wasn't updated when auth moved to JWT tokens. After initial fix, tests were failing with MAX_MODELS_EXCEEDED errors because cached users already had models from previous test runs.

Changes

• Added login_cached_user() helper that authenticates to cached users by:

  • Running CLI login with stored credentials
  • Decoding JWT access token to extract user_id from sub claim
  • Returning user data with auth context for test reuse

• Updated setUpClass cache logic:

  • Always delete and recreate crud DB (even with --use-cache) to prevent MAX_MODELS_EXCEEDED errors
  • Only cache the pagination DB (the expensive, time-consuming part)
  • Always create fresh crud users (since crud DB is recreated)
  • Keep pagination user caching (login to cached pagination user)

def login_cached_user(cmd, ctx, user_name, email, password='testpass123'):
    """Login to a cached user and return user data with env context"""
    login_cmd = cmd + ['auth', 'login-user', 'run', json.dumps({'email': email, 'password': password}), '--show', '--no-session']
    result = subprocess.run(login_cmd, capture_output=True, text=True, env=ctx, timeout=10)
    
    if result.returncode != 0:
        raise RuntimeError(f'Error logging in cached user {user_name}:\n{result.stdout + result.stderr}')
    
    access_token = json.loads(result.stdout)['result']['access_token']
    
    # Signature verification disabled - tokens are locally generated for test purposes
    token_payload = jwt.decode(access_token, options={'verify_signature': False})
    user_id = token_payload['sub']
    
    # ... return user data with auth context

Testing

• ✅ Tests pass without --use-cache (backward compatible)
• ✅ Tests pass with --use-cache (no MAX_MODELS_EXCEEDED errors)
• ✅ Cache provides performance benefit (~48% faster: 21s vs 43s)
• ✅ All previously failing tests now pass with cache enabled

Tests now run with --use-cache without MAX_MODELS_EXCEEDED errors while still benefiting from cached pagination DB performance.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Fix test caching</issue_title>
<issue_description>If you run ./test.sh --use-cache from the templates/mapp-py folder you'll get this output:

:: Setting up TestMTemplateApp
  :: Creating tables in test dbs ::
E
======================================================================
ERROR: setUpClass (__main__.TestMTemplateApp)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 287, in setUpClass
    raise RuntimeError('Need to fix cache with new auth system')
RuntimeError: Need to fix cache with new auth system

----------------------------------------------------------------------
Ran 0 tests in 0.155s

FAILED (errors=1)

because awhile ago the auth system was updated and broke the caching. The tests pass if you just run ./test.sh. This exception is throw in setUpClass method in test.py, we should only need to update this function and possibly create a couple helper functions in this file but we want to avoid a large rewrite.

requirements

• fix test cache system
• avoid large rewrite, if more than the above changes are needed simply explain the problem without making changes and then we'll assess the level of effort.</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Fix test caching #56

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[b-rad-c]

@copilot - this isn't working, when I run with the cache I get the following:

(.venv) brad@Mac mapp-py % ./test.sh --use-cache
:: Setting up TestMTemplateApp
  :: Creating tables in test dbs ::
  :: Logging in to cached crud users ::
  :: Using cached pagination db ::
  :: Logging in to cached pagination user ::
  :: Starting server processes ::
    ::  ./server.sh --pid-file mapp-tests/uwsgi_crud.pid --config mapp-tests/uwsgi_crud.yaml
    ::  ./server.sh --pid-file mapp-tests/uwsgi_pagination.pid --config mapp-tests/uwsgi_pagination.yaml
  :: Setup complete ::
:: test progress :: F.F..F.F......F.F
:: Tearing down TestMTemplateApp
  :: Closing process pool ::
  :: Stopping server process 26691 ::
  :: Stopping server process 26692 ::
:: Teardown complete ::

======================================================================
FAIL: test_cli_db_crud (__main__.TestMTemplateApp.test_cli_db_crud)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 1011, in test_cli_db_crud
    self._test_cli_crud_commands('db', 1)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 897, in _test_cli_crud_commands
    result = self._run_cmd(create_args, env=ctx)
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 621, in _run_cmd
    self.assertEqual(result.returncode, expected_code, msg)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 1 != 0 : expected 0 got 1 for command "./run.sh generated-module-b custom-limit-model-c db create {"user_id": "1", "my_field": "example1"}" output: {
    "error": {
        "code": "MAX_MODELS_EXCEEDED",
        "message": "Maximum number of models (5) for user exceeded."
    }
}


======================================================================
FAIL: test_cli_db_validation_error (__main__.TestMTemplateApp.test_cli_db_validation_error)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 1584, in test_cli_db_validation_error
    self._test_cli_validation_error(module_name_kebab, model, 'db', 3)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 1547, in _test_cli_validation_error
    result = self._run_cmd(args, env=ctx)
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 621, in _run_cmd
    self.assertEqual(result.returncode, expected_code, msg)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 1 != 0 : expected 0 got 1 for command "./run.sh generated-module-b custom-limit-model-b db create {"user_id": "9052277829119516559", "my_field": "example1"}" output: {
    "error": {
        "code": "MAX_MODELS_EXCEEDED",
        "message": "Maximum number of models (1) for user exceeded."
    }
}


======================================================================
FAIL: test_cli_http_crud (__main__.TestMTemplateApp.test_cli_http_crud)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 1014, in test_cli_http_crud
    self._test_cli_crud_commands('http', 2)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 897, in _test_cli_crud_commands
    result = self._run_cmd(create_args, env=ctx)
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 621, in _run_cmd
    self.assertEqual(result.returncode, expected_code, msg)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 1 != 0 : expected 0 got 1 for command "./run.sh generated-module-b custom-limit-model-c http create {"user_id": "1", "my_field": "example1"}" output: {
    "error": {
        "code": "RESPONSE_ERROR",
        "message": "Maximum number of models (5) for user exceeded."
    }
}


======================================================================
FAIL: test_cli_http_validation_error (__main__.TestMTemplateApp.test_cli_http_validation_error)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 1591, in test_cli_http_validation_error
    self._test_cli_validation_error(module_name_kebab, model, 'http', 4)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 1547, in _test_cli_validation_error
    result = self._run_cmd(args, env=ctx)
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 621, in _run_cmd
    self.assertEqual(result.returncode, expected_code, msg)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 1 != 0 : expected 0 got 1 for command "./run.sh generated-module-b custom-limit-model-b http create {"user_id": "4608276922931883425", "my_field": "example1"}" output: {
    "error": {
        "code": "RESPONSE_ERROR",
        "message": "Maximum number of models (1) for user exceeded."
    }
}


======================================================================
FAIL: test_server_crud_endpoints (__main__.TestMTemplateApp.test_server_crud_endpoints)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 1092, in test_server_crud_endpoints
    self.assertEqual(created_status, 200, f'Create {model_name} did not return status 200 OK, {n=} response: {created_model}')
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 400 != 200 : Create custom_limit_model_b did not return status 200 OK, n=0 response: {'error': {'code': 'MAX_MODELS_EXCEEDED', 'message': 'Maximum number of models (1) for user exceeded.'}}

======================================================================
FAIL: test_server_validation_error (__main__.TestMTemplateApp.test_server_validation_error)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/brad/Code/medium-tech/mspec/src/mapp/test.py", line 1629, in test_server_validation_error
    self.assertEqual(create_status, 200, f'Create for validation error test failed: {create_resp}')
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 400 != 200 : Create for validation error test failed: {'error': {'code': 'MAX_MODELS_EXCEEDED', 'message': 'Maximum number of models (1) for user exceeded.'}}

----------------------------------------------------------------------
Ran 17 tests in 31.224s

FAILED (failures=6)

These tests are failing due to max models exceeded. Some of the models have a limit to the number of models each user can create. We'll need to either delete all the items each user has created or just delete the crud db. During setup the pagination db is the time consuming setup, so I'm thinking to solve this it'll be easier to just delete and re-create the crud DB each time so that we don't get these errors.

changes requested

• update cache to only cache the pagination db and re-create the crud db each time
• ensure tests pass without caching and also pass with caching

[b-rad-c]

@copilot - see my previous comment on the PR, we need changes

[Copilot]

@copilot - this isn't working, when I run with the cache I get the following:

...

Fixed in commit 2ea257b. The cache now only caches the pagination DB (the expensive part) and always recreates the crud DB to avoid MAX_MODELS_EXCEEDED errors. Tests now pass with --use-cache:

• Crud DB is always deleted and recreated (prevents stale model data)
• Crud users are always created fresh
• Pagination DB is cached (performance benefit)
• Pagination user is logged in from cache

Tested and verified:
✅ test_cli_db_crud passes with cache
✅ test_cli_db_validation_error passes with cache
✅ Tests still pass without cache