Store world save variables at characters db

[schell244]

🍰 Pullrequest

Store world persistent variables in characters db.
This enables applying a fresh world db without losing any saved variables.

Refactored storage from a linear vector to an unordered_map for O(1) lookup, replaced the DELETE+INSERT transaction with a single REPLACE INTO, and removed the unused _InsertVariable and LoadVariable functions.

How2Test

• Apply the changes from this PR
• Start the server and verify that save states are now stored in world_persistent_variables in the characters db
• Trigger a world state change (start the Stranglethorn Vale Fishing event and hand in 40 tasty fish)
• Apply a fresh world db and reboot the server → the event should still be completed for that week

Todo / Checklist

• None

[ratkosrb]

World states is already a term that exists, and it's something that's sent to client. We should not have confusing naming like this, because it implies that those are the same world states blizzard sends if you name the table like this, and that is not what it contains. Variables is our own custom indexes and values, not using blizzard's ids for stuff.

[0blu]

variables is too generic, especially in a database like characters.
Maybe something like this:

• persistent_world_data
• persistent_state
• world_persistence
• world_progress
• game_state
• realm_state

or another permutation of these words?

[schell244]

maybe world_persistent_variables?

[shudza]

... and make world database read only.

game_events is written to as well, or am I wrong?

[schell244]

... and make world database read only.

game_events is written to as well, or am I wrong?

Oh yes, you're right, there are querys on game_event_creature and game_event_creature_data. Would probably make sense to move those tables to charcter db as well at some point.

[shudza]

I don't think it makes sense to change only variables then. For example, variables stores stuff like resources for war effort, but the stage it self is stored in game_events. So if the point was to make world db ephemeral/read-only, this would probably cause issues.

[schell244]

Did not experience any issues with that change, but I agree that it makes sense to move the other tables as well, together with the variables. Will add some more changes when I have time.

[0blu]

I don't think it makes sense to change only variables then. For example, variables stores stuff like resources for war effort, but the stage it self is stored in game_events. So if the point was to make world db ephemeral/read-only, this would probably cause issues.

We are focusing on non-admin-command induced changes to the DB.

The current war effort stage is stored in variables at index 30050 and not game_event.
This would be covered by Schell's change. 👍

But (what you probably mean,) game_event has the column disabled which is modified during runtime.
I think we need to come up with a solution to store this column in the charDb.

---

@schell244 Here are some sql snippets you can try to verify your changes.
At the end the server should work with just a read-only world db. (ignoring admin commands to modifying game objects or npcs)

USE world; -- whatever your db name is

-- Creates `root_readonly` user with pw `root` which only has read access to all tables
CREATE USER 'root_readonly'@'%' IDENTIFIED BY 'root';
GRANT SELECT ON *.* TO 'root_readonly'@'%';
FLUSH PRIVILEGES;

-- Grad write permissions to game_event
GRANT INSERT, UPDATE, DELETE ON game_event TO 'root_readonly'@'%';
FLUSH PRIVILEGES;

-- Revoke write permission to game_event
REVOKE INSERT, UPDATE, DELETE ON game_event FROM 'root_readonly'@'%';
FLUSH PRIVILEGES;

[mserajnik]

Why move only variables when moving game_events too costs basically nothing and saves an annoying step (even if that is just running some admin commands) every time someone wants to re-create the world DB from scratch?

Considering:

1. Migration files are getting edited relatively often* (non-exhaustive list, only the ones that I didn't catch before the next Docker build), and the easiest and fastest way to correct the world DB state is usually to just re-create and re-migrate the world DB instead of manually fucking around with the SQL required to get to the intended post-edit DB state.
2. We already provide a DB dump and advertise it as "no updates required," which probably makes quite a few users consider re-creating the world DB the "canonical" update method.

I think this move should be done sooner rather than later.

_{*: And yes, my stance still is that migration files should never be edited since that requires users to both actively track the repo for such changes AND manually intervene whenever they happen (unless they want to accept not getting those edits), when just creating a new migration that fixes whatever was wrong with the old one achieves the same end result without requiring manual intervention.}

(Side note: for vmangos-deploy I am in the process of automating my world DB correction system so I don't have to manually specify the edit commits anymore, and I will add the capture and re-application of those tables' data too since I had forgotten about that completely until now, which is how I found this PR)

[ratkosrb]

I don't know what are you guys talking about. Game events are not character data. They belong in world db.

[shudza]

I don't know what are you guys talking about. Game events are not character data. They belong in world db.

naming-wise sure, but architecturaly it makes sense to move it to char db, and have the world db become read only which can reduce ram usage, make upgrades and development easier, maybe even speed up server boot etc.

[schell244]

Here are some sql snippets you can try to verify your changes.
At the end the server should work with just a read-only world db. (ignoring admin commands to modifying game objects or npcs)

Sorry for the delay - I’ve been a bit short on time lately, but I have this on my list.
I actually applied that change about two years ago and haven’t noticed any issues from moving the variables table since then. My understanding is that currently all important game event save data is already using variables.
But, I agree it makes sense to track which tables in the world db still require write access to be 100% sure (excluding gm commands ofc).

[0blu]

Btw I also believe that the game_event table belongs to the world DB.
But the disabled column is being abused by some scripts.

We need to create an alternative script_done (or similar) column in a new game_event_state table inside the character DB.

[Gamemechanicwow]

naming-wise sure, but architecturaly it makes sense to move it to char db, and have the world db become read only which can reduce ram usage, make upgrades and development easier, maybe even speed up server boot etc.

I doubt on reduced ram usage and speed up boot.

[shudza]

I doubt on reduced ram usage and speed up boot.

there is many ways to do this, eg. if you use separate mysql servers for world and everythung else, you can tune it for read workload and reduce useless caching (most of it is read only once, during boot). This can even go to the point where you just load the data and turn off the mysql instance dedicated to the world, but that would require more core changes.