Burr -- Google Sheets Persister

[elijahbenizzy]

Is your feature request related to a problem? Please describe.
Heh, this is actually just a really cool idea. And, why not? Would make it super easy to debug, and google sheets is basically an extremely powerful (if not limited in scalability) database.

Describe the solution you'd like
Build a persister that saves to/loads from google sheets.

• Determine data models -- partition key = one sheet? App = one tab? All in a drive?
• Determine parameters for data model
• Figure out best way to wire through secrets (ensuring that we don't accidentally track it...)
• Test
• Add CSV/Excel as well, based on this

See https://github.com/DAGWorks-Inc/burr/blob/26cfcd272a26c58e88c6ec84b18682b8ad87a9bd/burr/integrations/persisters/b_redis.py#L19.

for a sample working persister

Describe alternatives you've considered
Not doing this for fear that someone will use it in prod (or maybe they should...).

[cmm25]

I am requesting more information as I am interested in this issue

[skrawcz]

@cmm25 sorry for the tardy response. What questions do you have here?

[cmm25]

Seeing this as a good first issue I am interested in handling this but I cannot quite understand the needed requirements.

[skrawcz]

@cmm25 sure let me try to unpack it.

1. The high level idea is that we could use google sheets as a database for persistence
2. e.g. use their python API to read and write data (see their quickstart)

Things to implement:

1. A GoogleSheetsPersister (e.g. here's the postgres one for example)
2. The persister should implement the required methods and then subsequently use the python google sheets API to read/write appropriately.
3. Then an example application that uses it and exercises the functionality.

How I would get started:

1. Have setup google credentials and are able to run https://developers.google.com/sheets/api/quickstart/python
2. Have forked the burr repository
3. Added the google python client as an optional dependency to the project
4. created the new persister class
5. design the "data model" for the google sheet. It might be simplest to mirror the table structure from the postgresql persister?
6. check in with us for feedback / thoughts.

Does that help?

[goutamk09]

@skrawcz I'd be interested in working on this.

Proposed plan:

• Mirror the existing Postgres persister structure first (same logical schema).
• Implement a GoogleSheetsPersister using the Google Sheets Python API.
• Add the Google client as an optional dependency (extras).
• Add unit tests (mocking the Sheets API) plus a minimal example app.

Before starting implementation, I’d like to confirm the data model direction:

• Is mirroring the Postgres schema the preferred starting point?
• For sheet layout, my initial proposal is:
  • one spreadsheet per deployment
  • a states tab keyed by (app_id, partition_key) storing serialized JSON + timestamps
  • optionally an events tab if needed for parity

Happy to adjust the layout if there’s a preferred structure before building too far.

[elijahbenizzy]

@goutamk09 This seems reasonable -- want to ipen up a PR and we can go from there?

[goutamk09]

@elijahbenizzy Thanks! I've opened PR #670 for the initial Google Sheets persister skeleton so we can iterate on it there.