feat: add statistics dashboard module (spp_statistics_dashboard)#77
feat: add statistics dashboard module (spp_statistics_dashboard)#77
Conversation
Materializes published statistics into spp.dashboard.data snapshot table and renders via kanban/list/pivot/graph views with area and program filtering. Includes queue_job background refresh, k-anonymity suppression, three-tier security, and daily cron.
Replace _sql_constraints with models.Constraint (Odoo 19 API) and remove invalid numbercall field from ir.cron definition.
Remove invalid groups_id field from ir.actions.server and add title attributes to FontAwesome icons in kanban view template.
Use draft_name instead of name when creating spp.area records (name is computed). Fix view load tests to assert on 'arch' key. Use correct area_level values (computed from parent hierarchy).
… names Replace models.Constraint with COALESCE-based unique index in init() to handle NULL area_id/program_id correctly. Fix groups_id -> group_ids for Odoo 19 user model.
Use has_group() for group membership checks instead of assertIn on group_ids. Avoid unique constraint collision in manager_can_create test.
Narrow except clause in _refresh_statistic to catch only expected
errors (ValueError, TypeError, KeyError, AttributeError) instead of
broad Exception. Fix ACL entry IDs to follow access_{model}_{group}
naming convention.
Three issues found during live testing with demo data: 1. System-wide scope used scope_type "area" with area_id=False, which the area resolver treats as empty (returns 0 registrants). Changed to scope_type "cel" with expression "true" to match all registrants. 2. Double suppression: aggregation service already applies suppression and returns display strings (e.g., "<5"). _upsert_data then called stat.apply_suppression() again. Removed the redundant call. 3. Refresh menu not accessible when dashboard is empty (chicken-and-egg). Added "Refresh Statistics" as a dedicated submenu item, always accessible to managers. Also removed program_id iteration from refresh since the scope resolver does not support program filtering (was creating duplicate identical rows).
The CEL scope type fails because the scope resolver's env.get() check on the AbstractModel executor returns a falsy empty recordset, causing it to log "CEL executor not available" and return 0 registrants. This made all system-wide values show as suppressed (<5) regardless of actual population size. Fix: for system-wide scope, query all registrant IDs directly and use the "explicit" scope type which doesn't depend on the CEL executor. Add integration tests that call through the real aggregation pipeline (no mocks) to catch scope resolution and suppression issues.
The disabled_members variable used `m.disabled != null` as its aggregate filter, but `res_partner.disabled` is Odoo's archival timestamp field (when a record was deactivated), not a disability indicator. The CEL evaluator returns False for NULL Datetime fields instead of None, so `False != None` evaluated to True for every member, producing incorrect counts (431 "disabled" members when none actually had disabilities). Removes both the CEL variable (cel_var_disabled_members) and the statistic (stat_disabled_members) from demo data, and updates tests.
Odoo ORM returns False for unset non-boolean fields (Datetime, Date, Char, Many2one, etc.), but CEL null maps to Python None. This caused expressions like `m.disabled != null` to evaluate as `False != None` which is True, matching every record even when the field is actually NULL in the database. _safe_getattr now detects Odoo records (via _fields attribute) and converts False to None for non-boolean fields, so CEL null comparisons behave correctly. Boolean fields retain their False value.
More descriptive module name that clearly indicates this is the dashboard layer for the statistics subsystem. Updates all XML ID references, system parameter prefixes, and test refs.
The dashboard now computes statistics for each active program in addition to system-wide and per-area dimensions. Program scope is built by querying enrolled members via get_beneficiaries() and passing their IDs as an explicit scope to the aggregation service. Refactors _refresh_statistic into _refresh_scope for clearer separation of the iteration logic from the per-scope computation.
- Add Program to search panel sidebar (multi-select with counters) - Add area level filters (Level 0-3) for quick admin-level filtering - Add "Per Program" scope filter - Add "Area Level" group-by option - Show program_name column by default in list view
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a significant new feature: a comprehensive statistics dashboard module ( Highlights
Changelog
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new spp_statistics_dashboard module, which provides a materialized dashboard for statistics. The implementation is very thorough, including background refresh jobs, k-anonymity suppression, comprehensive views, and well-defined security roles. The PR also includes a fix for spp_cel_domain to correctly handle Odoo's False values for unset fields in CEL expressions. The code quality is high, with excellent test coverage across unit, integration, and access control scenarios. I have one suggestion regarding a potential scalability issue in how system-wide scope is handled, which could be problematic on very large datasets.
| # System-wide scope: query all registrant IDs directly and use | ||
| # explicit scope. We can't use CEL scope because the scope resolver's | ||
| # env.get() check on the AbstractModel executor returns falsy. | ||
| all_ids = self.env["res.partner"].sudo().search([("is_registrant", "=", True)]).ids |
There was a problem hiding this comment.
Loading all registrant IDs into memory with search(...).ids could lead to high memory consumption and performance issues on systems with a very large number of registrants (e.g., millions). This presents a potential scalability bottleneck.
While the comment explains this is a workaround, it would be ideal to investigate if the underlying issue with the CEL scope resolver can be fixed to allow for a domain-based aggregation, which would be more memory-efficient. If that's not feasible in the short term, this implementation carries a scalability risk that should be acknowledged.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 19.0 #77 +/- ##
==========================================
+ Coverage 58.09% 64.91% +6.82%
==========================================
Files 188 384 +196
Lines 10797 20487 +9690
==========================================
+ Hits 6272 13300 +7028
- Misses 4525 7187 +2662
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
…alls Both sudo() usages are intentional: - ir.config_parameter.sudo(): standard Odoo pattern for system params - res.partner.sudo(): needed for queue_job cron context, reads IDs only
Adds an all_registrants scope type to the scope resolver that searches
for all registrants server-side. This avoids loading millions of IDs
into the caller's memory, which was a scalability concern when the
dashboard used explicit scope with all registrant IDs.
The dashboard's system-wide scope now uses {"scope_type": "all_registrants"}
instead of querying all IDs and passing them via explicit_partner_ids.
…nor issues - Use scope builders from spp_aggregation.services instead of inline dicts - Bulk-load existing rows to eliminate N+1 search per upsert - Hoist label/config lookup out of per-scope loop - Pass program_ids from action_refresh_all to avoid repeated queries - Remove dead `is not None` branch and redundant `or 0` - Fix misleading docstrings on all_registrants scope
1 participant
Summary
spp_statistics_dashboardmodule: materialized dashboard for published statistics with kanban KPI cards, list, pivot, and graph viewsqueue_job(manual trigger + daily cron) computes statistics across three dimensions: system-wide, per-area, and per-programAlso includes
Falsefor unset non-boolean fields (Datetime, Date, etc.), but CELnullmaps to PythonNone._safe_getattrnow normalizesFalsetoNonefor non-boolean Odoo fields, fixing null comparisons likem.disabled != nulldisabled_membersCEL variable/statistic that referenced Odoo's archivaldisabledtimestamp field instead of a disability indicatorTest plan
spp_statistics_dashboard: 48 tests (unit, integration, access control, views)spp_cel_domain: 571 tests (includes 2 new null normalization tests)spp_mis_demo_v2: 254 tests (updated for removed disabled_members)