View Reports in UI

README.md

@@ -72,6 +72,8 @@ The following environment variables can be used to define UI behaviour.
 
 **BUCKET_NAME** - the name of the GCP bucket for staging uploaded files.  Only required when FILE_STORE is set as 'GCP'
 
+**OUTPUT_BUCKET_NAME** - the name of the GCP bucket containing generated analysis reports and workflow outputs. Used by the reports UI to list and render markdown reports. Required for the 'View Reports' feature to list the generated reports.
+
 **UPLOAD_DIR** - defaults to 'uploads' when running in a local filestore
 
 **FLASK_SECRET_KEY** - must be a secure key in deployment

example_reports/example_in_numbers.md

@@ -0,0 +1,111 @@
+# Survey Feedback Analysis Report
+
+## 1. Introduction
+
+This report presents an analysis of feedback collected from a public survey designed to categorize organizational types, which included AI-generated questions. The feedback was gathered from the "other feedback" section of the survey, where respondents answered the question: "Do you have any other feedback about this survey?". The responses have been processed using ThemeFinder, a library that identifies themes, categorizes sentiment, and assesses the detail level of each response.
+
+The goal of this report is to provide a clear, data-driven overview of the feedback, highlighting key insights for easy consumption by both technical and non-technical stakeholders.
+
+## 2. Overall Response Statistics
+
+A total of **318 individual responses** were analyzed for this report.
+Encouragingly, there were **0 unprocessable responses**, indicating that all submitted feedback could be categorized and analyzed by ThemeFinder.
+
+## 3. Thematic Analysis
+
+ThemeFinder identified **8 distinct themes** within the feedback provided by respondents. These themes shed light on various aspects of the survey experience, from its design and functionality to the clarity of its questions and overall purpose.
+
+The distribution of responses across these themes is as follows:
+
+| Theme ID | Theme Description                                                                                                                                                              | Number of Responses | Percentage |
+| :------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------ | :--------- |
+| **C**    | **No additional feedback:** No additional feedback or comments are provided for the survey.                                                                                     | **159**             | 50.0%      |
+| **F**    | **Positive survey design:** The survey design is clear, easy-to-navigate, and well-structured, with relevant, insightful, and effective questions.                               | **98**              | 30.8%      |
+| **G**    | **Unclear survey questions:** Survey questions are unclear, repetitive, or irrelevant, and require more specific, flexible, and comprehensive answer options.                     | **51**              | 16.0%      |
+| **A**    | **Insufficient survey context:** The survey lacks sufficient context regarding its purpose, data usage, and question rationale. This leads to concerns about intrusiveness and data privacy. | **17**              | 5.3%       |
+| **E**    | **Inconsistent AI functionality:** AI functionality is inconsistent, with effective question generation but also unmet expectations for advanced features and ineffective AI questions. | **12**              | 3.8%       |
+| **D**    | **Survey length too brief:** The survey is too brief and could be slightly longer.                                                                                             | **7**               | 2.2%       |
+| **H**    | **Technical and UI issues:** Technical problems exist, including slow loading times, survey crashes, access difficulties, and UI/UX issues.                                       | **6**               | 1.9%       |
+| **B**    | **Future surveys encouraged:** Future surveys are encouraged, and participants are willing to engage in subsequent studies.                                                      | **4**               | 1.3%       |
+
+*(Note: Some responses mapped to multiple themes, so the sum of theme counts exceeds the total number of individual responses.)*
+
+### Key Thematic Insights:
+
+1.  **Dominant "No Feedback" (Theme C):** A significant portion of respondents (**159 responses, 50.0%**) indicated they had no further feedback. This often suggests a neutral or satisfactory experience, as seen in responses like "No" (response_id 1) or "none" (response_id 3). While not providing actionable insights, it's a baseline for overall contentment.
+
+2.  **Strong Positive Sentiment for Design (Theme F):** The second most frequent theme, with **98 responses (30.8%)**, highlighted positive aspects of the survey's design. Respondents appreciated its clarity, ease of navigation, and well-structured nature. Examples include "Very easy to navigate" (response_id 4), "Nice and simple" (response_id 8), and "The survey was clear and easy to follow." (response_id 69). This indicates a solid foundation for the survey's user experience.
+
+3.  **Area for Improvement: Unclear Questions (Theme G):** A notable **51 responses (16.0%)** pointed to issues with question clarity, repetitiveness, or relevance. This is a critical area for improvement. For instance, one respondent noted, "Questions were a little vague" (response_id 11), while another stated, "Two of the questions seem very similar - describing what my business does and describing my employer's activity. Wasn't sure how to answer these differently." (response_id 38). This suggests a need to refine question wording and provide more comprehensive answer options.
+
+4.  **Need for More Context (Theme A):** **17 responses (5.3%)** expressed concerns about the lack of context regarding the survey's purpose, data usage, and question rationale. This led to feelings of intrusiveness or discomfort. A respondent suggested, "More information as to why follow up questions were given would be helpful and make the user more comfortable" (response_id 6), and another felt, "It was intrusive with no context for why" (response_id 206). Providing a clear introduction and privacy statement could alleviate these concerns.
+
+5.  **Mixed AI Functionality (Theme E):** The AI-generated questions received mixed feedback from **12 respondents (3.8%)**. While some found them effective, others had unmet expectations or found them inaccurate. For example, "The automated questions felt a little inaccurate but otherwise easy to fill in" (response_id 14) and "I wasn't impressed with AI's attempts to raise intelligent questions" (response_id 257) highlight areas where AI performance could be enhanced.
+
+6.  **Minor Technical Issues (Theme H):** A small number of respondents (**6 responses, 1.9%**) reported technical or UI issues, such as "It took longer time to load when going to next question near the end of survey" (response_id 22) or "I had to complete the survey twice because the first time the survey crashed" (response_id 80). While low in frequency, these can significantly impact user experience for those affected.
+
+7.  **Desire for Longer Survey (Theme D):** A small group of **7 respondents (2.2%)** felt the survey was too brief and could be slightly longer, as indicated by responses like "Very short" (response_id 158) or "Maybe the questionnaire could have been a little bit longer" (response_id 175). This suggests an opportunity to gather more in-depth information without overburdening users.
+
+8.  **Encouragement for Future Engagement (Theme B):** Positively, **4 respondents (1.3%)** expressed willingness to participate in future studies, such as "My positive feedback is to encourage the repetion of these survey to many people" (response_id 64) and "I enjoyed this survey and would like to participate in future studies." (response_id 308). This indicates a segment of engaged users.
+
+## 4. Sentiment Analysis
+
+The sentiment analysis provides a numerical breakdown of how respondents generally felt about the survey:
+
+| Sentiment Category | Number of Responses | Percentage |
+| :----------------- | :------------------ | :--------- |
+| **AGREEMENT**      | **142**             | 44.7%      |
+| **UNCLEAR**        | **112**             | 35.2%      |
+| **DISAGREEMENT**   | **64**              | 20.1%      |
+| **Total**          | **318**             | 100.0%     |
+
+### Sentiment Interpretation:
+
+*   **Predominantly Positive:** The highest number of responses fall under **AGREEMENT (142 responses)**, indicating a generally positive reception of the survey. This aligns with the high frequency of "Positive survey design" (Theme F) and the "No additional feedback" (Theme C) responses, which often imply satisfaction.
+*   **Significant Neutrality/Ambiguity:** A substantial portion of responses were categorized as **UNCLEAR (112 responses)**. Many of these likely correspond to the "No additional feedback" theme, where a simple "No" or "None" doesn't convey a strong positive or negative stance.
+*   **Identifiable Areas of Discontent:** While lower than agreement, **DISAGREEMENT (64 responses)** highlights specific pain points. These responses are crucial for identifying areas needing improvement, such as "Unclear survey questions" (Theme G), "Insufficient survey context" (Theme A), and "Inconsistent AI functionality" (Theme E).
+
+## 5. Response Detail (Evidence Richness)
+
+The analysis of response detail indicates the depth and usefulness of the feedback for further investigation:
+
+| Detail Level      | Number of Responses | Percentage |
+| :---------------- | :------------------ | :--------- |
+| **Surface-level** | **233**             | 73.3%      |
+| **Evidence-rich** | **85**              | 26.7%      |
+| **Total**         | **318**             | 100.0%     |
+
+### Detail Level Interpretation:
+
+*   **Majority Surface-Level:** The majority of responses (**233 responses, 73.3%**) were surface-level. These are typically short, concise comments that don't offer extensive detail or specific examples, such as "All great" (response_id 2) or "No" (response_id 9). While they contribute to overall sentiment, they offer limited actionable insights.
+*   **Valuable Evidence-Rich Responses:** Despite being a smaller proportion (**85 responses, 26.7%**), the evidence-rich responses are highly valuable. These responses provide specific examples, suggestions, or detailed descriptions of issues, making them ideal for driving improvements. Examples include "I wasn't sure about the automated follow-up questions as they seem repetitive (at least the 1st one)" (response_id 5) and "It took longer time to load when going to next question near the end of survey" (response_id 22). These responses directly inform the thematic analysis and provide concrete points for action.
+
+## 6. High-Level Summary & Recommendations
+
+Overall, the survey received a **generally positive reception**, with a large number of respondents either expressing satisfaction or having no further comments. The **design and ease of use are clear strengths**, frequently praised by participants. This indicates that the basic user experience is well-executed.
+
+However, there are **clear opportunities for improvement**, particularly concerning the **clarity and relevance of survey questions**, and the **provision of sufficient context** about the survey's purpose and data handling. The AI-generated questions, while innovative, also show room for refinement to ensure consistency and accuracy.
+
+### Key Recommendations:
+
+1.  **Enhance Question Clarity and Options:**
+    *   **Action:** Review and refine survey questions, especially those identified as vague or repetitive (Theme G). Consider providing more specific, flexible, and comprehensive answer options, including "other" boxes where appropriate.
+    *   **Example:** Address feedback like "Questions were a little vague" (response_id 11) and "Two of the questions seem very similar" (response_id 38).
+
+2.  **Improve Survey Context and Transparency:**
+    *   **Action:** Add a clear introduction to the survey explaining its purpose, how the data will be used, and any privacy considerations (Theme A). This will build trust and encourage more detailed responses.
+    *   **Example:** Respond to suggestions such as "More information as to why follow up questions were given would be helpful" (response_id 6) and "It was intrusive with no context for why" (response_id 206).
+
+3.  **Refine AI Question Generation:**
+    *   **Action:** Continuously monitor and improve the AI's ability to generate relevant and accurate follow-up questions (Theme E). Ensure the AI avoids repetition and provides genuinely insightful prompts.
+    *   **Example:** Address comments like "The automated questions felt a little inaccurate" (response_id 14) and "I wasn't impressed with AI's attempts to raise intelligent questions" (response_id 257).
+
+4.  **Address Minor Technical/UI Issues:**
+    *   **Action:** Investigate and resolve reported technical glitches, such as slow loading times and occasional crashes (Theme H). Ensure the survey is accessible and functions smoothly across various devices and accessibility settings.
+    *   **Example:** Prioritize fixes for issues like "It took longer time to load" (response_id 22) and "survey crashed" (response_id 80).
+
+5.  **Consider Optimal Survey Length:**
+    *   **Action:** While most found the length appropriate, a small segment desired a slightly longer survey (Theme D). Evaluate if there are opportunities to add a few more relevant questions without increasing respondent burden, potentially gathering more "evidence-rich" data.
+    *   **Example:** Take into account feedback like "Very short" (response_id 158) when planning future iterations.
+
+By focusing on these actionable insights, the survey application can be significantly improved, leading to even higher respondent satisfaction and more valuable data collection in the future.

pyproject.toml

@@ -15,6 +15,7 @@ pyyaml = "^6.0.3"
 google-cloud-workflows = "^1.21.0"
 more-itertools = "^11.0.1"
 survey-assist-pii = { git = "https://github.com/ONSdigital/survey-assist-pii.git" }
+markdown = "^3.10.2"
 
 [tool.poetry.group.dev.dependencies]
 black = "^24.10.0"
@@ -29,6 +30,7 @@ mkdocs-material = "^9.7.1"
 mkdocstrings = {extras = ["python"], version = "^1.0.3"}
 types-pyyaml = "^6.0.12.20250915"
 detect-secrets = "^1.5.0"
+types-markdown = "^3.10.2.20260508"
 
 [tool.black]
 line-length = 100

src/theme_analysis_ui/app_templates/confirm.html

@@ -1,15 +1,15 @@
 {% extends "core.html" %}
-{% set page_title = "Confirm choices" %}
+{% set page_title = "Confirm progress" %}
 
 {% block pageContent %}
 <div class="ons-page__container ons-container ons-u-pt-m ons-u-pb-l">
     <main id="main-content" class="ons-grid ons-grid--column@xxs@s">
         <div class="ons-grid__col ons-col-12 ons-col@l-10">
 
             {% block page_body %}
-            <h1 class="ons-u-fs-xxl ons-u-mb-m">Confirm choices</h1>
+            <h1 class="ons-u-fs-xxl ons-u-mb-m">Confirm progress</h1>
             <p class="ons-p">
-                Coming soon - for now check the job started in GCP.
+                Coming soon - for now check the job progress in GCP.
                 <br>
                  {{ execution_name }}
             </p>

src/theme_analysis_ui/app_templates/core.html

@@ -2,17 +2,27 @@
 {%- if page_title -%} {%- set page_title_value = page_title + ' - Theme Analysis Prototype - Office for National Statistics' -%} {%- else -%} {%- set page_title_value = 'Theme Analysis Prototype - Office for National Statistics' -%} {%- endif -%}
 {%- block head -%}
 {% endblock %}
-{% set navigation= {
+{% set nav_items = [
+    {
+        "url": url_for("ui.index"),
+        "title": "Home"
+    }
+] %}
+
+{% if session.get("authenticated_user") %}
+    {% set _ = nav_items.append({
+        "url": url_for("ui.reports"),
+        "title": "View Reports"
+    }) %}
+{% endif %}
+
+{% set navigation = {
     "navigation": {
         "id": "main-navigation",
-        "itemsList": [
-        {"url": url_for("ui.index"),
-        "title": "Home"},
-        ],
-        "currentPath": current_path
-        }
+        "itemsList": nav_items,
+        "currentPath": request.path
     }
-%}
+} %}
 {% set pageConfig = {
     "meta": {
         "canonicalUrl": "/",

src/theme_analysis_ui/app_templates/example_report.html

@@ -0,0 +1,21 @@
+{% extends "core.html" %}
+{% set page_title = page_title or "Report" %}
+
+{% block pageContent %}
+<div class="ons-page__container ons-container ons-u-pt-m ons-u-pb-l">
+    <main id="main-content" class="ons-grid ons-grid--column@xxs@s">
+        <div class="ons-grid__col ons-col-12 ons-col@l-10">
+
+            {% block page_body %}
+            <h1 class="ons-u-fs-xxl ons-u-mb-m">{{ page_title }}</h1>
+
+            <p class="ons-u-mb-m">
+                <a href="{{ url_for('ui.reports') }}">Back to reports</a>
+            </p>
+
+            {{ html_report | safe }}
+            {% endblock %}
+        </div>
+    </main>
+</div>
+{% endblock %}