Sync with upstream Ghost v6.10.1 #29
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ref https://linear.app/ghost/issue/NY-736/ ref https://linear.app/ghost/issue/NY-738/ - removed UTM dropdown from Web tab (stats and posts apps) - added stats filter component (behind UTM labs flag) - wired up new filters, updated endpoints to support all values - added filter sync logic between Web and Locations tabs for Audience filters It became clear that UTMs require a different approach when wanting to analyze the data. Rather than seeing which values were used for a specific page, it could often be more instructive to see where the user went *when* they arrived with a certain set of values, as UTM fields are all about identifying how users got to a particular site (e.g. ad or email campaign). We removed the old dropdown from the Web tab and created new filters that allow you to search and filter via Post, Source, Audience, and the various UTMs, as well as combine filters. For now, it's a more naive search with support for a single value and 'is equal to' logic. We'll look to expand this as we explore more use cases. We're less interested in allowing a full data slicer and want to be more intentional about providing what's sensible within the context.
no ref When trying to run e2e tests locally, the Tinybird container was sufficiently slow such that the script failed. I've added a retry mechanism to improve resiliency.
ref TryGhost@cc4eee9 We changed the UTM implementation and didn't disconnect the tests. I need to look in to why this didn't block the ref'd commit/PR merge. I'm skipping these tests as we'll be coming back through and want to cover similar cases.
ref https://linear.app/ghost/issue/PRO-1538 The PutObjectCommand doesn't play well with streams on GCS as there are some incompatibilities with signatures. By using `readFile` we can pull the whole file into a buffer in memory and upload it in a single chunk - which fixes the errors. This does increase the memory footprint with larger files, but it unblocks e2e testing for now, we're working on a chunked multi-part upload which will supercede this.
ref https://linear.app/ghost/issue/NY-722/migrate-ghost-codebase-to-kebab-case-file-naming-convention * Renamed all files to match to kebab-case format * Added ESLint kebab-case rule * To match our working agreement on filenames in Ghost repo
ref TryGhost/Ghost-Release#122 - having a prefix allows our release script to ignore failed steps when deciding if commit status is green and makes status clearer for anyone viewing job status in GitHub
ref https://linear.app/ghost/issue/NY-722/migrate-ghost-codebase-to-kebab-case-file-naming-convention * Renamed folders to match to kebab-case format * To match better our working agreement on filenames in Ghost repo
Changelog for v2.56.1 -> 2.56.2: - Updated i18n translations - [Auto-submit OTC code when all 6 digits are pasted or typed](TryGhost@56ea3fffde)
…Ghost#25577) * aligned closer to playwright conventions
refs https://linear.app/ghost/issue/BER-3077/improve-e2e-testing-integration The yarn dev:forward analytics integration was incomplete. These changes ensure we're properly configuring the necessary environment variables.
no ref We have had a lot of iterations on development setup for our analytics/Tinybird configuration. We've recently improved the `yarn dev:analytics` script with some other improvements to the `yarn dev` script (in the form of dev:forward for now). This PR cleans up all the various ways of generating tests data with the intended flow being: - `yarn dev:analytics` to spin up dev server(s), including Tinybird local - `yarn data:analytics:generate` to build data based on the Ghost db content; this could be seeded with `yarn reset:data` or our data-generator script - `yarn data:analytics:clear` to reset the data in Tinybird (truncates the db)
ref https://linear.app/ghost/issue/NY-787/ - added Koenig editor to welcome emails modal This editor allows the user to manipulate the content of the email. However, this isn't wired up to anything while we're waiting for the persistence changes to be merged.
ref https://linear.app/ghost/issue/NY-770/create-a-database-migration-to-add-automated-emails-table ## What This PR adds a new table to Ghost's database, called `automated_emails`. It also adds a new Claude Code "skill," which provides Claude Code with instructions on how to create a database migration using the slimer CLI tool. ## Why We are working on adding welcome emails for new members to Ghost. Currently Ghost can be configured to send a welcome email to members as soon as they sign up, but the content of the welcome email is hardcoded. Our next step is to allow users to update the content in the welcome email, including the subject and body of the email. This table is where we'll store the content of the welcome email, and any other future automated emails we may add to Ghost.
ref https://linear.app/ghost/issue/NY-771 This model enables CRUD operations on the automated_emails table, which will store customizable welcome email content for new members. Includes URL transformation for the lexical field to handle __GHOST_URL__ placeholders consistently with other models like posts.
… outbox (TryGhost#25521) ref https://linear.app/ghost/issue/NY-805 - this makes it so the welcome emails job works similar to the email analytics job, in that it just emits a `StartMemberWelcomeEmailJobEvent` - The `MemberWelcomeEmailsService` subscribes to that event and then processes the outbox - This had a positive improvement on CPU utilization, most visible/useful for app servers running many Ghost instances; TryGhost#25534 will further improve it by randomizing when the job starts for each instance
ref https://linear.app/ghost/issue/PRO-1538 - GCS doesn't support AWS streaming signature format, causing uploads to fail with SignatureDoesNotMatch. This can be fixed by using buffer instead of stream for uploads. But this will cause memory issues for large files - This commit implements a multipart upload for files > 10MB(Configurable) - Small files (<10MB): Use buffer upload (fs.readFile) - Large files (≥10MB): Use S3 multipart upload API for memory efficiency. Multipart uploads read file in 10MB chunks, keeping RAM usage constant - Added automatic cleanup with AbortMultipartUpload on errors - Multipart upload is S3-standard and works with both AWS S3 and GCS
ref TryGhost#23361 The German locale was lacking a couple of translations (mostly phrases related to the new verification code for member logins). As a native German speaker and freelance dev with German clients, it made sense for me to add these translations to the Ghost repo, so that everyone could profit from it. --------- Co-authored-by: Cathy Sarisky <42299862+cathysarisky@users.noreply.github.com>
ref https://linear.app/ghost/issue/BER-3096 We observed a lot of flakiness when running multiple workers on a single CI e2e test runner. This change-set makes test worker counts depend on an environment variable, and sets that variable to 1 on CI which reduces the worker count. In order to maintain CI test run durations, we increased the shards in the e2e matrix to 4.
…5596) ref https://linear.app/ghost/issue/NY-693/ - updated middleware to pass through query/search params with path With web analytics data, we need to pass through query parameters like `utm_campaign`. This is true of other parts of Ghost, though to a lesser extent (e.g. click tracking).
) ref https://linear.app/ghost/issue/NY-799/ - removed naive 'outlier' logic from summations Our YTD and monthly summaries were excluding days of high traffic (>10k). I don't recall why this check was in here, and it's not something we should have.
ref no-issue - Ghost redirects paths not ending with `/`. This avoids the redirect.
…host#25601) ref https://linear.app/ghost/issue/NY-710/ We weren't formatting all of the text using `toLocaleString`.
* Fixed member signup test that was failing on welcome email when member signs up * The solution was to use blog title as a from name, instead of generic Ghost name that was used before
ref https://linear.app/ghost/issue/BER-3070 This PR re-introduces the network app notification badge to the new sidebar. It displays conditionally when count > 0 and the network app isn't open, matching the previous Ember behaviour.
…5599) ref https://app.incident.io/ghost/incidents/220 ref TryGhost#25590 - during incident INC-120, we noticed duplicate requests to the ActivityPub site registration endpoint (`./ghost/activitypub/v1/site`) on Ghost (Pro) - Ghost (Pro) redirects requests without a trailing slash to the trailing slash version. Adding the trailing slash in the code avoids the unnecessary redirect
…TryGhost#25605) ref https://linear.app/ghost/issue/NY-377/ For various reasons, we had a few different implementations of the 'member sources is disabled'. This commit unifies them to a common component. Because we need to use framework and shade, we had to duplicate the component. This will be remediated when we eventually combine these apps.
Changelog for v2.56.2 -> 2.56.3: - Updated i18n translations
ref https://linear.app/ghost/issue/NY-772/add-automated-emails-crud-api-endpoints The new `automated_emails` model should only be accessible by Administrators+ in Ghost via API. This adds the appropriate permissions to Ghost's DB to enforce these permissions in the API endpoints, which will be included in a subsequent commit.
ref https://linear.app/ghost/issue/NY-810/general-improvements-in-analytics-to-work-better-with-the-new-filter - This PR moves the Locations tab under the Web tab in site-wide analytics. The main reason is that Locations is very much related to Web traffic both in terms of data and functions (filtering). Now that we add more filtering options for web related data it makes sense to merge Locations under Web. - This change also allows "click-through" filtering — users can click on a Location, Source etc. to activate filtering by that value. With Locations on a separate tab, this experience becomes very scattered / impossible to do it in a nice way. - Quite a few visual details were off in Analytics in general which are all related to this change: spacings were too wide, some emphasis (icons) was missing on the sub navigation, keyboard navigation was missing for the filters component (for advanced users) etc. - Again, related to the above, the current NavBar component was not flexible enough to be used for the new filtering mechanism. The usage of CSS grid makes it much easier to use it for the new filtering UX. --------- Co-authored-by: Steve Larson <9larsons@gmail.com>
ref https://linear.app/ghost/issue/NY-772 This PR adds an API endpoint for the newly created`automated_emails` table. The endpoint will be used by the frontend settings UI to persist the user generated email content for welcome emails.
…eanup (TryGhost#25609) ref https://linear.app/ghost/issue/NY-821 - makes it so if welcome emails are configured and a member signs up via portal, a welcome email will send right away instead of waiting for the periodic job - checking `this.processing` prevents multiple instantiations; and because of how the batching in the outbox processing works, new members created while the outbox is processing will get picked up - we still have the periodic (5min) job that runs to process the outbox - this will be useful for retries and making sure entries don't get missed
no issue This tab has been removed from the analytics UI, so we no longer need this test. It was able to be merged with a failing test because the E2E tests are not currently taken into account by this repo's branch protection, which I'll fix in a separate PR. For now, this gets the E2E tests back into a passing state.
ref https://linear.app/ghost/issue/BER-3081/slice-1-the-search-result-shouldnt-be-cleared-on-keystroke Previously, when typing search queries in the search modal, there was a jarring flash where the UI would go blank while fetching results. This happened because the results state was cleared immediately when a new search started.
* added a way to merge e2e reports for React failed test runs separately, so that you can see traces and screenshots of the tests failed for Ember and React e2e test runs * added React E2E Tests Failed PR comment generator, now you will see one for React and one for Ember in PR
…#25621) ref https://linear.app/ghost/issue/PRO-1538/ - Setting highWaterMark to chunkSize so Node.js reads larger chunks from disk, reducing the number of Buffer.concat operations needed. - This is a Node.js internal optimization that doesn't change functional behavior. Existing tests already verify chunks are yielded correctly and multipart uploads work as expected. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Configures fs.createReadStream to use highWaterMark equal to the multipart chunk size in S3Storage’s chunk reader. > > - **Storage**: > - Update `ghost/core/core/server/adapters/storage/S3Storage.ts`: > - In `readFileInChunks`, set `fs.createReadStream(filePath, {highWaterMark: chunkSize})` to align read size with multipart chunking. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit c528110. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
* updated eslint playwright plugin
ref https://linear.app/ghost/issue/NY-824 - this decouples the processing of the outbox from the member welcome emails - outbox has a handler pattern so it knows what to do based on certain events (like `MemberCreated`) - this will allow us to reuse the member welcome email sending elsewhere too, like a "test email" in the UI
ref https://linear.app/ghost/issue/BER-2995 ref TryGhost#25388 Private sites should not federate content via ActivityPub, as the content is intended to be restricted to authenticated users only - Added `is_private` check to `isSocialWebEnabled()` calculated setting - Added `is_private` to dependents array for cache invalidation - Added event listener for `settings.is_private.edited` - Updated Network settings UI to show contextual error message when disabled
) ref https://linear.app/ghost/issue/ONC-1305 When a member's subscription was cancelled while an admin had the member page open, saving the member (e.g. changing a note) would send stale tier data to the API. The backend interpreted this as adding a complimentary tier, incorrectly changing the member's status from 'free' to 'comped' The fix removes tiers from the member serializer payload since tier changes are handled through dedicated API calls (add/remove complimentary), not through normal member saves
closes https://linear.app/ghost/issue/BER-3087/make-admin-forward-feature-flag-work-for-self-hosters Copies the admin forward build assets (a superset of the Ember admin assets) to the location expected by Ghost core and conditionally serves the correct entrypoint based on the existing feature flag. The development Dockerfile and the archive task has been updated to correctly copy the assets as well.
ref https://linear.app/ghost/issue/NY-833/refine-design-details-for-utm-filtering - Post analytics header Y gap was too large - Title attribute of values was not set in the filter dropdown so when it's truncated - The filter row spacing was 1px off related to the sidebar - Further generic minor design improvements
ref https://linear.app/ghost/issue/NY-812/ - added new tests for analytics filters (all behind the utmTracking labs flag)
ref TryGhost@eb5ef8e CI failed to check this job and it merged early - this is a patch to the failing tests. We're looking separately on the fix to CI.
…5626) ref https://linear.app/ghost/issue/NY-773 - Updates member welcome emails so that they fetch content from `automated_emails` instead of hard-coded - Includes logic to check if the automated email is active - Cleans up some of the boundaries between member welcome emails and the outbox; outbox doesn't need to worry about the mail configuration anymore
ref https://linear.app/ghost/issue/NY-827/ - added api_top_devices - added device as a session-level field User agent/device is a session-level metric, so it's been included in the session_data mv. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Adds a top devices endpoint and promotes device to a session-level field with filtering, updating pipes, fixtures, and tests. > > - **Tinybird endpoints**: > - `endpoints/api_top_devices.pipe`: new endpoint returning `device` with visit counts; integrates with `filtered_sessions` and supports pagination. > - **Pipes/data model**: > - `pipes/mv_hits.pipe`: extract `device` from payload and default to `unknown`; remove UA-derived device; propagate `device` downstream. > - `pipes/mv_session_data.pipe`: aggregate and expose session-level `device` via `argMin`. > - `pipes/filtered_sessions.pipe`: add session-level filter `device` and update description. > - **Fixtures & tests**: > - `fixtures/analytics_events.ndjson`: add `device` field values (e.g., `desktop`, `bot`). > - Add `tests/api_top_devices.yaml`; extend `tests/api_kpis.yaml`, `tests/api_top_locations.yaml`, `tests/api_top_pages.yaml` with device-filter cases. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 992fc00. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
ref https://linear.app/ghost/issue/NY-836/add-minimaleditor-component-to-admin-x-design-system In the member emails section of settings, we need to allow the user to edit the content of the welcome email. Initially we'll just include basic text formatting, but soon we'll want to add additional cards, like buttons for call-to-actions. Currently this modal uses the HtmlEditor, which already existed and was used in e.g. newsletter settings to edit the footer content. This worked initially, but the HtmlEditor outputs HTML, not Lexical, which breaks when we try to wire it up to the API (which expects lexical). This PR: - Creates a new `KoenigEditorBase` component with the common boilerplate needed to launch an instance of the editor - Updates `HtmlEditor` to be a thin wrapper around `KoenigEditorBase` to add the HTML output functionality - Exports the `KoenigEditorBase` from the design system so we can create custom editor instances for other use-cases, i.e. for member welcome emails
ref https://linear.app/ghost/issue/NY-827/ ref TryGhost#25634 - added api_top_devices - enabled endpoint in tinybird service - passed through devices option in ghost stats endpoints This is the front end changes that accompany the Tinybird changes (see ref) to enable the Devices filter/dropdown in the Analytics > Web view.
…5614) ref https://linear.app/ghost/issue/NY-829/add-ability-to-toggle-each-type-of-welcome-email-onoff The Member Welcome Emails settings UI was previously static with no backend integration. Toggle states were not persisted and emails could not be created or managed. This connects the UI to the new automated_emails API endpoint, enabling users to toggle welcome emails on/off with state persisting across page refreshes. Different default Lexical content is used for free vs paid member emails. This doesn't yet allow the content to be edited; this will be included in a subsequent comment.
ref https://linear.app/ghost/issue/NY-830/add-ability-to-edit-welcome-emails-subject-content-and-sender-info The member welcome emails UI in settings currently allows the user to toggle the emails on/off, but does not allow them to edit the content or any other fields. This adds the ability to edit the content, subject, and sender info of the welcome emails after enabling them.
ref https://linear.app/ghost/issue/DES-1256/thousands-separator-missing-from-percentage-format-in-analytics - Thousands separator was missing for percentage values in Analytics charts
…t#25641) ref https://linear.app/ghost/issue/NY-816/ ref TryGhost@cc4eee9 - ported stats-filter component to the Post Analytics (posts) app - added tests Unfortunately, given the current structure of our React apps, we have limited options on how to treat shared components. Until we merge these apps into one (ideally into `admin`), we'll need to copy components like this and do dual maintenance. The ref'd commit added the new stats filter popover to the Analytics page. This commit brings that over to Posts.
no ref This test is passing locally and we have redundant other tests checking filters, so it's not critical that this test is sorted out at the moment. We've had some flakiness due to slowness with the Tinybird event ingestion that we can revisit at large vs. extending a timeout on this test.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Syncing fork to upstream release
v6.10.1.