Conversation
Vale Linting ResultsSummary: 2 warnings, 2 suggestions found
|
| File | Line | Rule | Message |
|---|---|---|---|
| manage-data/migrate-data.md | 55 | Elastic.Spelling | 'Snaphot' is a possible misspelling. |
| manage-data/migrate/migrate-data-using-reindex-api.md | 69 | Elastic.Spelling | 'multiiple' is a possible misspelling. |
💡 Suggestions (2)
| File | Line | Rule | Message |
|---|---|---|---|
| deploy-manage/deploy/cloud-enterprise/create-deployment.md | 50 | Elastic.Semicolons | Use semicolons judiciously. |
| manage-data/migrate/migrate-data-between-elasticsearch-clusters-with-minimal-downtime.md | 53 | Elastic.WordChoice | Consider using 'can, might' instead of 'may', unless the term is in the UI. |
The Vale linter checks documentation changes against the Elastic Docs style guide.
To use Vale locally or report issues, refer to Elastic style guide for Vale.
|
|
||
| The [reindex API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-reindex) offers a convenient way for you to migrate your {{es}} documents from a source index, data stream, or alias in one deployment to another. | ||
|
|
||
| This guide gives the example of reindexing a full index from an {{ech}} deployment an {{serverless-full}} project using the remote host parameters as shown in the [reindex from remote](elasticsearch://reference/elasticsearch/rest-apis/reindex-indices.md#reindex-from-remote) example. These steps can be adapted to other deployment types according to the tables listed in [User data migration guides](/manage-data/migrate-data.md#data-migration-guides). |
There was a problem hiding this comment.
"from an ECH deployment to an Elastic Cloud Serverless project"
|
|
||
| For more advanced use cases, including data modification using scripts or ingest pipelines, refer to the [Reindex indices examples](elasticsearch://reference/elasticsearch/rest-apis/reindex-indices.md#reindex-from-remote) and the [reindex API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-reindex) documentation. | ||
|
|
||
| :::{tip} |
There was a problem hiding this comment.
This tip doesn't apply for serverless destinations (as serverless currently only supports ECH sources). This is a general page and the example just happens to be ECH -> ECS, so the tip itself is still valid. But there's something about the placement right here that causes a bit of a context switch for me - or might tend to have readers expecting that it could apply in the example. I wonder if it could fit better somewhere else - maybe even back on the main migration page.
There was a problem hiding this comment.
Good point! I've moved this and the whitelisting guidance to a "Notes for migrating between other deployment types" section at the bottom of the page. On the main "Migrate your Elasticsearch Data" page there's also a note guiding people using private CA to the page for that.
|
|
||
| Basic authentication can be used in place of an API key, but an API key is recommended as a more secure option. | ||
|
|
||
| - The target deployment must be able to access your original source cluster to perform the reindex operation. Access is controlled by the {{es}} `reindex.remote.whitelist` user setting. |
There was a problem hiding this comment.
This section does not apply for serverless destinations - in serverless, we automatically allow access to all ECH endpoints as source. But this is great information and it needs to be kept somewhere. Do you think we could take this (and perhaps the tip about the certificates above) and shift it to a different section of this page? Somewhere clearly delineated from the guided example?
There was a problem hiding this comment.
For sure! I've put this in a new "Notes for migrating between other deployment types" section at the bottom of the page, along with the note for folks using private CA.
| :::{important} | ||
| Kibana assets must be migrated separately using the {{kib}} [export/import APIs](https://www.elastic.co/docs/api/doc/kibana/group/endpoint-saved-objects) or recreated manually. Refer to [Migration options](/manage-data/migrate-data.md#migration-options) for details about migrating different types of {{es}} data. | ||
|
|
||
| Index templates, data stream definitions, and ILM policies, must be in place _before_ you start reindexing data. However, be mindful that if you have any [ingest pipelines](/manage-data/ingest/transform-enrich/ingest-pipelines.md) configured, it's typically best to add these _after_ data migration so as to avoid re-transforming data that had already been transformed at the time that it was ingested into your source deployment (though if the data is idempotent, re-transforming is not a concern). |
There was a problem hiding this comment.
ILM does not apply for serverless - perhaps we should mention DLM too? Or just generalize with something like "data lifecycle settings" ?
There was a problem hiding this comment.
Changed to Index templates, data stream definitions, and data lifecycle settings must be in place...
|
Thanks for the superb suggestions @pete-naylor! All fixed. :-) |
shainaraskas
left a comment
There was a problem hiding this comment.
really great work.
I think that there are some surprisingly quick wins left in terms of recontextualizing the tutorials that currently only look at serverless/ech ... otherwise, some super nice improvements and a win for migrators everywhere.
left a bunch of comments so you prob want a re-review after taking a look. I am super slow so don't consider getting my review to be a requirement (but I can come do another round if you like)
There was a problem hiding this comment.
I don't think this title was too gory ... we might avoid renaming it to retain some seo signals even if the title was not perfectly ideal?
| ## Step 5: Reindex from remote `Source` cluster [ec-remote-reindex-step5] | ||
|
|
||
| You can now run a remote reindex operation on the {{ech}} `Destination` cluster from the `Source` cluster, as described in the [migration guide](/manage-data/migrate.md#ech-reindex-remote): | ||
| You can now run a `reindex from remote` reindex operation on the {{ech}} `Destination` cluster from the `Source` cluster, as described in [Migrate Elasticsearch data using the reindex API](/manage-data/migrate/migrate-data-using-reindex-api.md): |
There was a problem hiding this comment.
would leave these blank so the titles stay in sync
| You can now run a `reindex from remote` reindex operation on the {{ech}} `Destination` cluster from the `Source` cluster, as described in [Migrate Elasticsearch data using the reindex API](/manage-data/migrate/migrate-data-using-reindex-api.md): | |
| You can now run a `reindex from remote` reindex operation on the {{ech}} `Destination` cluster from the `Source` cluster, as described in [](/manage-data/migrate/migrate-data-using-reindex-api.md): |
| ## Migrate system indices using snapshot and restore | ||
|
|
||
| To restore system indices from a snapshot, follow the same procedure described in [](../migrate.md#ec-restore-snapshots) and select the appropriate feature states when preparing the restore operation, such as `kibana` or `security`. | ||
| To restore system indices from a snapshot, follow the same procedure described in [Migrate Elasticsearch data with minimal downtime using snapshots](/manage-data/migrate/migrate-data-between-elasticsearch-clusters-with-minimal-downtime.md) and select the appropriate feature states when preparing the restore operation, such as `kibana` or `security`. |
There was a problem hiding this comment.
| To restore system indices from a snapshot, follow the same procedure described in [Migrate Elasticsearch data with minimal downtime using snapshots](/manage-data/migrate/migrate-data-between-elasticsearch-clusters-with-minimal-downtime.md) and select the appropriate feature states when preparing the restore operation, such as `kibana` or `security`. | |
| To restore system indices from a snapshot, follow the same procedure described in [](/manage-data/migrate/migrate-data-between-elasticsearch-clusters-with-minimal-downtime.md) and select the appropriate feature states when preparing the restore operation, such as `kibana` or `security`. |
|
|
||
| # Migrate your {{es}} data [migrate-your-elasticsearch-data] | ||
|
|
||
| Transitioning between Elastic deployment types involves migrating your {{es}} data. This page helps you plan your migration by describing the main categories of data you may need to move (ingested user data, {{es}} system data, {{kib}} saved objects, and feature-specific data), the migration methods available for each, and where to find step-by-step guides for your scenario. |
There was a problem hiding this comment.
This is close enough to the next section that i'd skip the parenthetical personally
| Transitioning between Elastic deployment types involves migrating your {{es}} data. This page helps you plan your migration by describing the main categories of data you may need to move (ingested user data, {{es}} system data, {{kib}} saved objects, and feature-specific data), the migration methods available for each, and where to find step-by-step guides for your scenario. | |
| Transitioning between Elastic deployment types involves migrating your {{es}} data. This page helps you plan your migration by describing the main categories of data you may need to move, the migration methods available for each, and where to find step-by-step guides for your scenario. |
|
|
||
| - **Ingested user data**: All of the data that you've added into {{es}}, structured or unstructured, for your own applications. | ||
|
|
||
| - **{{es}} system data**: Configuration and state information stored in {{es}} [system indices](elasticsearch://reference/elasticsearch/rest-apis/api-conventions.md#system-indices). This data is used by {{es}} for its internal operations. |
There was a problem hiding this comment.
when I read "configuration data" I assume things like users might be involved, but that looks like it's feature/component data. can we be more precise here?
| - If you're migrating from a self-managed cluster that uses non–publicly trusted TLS certificates, including self-signed certificates and certificates signed by a private certificate authority (CA), refer to our guide [Reindex from a self-managed cluster using a private CA](/manage-data/migrate/migrate-from-a-self-managed-cluster-with-a-self-signed-certificate-using-remote-reindex.md). | ||
|
|
||
| - The target deployment must be able to access your original source cluster to perform the reindex operation. When you migrate to {{serverless-short}}, access to all {{ech}} endpoints is allowed automatically. For migrating to other deployment types, access is controlled by the {{es}} `reindex.remote.whitelist` user setting. |
There was a problem hiding this comment.
I am surprised there are so few "differences" here. it would really improve this tutorial to have an "ECH -> serverless" and "anything -> anything" pathway because there are so few differences. I think that the prerequisites could easily be generalized.
thinking harder about this, it just feels like there is one non-serverless step that needs to happen before the reindex call. could we just make this a tutorial with two major steps?
we could then turn the console step into a tabbed experience with one tab for serverless and one for everything else (if the API calls are different)
| navigation_title: Migrate data using Logstash | ||
| applies_to: | ||
| serverless: | ||
| deployment: |
| # Migrate {{ech}} data to {{serverless-full}} with {{ls}} [migrate-with-ls] | ||
| # Migrate {{es}} data using {{ls}} [migrate-with-ls] | ||
|
|
||
| [{{ls}}](logstash://reference/index.md) is a data collection engine that uses a large ecosystem of [plugins](logstash-docs-md://lsr/index.md) to collect, process, and forward data from a variety of sources to a variety of destinations. Here we focus on using the [Elasticsearch input](logstash-docs-md://lsr/plugins-inputs-elasticsearch.md) plugin to read from your {{ech}} deployment, and the [Elasticsearch output](logstash-docs-md://lsr/plugins-outputs-elasticsearch.md) plugin to write to your {{{serverless-full}} project. |
There was a problem hiding this comment.
could you update the admonition on line 19 to make it clearer that it's about deployment types? the title "basic migration" isn't very meaningful
| Kibana assets much be migrated separately using the {{kib}} [export/import APIs](https://www.elastic.co/docs/api/doc/kibana/group/endpoint-saved-objects) or recreated manually. Refer to [Migration options](/manage-data/migrate-data.md#migration-options) for details about migrating different types of {{es}} data. | ||
| Templates, data stream definitions, and ILM policies, must be in place _before_ you start data migration. | ||
|
|
||
| Visual components, such dashboard and visualizations, can be migrated after you have migrated the data. |
There was a problem hiding this comment.
again, lower on this page, we only have 3 config options called out. I wonder how specific to ech/serverless this tutorial actually is, and how it could be generalized. feels like it wouldn't need much at all.
might be worth considering as our only link we are using on the migration overview page.
|
|
||
| ## User data migration guides [data-migration-guides] | ||
|
|
||
| To migrate your {{es}} ingested user data, choose one of the available migration options depending on your source and target deployment types. |
There was a problem hiding this comment.
consider hinting that some of these tutorials use specific deployment types as examples, but can be altered for your scenario (less shock/disorientation)
This is a redo of #4914 to reorganize the "Migrate your Elasticsearch data" section of the docs. It:
reindex from remoteto migrate data from ECH to Serverless).Note: one source file was renamed so a few of the files here are included just for link fixes.
Closes: #4353
Rel: https://github.com/elastic/docs-team/issues/30
Rel: https://github.com/elastic/docs-content-internal/issues/467
Generative AI disclosure