Skip to content
This repository was archived by the owner on Oct 15, 2025. It is now read-only.

Conversation

@mfs4rd
Copy link

@mfs4rd mfs4rd commented Nov 4, 2024

Purpose

This updates the quilt paths used to download data used in the paper to match what is on the publicly available dataset

Key changes

  • Downloads of full datasets have been updated. This includes downloading multiple directories at once when the original data has been split into multiple numbered folders within the package's root directory
  • Downloading of small test datasets has been removed as there is no subset of test data provided within the public dataset
  • Additional updates to analysis code and data retrieval

@mfs4rd mfs4rd requested a review from vianamp November 4, 2024 21:21
@vianamp
Copy link
Collaborator

vianamp commented Nov 4, 2024

@mfs4rd can you add some description of what this PR is doing? For example why removing the test dataset downloading?

@mfs4rd
Copy link
Author

mfs4rd commented Nov 5, 2024

@vianamp Updated PR description. Test data downloading has been removed since there are no subsets of the data provided in the public dataset as in the private dataset. We would need to implement a method of properly sampling the dataset and trimming the relevant manifests if we want to keep this feature.

Copy link
Collaborator

@vianamp vianamp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, Filip. Looks great!

--image_csv ./quilt_data_contrasted_test/metadata.csv \
--defaults_json fish_morphology_code/cellprofiler/cellprofiler_image_set_defaults.json \
--path_key rescaled_2D_fov_tiff_path \
--local_path ./quilt_data_contrasted_test \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we remove test from the path since there is no longer a test set?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants