Skip to content

Conversation

@alxvth
Copy link
Contributor

@alxvth alxvth commented Jan 20, 2025

This PR uses the SparseMatrix implementation from #556 but moves it into an Experimental class to clearly discourage usage since the sparse data API will change.

You may use it right now like this:

auto points = static_cast<Points*>(numericalDataset.getDataset());
size_t numRows = 50;
size_t numCols  = 50;
std::vector<size_t> rowPointers = {};
std::vector<size_t> colIndices = {};
std::vector<float> values = {};

Points::Experimental::setSparseData(points, numRows, numCols, rowPointers, colIndices, values);

events().notifyDatasetDataChanged(numericalDataset);

Additions on top of #556:

  • Template row pointer type
  • use size_t instead of uint16_t for index type
  • use float instead of uint16_t for data type
  • Updated setter API in Point data

One usage example is in the HDF5 loader where we currently lock sparse data since no other plugins support it yet.

@alxvth alxvth requested a review from ThomasKroes January 20, 2025 14:13
Copy link
Contributor

@ThomasKroes ThomasKroes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This solution seems to work for me on Windows 11. Tried 10 projects with data and they all load without issues.

@alxvth alxvth merged commit 552109b into master Jan 21, 2025
@alxvth alxvth deleted the feature/sparse_data_experimental branch January 21, 2025 14:54
ThomasKroes pushed a commit that referenced this pull request Jan 21, 2025
* Add initial sparse matrix implementation

* Fix issue with reading in sparse data from project files

* Add fromValue

* Add getters and setters

* Move Sparse Data into Experimental Nested Points class

* Move templated code to header file

* Use uint32 instead of uint16, float instead of uint16
Add move data setter
Fix some conversion warnings

* Some serialization steps are common

* Fix pointer error

* Consistent API

* Return empty vec for dense getRow

* Set dense data empty when setting sparse data

* Template row index in sparse data and use size_t instead of uint for col index

* Fix template compilation issue

* Fix comment [skip ci]

---------

Co-authored-by: Julian Thijssen <julianthijssen@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants