Skip to content

added a simple program to export files in .vdb format#148

Open
spyke7 wants to merge 40 commits intoMDAnalysis:masterfrom
spyke7:add_openvdb
Open

added a simple program to export files in .vdb format#148
spyke7 wants to merge 40 commits intoMDAnalysis:masterfrom
spyke7:add_openvdb

Conversation

@spyke7
Copy link
Copy Markdown
Contributor

@spyke7 spyke7 commented Dec 27, 2025

Hi @orbeckst
I have added OpenVDB.py inside gridData that simply export files in .vdb format. Also I have added test_vdb.py inside tests and it successfully passes.
fix #141

Required Libraries -
openvdb

  • conda install -c conda-forge openvdb

There are many things that need to be updated like docs, etc, but I have just provided the file and test so that you can review it, and I can fix the problems. Please let me know if anything needs to be changed and updated.

@codecov
Copy link
Copy Markdown

codecov Bot commented Dec 27, 2025

Codecov Report

❌ Patch coverage is 93.75000% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.00%. Comparing base (947226b) to head (3472d4f).

Files with missing lines Patch % Lines
gridData/OpenVDB.py 92.07% 6 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #148      +/-   ##
==========================================
+ Coverage   89.69%   90.00%   +0.30%     
==========================================
  Files           5        6       +1     
  Lines         844      950     +106     
  Branches      108      125      +17     
==========================================
+ Hits          757      855      +98     
- Misses         52       58       +6     
- Partials       35       37       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@spyke7
Copy link
Copy Markdown
Contributor Author

spyke7 commented Dec 27, 2025

@orbeckst , please review the OpenVDB.py file. After that, I will add some more test covering all the missing parts

@orbeckst
Copy link
Copy Markdown
Member

orbeckst commented Dec 27, 2025 via email

Copy link
Copy Markdown
Member

@orbeckst orbeckst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution. Before going further, can you please try your own code and demonstrate that it works? For instance, take some of the bundled test files such as 1jzv.ccp4 or nAChR_M2_water.plt, write it to OpenVDB, load it in blender, and show an image of the rendered density?

Once we know that it's working in principle, we'll need proper tests (you can look at PR #147 for good example of minimal testing for writing functionality).

Comment thread CHANGELOG Outdated
Comment thread CHANGELOG Outdated
Comment on lines +24 to +26
Fixes

* Adding openVDB formats (Issue #141)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a fix but an Enhancement – put it into the existing 1.1.0 section and add you name there.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the CHANGELOG, this PR and issue are in the 1.1.0 release, so should I add my name in the 1.1.0 release or remove those lines and put them in the new section?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, now move it to the new section above since we released 1.1.0.

Comment thread gridData/OpenVDB.py
Comment thread gridData/OpenVDB.py
Comment thread gridData/OpenVDB.py Outdated
Comment on lines +183 to +188
for i in range(self.grid.shape[0]):
for j in range(self.grid.shape[1]):
for k in range(self.grid.shape[2]):
value = float(self.grid[i, j, k])
if abs(value) > threshold:
accessor.setValueOn((i, j, k), value)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really slow — iterating over a grid explicitly. For a start, you can find all cells above a threshold with numpy operations (np.abs(g) > threshold) and then ideally use it in a vectorized form to set the accessor.

Comment thread gridData/tests/test_vdb.py
@orbeckst orbeckst self-assigned this Jan 9, 2026
@spyke7 spyke7 requested a review from orbeckst January 18, 2026 06:48
@spyke7
Copy link
Copy Markdown
Contributor Author

spyke7 commented Jan 18, 2026

fixed the CHANGELOG and OpenVDB.py. I didn't get the time to work on the blender part due to exams. I will surely try do it!

@spyke7
Copy link
Copy Markdown
Contributor Author

spyke7 commented Jan 18, 2026

Screenshot (125) Screenshot (126) Screenshot (128) Screenshot (129)

The first two are for naChR_M2_water.vdb and the last two are for 1jzv.vdb.
Also in the OpenVDB.py, the function should be transform.preTranslate which I will fix with the new tests.
Can you please confirm that these are the correct rendering?
I can provide the .vdb files as well here.

@orbeckst
Copy link
Copy Markdown
Member

Good that you're able to load something into Blender. From a first glance I don;t recognize what I'd expect but this may be dependent on how you render in Blender. As I already said on Discord: Try to establish yourself what "correct" means. Load the original data in a program where you can reliably look at it. ChimeraX is probably the best for looking at densities; it can definitely read DX.

Btw, the M2 density should look similar to the blue "blobs" on the cover of https://sbcb.bioch.ox.ac.uk/users/oliver/download/Thesis/OB_thesis_2sided.pdf

@spyke7
Copy link
Copy Markdown
Contributor Author

spyke7 commented Jan 19, 2026

Screenshot (131) Screenshot (132)

The first one is for 1jzv.vdb and second for nAChR_M2_water.vdb (not done the shading/coloring)
I think the .vdb files as generated by the OpenVDB.py are now correctly rendering in blender.

Can I proceed with the tests part?

@BradyAJohnston
Copy link
Copy Markdown
Member

Mentioned in the Discord but also bringing up here: In your current examples (most obvious with the pore) is that the axis is flipped so that X is "up" compared to atomic coordinates which would have Z as up.

@spyke7
Copy link
Copy Markdown
Contributor Author

spyke7 commented Jan 19, 2026

Mentioned in the Discord but also bringing up here: In your current examples (most obvious with the pore) is that the axis is flipped so that X is "up" compared to atomic coordinates which would have Z as up.

Thank you for the update! will try to fix this

@spyke7
Copy link
Copy Markdown
Contributor Author

spyke7 commented Jan 19, 2026

Screenshot (133) Screenshot (136)

I think this fixes the axis..

@BradyAJohnston
Copy link
Copy Markdown
Member

Ideally we would see this alongside the atoms or density from MN as well - to double check alignment because you might also need to flip one of the X or Y axes.

@BradyAJohnston
Copy link
Copy Markdown
Member

The scales might be different (larger or smaller by factors of 10) but you can just scale inside of Blender by that amount to align the scales, but we want to be double checking alignemnt and axes.

@spyke7
Copy link
Copy Markdown
Contributor Author

spyke7 commented Jan 20, 2026

Hi @BradyAJohnston
Screenshot (138)
Screenshot (139)

I have first of all added the MolecularNode add-on as given in the https://github.com/BradyAJohnston/MolecularNodes, and imported the 1jzv.pdb. After that import the .vdb file and there was difference in size of two. So I made the size the .pdb bigger. The centers of both of them are same and I didn't flipped any of the axes in the ss provided.

I wrote a small blender py script to compare bounding boxes of the pdb and vdb objects to verify centroids, extents and axis alignment-

import bpy
from mathutils import Vector

def bbox_world(obj):
    bbox = [obj.matrix_world @ Vector(c) for c in obj.bound_box]
    mn = Vector((min(p[i] for p in bbox) for i in range(3)))
    mx = Vector((max(p[i] for p in bbox) for i in range(3)))
    return mn, mx

def centroid_world(obj):
    mn, mx = bbox_world(obj)
    return (mn + mx) / 2.0

def size_world(obj):
    mn, mx = bbox_world(obj)
    return mx - mn

pdb = bpy.data.objects.get("1jzv.001")
vdb = bpy.data.objects.get("1jzv")

print("pdb centroid:", centroid_world(pdb))
print("pdb size:", size_world(pdb))
print("vdb centroid:", centroid_world(vdb))
print("vdb size:", size_world(vdb))

output -
pdb centroid: <Vector (7.6985, 23.7885, 76.0560)>
pdb size: <Vector (33.1410, 45.4170, 29.3960)>
vdb centroid: <Vector (8.7238, 23.4452, 76.7628)>
vdb size: <Vector (43.6190, 52.3429, 40.3425)>

The centroids are almost same I guess...
the data seems to be correctly aligned

@BradyAJohnston BradyAJohnston self-assigned this Jan 20, 2026
@BradyAJohnston
Copy link
Copy Markdown
Member

@spyke7 It's still not 100% clear from your screenshots - can you import with the pore instead as that is more clear? And when you are taking a screenshot it would be more helpful to have the imported density in the centre of the screen rather than mostly empty space.

@PardhavMaradani
Copy link
Copy Markdown

Looks like you are attempting a standalone export to .vdb files from GridDataFormats. (If your end use case is to use this only within Blender, I'd strongly recommend using MolecularNodes to import various grid formats as it already uses GridDataFormats internally and provides a lot of cool features like varying ISO values, different colors for positive and negative ISO values, slicing along all three major axes, showing contours, centering, inverting etc - both from GUI and API) From a quick scan of the code, you seem to want to support both pyopenvdb (the older one) and openvdb (the newer one) - note that there are some minor differences to take into account between them. You can take a look at the grid_to_vdb method from an earlier version in MN that shows the differences and handles the export to .vdb within MolecularNodes. Hope this helps. Thanks

@BradyAJohnston
Copy link
Copy Markdown
Member

If this functionality can be added directly to GDF then we can also take advantage of that in MN going forwards.

@PardhavMaradani
Copy link
Copy Markdown

If this functionality can be added directly to GDF then we can also take advantage of that in MN going forwards.

Agreed. In addition to exporting to .vdb format, we also add some additional metadata (currently, info about inversion, centered) that we later use. So as long as the metadata for Grids is carried over during export, we should probably be good. Thanks

@BradyAJohnston
Copy link
Copy Markdown
Member

In addition to exporting to .vdb format, we also add some additional metadata

This is a good point and something to consider as well. As far as I am aware Blender / MN (and other 3D animation packages) might be the only ones who use .vdb as a format rather than any scientific packages / pipelines.

If there is anything out there that does take .vdb then we might want to consider if any relevant metadata should be saved. We might want to standardise on relevant metadata entries (we could either re-use from MN or update inside of MN to more general ones) so that GDF interactions with .vdb attempt to approach some kind of standard. This might be a larger question outside of scope for a simple read / write, but certainly functionality to pass in custom metadata like we do in MN would be ideal.

@spyke7 spyke7 mentioned this pull request Mar 6, 2026
@spyke7
Copy link
Copy Markdown
Contributor Author

spyke7 commented Mar 6, 2026

Removed the last few commits, so as to remove the new edits/updates for core.py
And added a new PR, containing all the things of this PR with the removed commits. As, without this , I cannot implement those convert_to() from grid to native and native to grid in core.py.

@orbeckst orbeckst mentioned this pull request Apr 24, 2026
4 tasks
@spyke7
Copy link
Copy Markdown
Contributor Author

spyke7 commented Apr 24, 2026

@orbeckst I'm adding the new updates in this PR only, that way there would not be need of #163 (and the tests will pass!)

@orbeckst
Copy link
Copy Markdown
Member

Ok, then close PR #163 please!

Copy link
Copy Markdown
Member

@orbeckst orbeckst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good. I have a few comments that need addressing but for most of them you can probably just "accept suggestion".

There's some code that's not yet covered by tests. Please look at the coverage reports. For instance https://app.codecov.io/gh/MDAnalysis/GridDataFormats/pull/148?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=checks&utm_campaign=pr+comments&utm_term=MDAnalysis shows that OpenVDBFiled.from_grid() and .native are not tested but these are important for the API so we need explicit tests.

Also run black on the whole file. Hopefully it re-orders the imports so that standard lib imports come before external packages.

The core code looks all good so should be an easy merge.

Comment thread CHANGELOG Outdated
Comment thread gridData/OpenVDB.py Outdated
Comment thread gridData/OpenVDB.py Outdated
Comment thread gridData/OpenVDB.py Outdated
Comment thread gridData/core.py Outdated
Comment thread gridData/core.py Outdated
Comment thread gridData/tests/test_vdb.py
spyke7 and others added 5 commits May 2, 2026 18:00
Co-authored-by: Oliver Beckstein <orbeckst@gmail.com>
Co-authored-by: Oliver Beckstein <orbeckst@gmail.com>
Co-authored-by: Oliver Beckstein <orbeckst@gmail.com>
@orbeckst
Copy link
Copy Markdown
Member

orbeckst commented May 2, 2026

The latest change improved the coverage but it will still be important to test the new attributes and functions explicitly.

@spyke7
Copy link
Copy Markdown
Contributor Author

spyke7 commented May 7, 2026

Hi @orbeckst, sorry for late reply.
I have tested it by running some codes on 1jzv and nAChR_M2_water
And everything looks fine. But I was thinking about passing the native wrapper (i.e openvdbgrid) into Grid class

This piece of test code works fine -

from gridData import Grid
g = Grid(str(filename))
g.export(str(outputfile), tolerance=1e-2)

But if I try to take out the wrapper and pass it to the same Grid it is showing error (obviously)

g = Grid(str(filename))
grid1 = g.convert_to("vdb")    #  -> native wrapper
grid2 = Grid(grid=grid1)          # -> Error

as edges, origin and delta are None.

Quick workaround is using the openvdb itself

# Above code without grid2
import openvdb
openvdb.write(str(output_file), grids = [grid1])

So I was thinking if inside from_grid, we can extract origin, delta, and edges from provided openvdb.FloatGrid
I have tried to get out the things but was looking how to reuse it. What are your thoughts on it?

@orbeckst
Copy link
Copy Markdown
Member

orbeckst commented May 7, 2026

grid1 = g.convert_to("vdb")    #  -> native wrapper
grid2 = Grid(grid=grid1)          # -> Error

grid1 is the native OpenVDB FloatGrid.

Then the second line is really what #162 is about — that could wait until we address that issue.

@orbeckst
Copy link
Copy Markdown
Member

orbeckst commented May 7, 2026

At this stage I'd be happy to merge this PR if you address my comments and include explicit tests for the from_grid() and .native functionality.

The changes that I made test these implicitly so coverage is now sufficient but we should have explicit tests for key API functionality.

@PardhavMaradani I know that you're busy. Let us know if you would like to have some extra time for reviewing or if you're ok with proceeding as necessary.

@spyke7
Copy link
Copy Markdown
Contributor Author

spyke7 commented May 7, 2026

The changes that I made test these implicitly so coverage is now sufficient but we should have explicit tests for key API functionality.

Yeah, I was looking into other comments, and will create the tests as required.
Once it is implemented, we can focus on solving #162

@PardhavMaradani
Copy link
Copy Markdown

@PardhavMaradani I know that you're busy. Let us know if you would like to have some extra time for reviewing or if you're ok with proceeding as necessary.

This looks good to me. I am ok with proceeding as needed. A few minor points:

  • For the metadata test (test_write_vdb_with_metadata), given name is already an existing attribute, it might be better to add some non-standard ones and verify (preferably using vdb.readAllGridMetadata)
  • I believe the test test_write_vdb_origin_and_spacing is to check for the correctness of the transform.preScale and transform.postTransform transforms. Only the spacing (scale) is being verified. Maybe use something like transform.indexToWorld to verify the world space coords and check the translation too?
  • The native vdb object access doesn't seem to be tested. With the changes in this PR, most of the MN code would get simplified and there would be only a couple of calls to get the native object (using convert_to) and create a new linear transform with the scale and translation that MN needs and use the GDF export method to write it out. So, it might be a good test to add too if possible

Thanks

@spyke7 spyke7 requested a review from orbeckst May 8, 2026 14:53
@spyke7
Copy link
Copy Markdown
Contributor Author

spyke7 commented May 8, 2026

@orbeckst pls check the test functions when you are free. I have added new tests for from_grid and native and covered the points given by @PardhavMaradani .

Copy link
Copy Markdown
Member

@orbeckst orbeckst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only have minor final requests, please see comments.

Thanks for adding the tests for from_grid() and native — I think this also addresses directly one of @PardhavMaradani's points.

Comment thread CHANGELOG Outdated
Comment thread CHANGELOG

Enhancements

* `Grid` now accepts binary operations with any operand that can be
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This duplicated line needs to be removed

Suggested change
* `Grid` now accepts binary operations with any operand that can be

(not sure where it came from, may have been a poor merge)


assert tmpdir.join("zero_tolerance.vdb").exists()

def test_vdb_non_orthrhombic_raises(self):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spelling

Suggested change
def test_vdb_non_orthrhombic_raises(self):
def test_vdb_non_orthorhombic_raises(self):


g.export(outfile, tolerance=0)

assert tmpdir.join("zero_tolerance.vdb").exists()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test some actual values that would be affected by tolerance=0, not just file exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add OpenVDB format

4 participants