Project 5: Matt Schwartz

[mzschwartz5]

Repo link

Live demo

Features:

• Point cloud rendering
• Gaussian splatting

Feedback:

• Code needs LOOKATs to help students orient themselves.
  • There should be one around the GaussianRenderer interface explaining what it's for
  • One around this comment in renderer.ts: "//Bind constants to the gaussian renderer."
  • One or more around in load.ts to help students understand the sh coefficients / explain how they are stored.
• Instructions or recitation should include explanation of indirect drawing (don't assume we know what that is).
• In my opinion, the recitation needs more detail about the math involved in computing the 3D covariance / 2D projection, etc. Right now we're just copying the math from random repos. At the very least, let us copy from recitation slides so we know it's trustworthy and will function correctly.
• The linked paper wasn't really worth the read; it was largely about the construction of the gaussians and not the rendering of them. It confused me at first more than it helped me (reading it later, it made more sense).
• Big one: testing/iterating is very slow and painful. Every time you make a code change, you have to upload a 300MB file and it takes a long time.
  • Similarly, hot reloading is somewhat pointless when each time you have to reupload the .ply and camera files.
  • Also, this project isn't a great candidate for webgpu, because it's not really shareable; the viewer has to have the files, which are in an access-restricted drive. And what viewer is going to go to the trouble of uploading the files anyway?
• Since the far plane of the camera is used in depth sorting calculations, it should be an explicit variable / part of the camera uniforms, instead of secretly hardcoded to 100.
• There's something weird about the camera controls - panning left/right does some y-direction motion as well.
• In general, I felt that the instructions were not very clear. I think I spent more time trying to decipher the instructions and various papers and youtube videos, than I did implementing the project. The project instructions would benefit from proof reading and restructuring and more explicit detail.
  • An example from the instructions: "In the preprocess shader, transform the Gaussian points from world space to NDC space and stored data in Splat (similar to point cloud renderer)." What do you mean similar to point cloud renderer? The point cloud renderer doesn't use the Splat struct. This is just one example of something confusing in the instructions, but there are several others.
• In the same vein, the TODOs in the code could be clearer.
• The github pages don't work by default. There are several totally miscellaneous fixes that are required for the base code to work with gh pages. These should be fixed in the base code.

[Cryszzz]

The deployment thru github pages should be already fixed (in last two commits from main repo) last week. And you should be able to get far plane through projection matrix. (https://stackoverflow.com/questions/56428880/how-to-extract-camera-parameters-from-projection-matrix)