Add headless renderer

[Shatur]

What problem does this solve or what need does it fill?

When creating multiplayer, often the logic of the game is executed on the headless server. But registration of renderer types such as Asset<Meshes> is now tied to the renderer. Therefore, to use exactly the same logic on the server, I have to add compile-time checks:

    pub fn spawn(
        commands: &mut Commands,
        transform: Transform,
        #[cfg(not(feature = "headless"))] meshes: &mut Assets<Mesh>,
        #[cfg(not(feature = "headless"))] materials: &mut Assets<StandardMaterial>,
    ) -> Self {
    // ...
    }

This looks ugly and very inconvenient to use.

What solution would you like?

It would be great to have a headless renderer.

[cart]

The new renderer does have a clear separation between "render code" and "app code", but the types are still exported from the new bevy_render crate (Mesh) and the new bevy_pbr crate (StandardMaterial). I think it makes sense logically to keep it that way.

I do agree that we should support headless mode, but I think it would probably look more like this:

• Add a "null / headless" render backend to wgpu (to ensure that people writing render code don't need to think too hard about adding "headless" support to their apps)
• Add the ability to disable render logic entirely (this is straightforward as this is already fully decoupled)

[Shatur]

Sounds like a great idea! The same approach is used in Godot.

I understand that the 0.6 release is already huge, but it there any chance to see headless render mode in 0.6?

[cart]

I think wgpu already supports "null" mode in that it still compiles / runs if you dont have any supported devices + backends available. So the biggest missing piece is just ensuring the right things are disabled / nothing panics in headless mode. This seems like a pretty small chunk of work. I can't promise that it will happen in time, but we might be able to make it work!

[alice-i-cecile]

Related to #2896 and #1057.

[cart]

A relevant message I just left on discord in response to @Shatur:

I would like to start working on #3155 Is it a good time to do it?

Yeah now seems like as good a time as any! I think we should start by using "runtime configuration", such as alternative plugins (or maybe configuration of RenderPlugin). I don't have a strong opinion on the UX yet, so feel free to experiment and present your thoughts.

How can I enable null mode in wgpu?

I know that wgpu will fail to initialize a device when there are no backends available:

bevy/pipelined/bevy_render2/src/renderer/mod.rs

Line 70 in bab4ee9

let adapter = instance

So step one will either be:

A) ensuring we don't panic when there is no device available (and headless mode is enabled)
B) just not trying to create the device at all when headless mode is enabled
C) add a "null backend" to wgpu

Using (A) would probably require compiling wgpu with backends disabled (which may or may not be possible right now). (B) would avoid the failure in the first place and feels cleaner to me than (A). Both (A) and (B) would require gracefully handling the fact that the RenderDevice and RenderQueue resources dont exist at all. This would mainly mean that we don't run the render app schedule when headless mode is enabled.

However we do currently support the ability for normal app code to access the RenderDevice as a resource (although we don't currently use it for anything). If we go with (A) or (B), that type of code would panic when we don't insert the RenderDevice. Theres also a future where asset loaders directly produce gpu assets in a separate thread instead of going through the Render App schedule. The only way to support these scenarios properly is with a "null wgpu backend" that does nothing, but still results in successful creation of a RenderDevice. I don't think wgpu has an actual "null backend" yet, but I'm hoping they're open to it.

Imo we should start with (B) as it is easy and should generally work.
I think its also important to distinguish between "headless without any renderering or windows at all" and "headless without windows but with rendering"
as people have expressed interest in using bevy to render to textures and write them to files (or send them somewhere else via RPC)
I know you're mainly interested in the former scenario, but we should figure out the best way to present those two modes

[Shatur]

Here is the summary of messages that I wrote in the discord.

I decided to try researching the suggested step B. I managed to get the 3d_scene_pipelined example to work without the device by commenting out just a couple of lines:

• RenderApp creation with wgpu device
• All accesses to RenderApp (13 references).

After that I had a window with no render. Then I commented out window creation - it worked.
Now we need to decide how it will look.

Renderer

Maybe make it a compile option because wgpu is not needed on servers? E.g. just allow bevy_render2 compilation without wgpu. Here is some options on how this can be implemented:

1. Fake RenderApp. In this case, the other plugin does not have to worry about the renderer not existing and it's great. But is it possible to implement it? What if other plugins add systems to the RenderApp that require a RenderDevice?
2. Return Option from sub_app and force plugin authors to handle RenderApp absence:
   

   bevy/crates/bevy_app/src/app.rs

   Lines 857 to 862 in 2def902

   	pub fn sub_app(&mut self, label: impl AppLabel) -> &mut App {
   	match self.get_sub_app(label) {
   	Ok(app) => app,
   	Err(label) => panic!("Sub-App with label '{:?}' does not exist", label),
   	}
   	}

3. Force plugin users to support headless mode explicitly (via compile option). This is the least obvious way, but it may allow plugin authors to exclude some dependencies for headless mode.

Window

With a window, things are more complicated. If we just disable the bevy_winit, the game loses its game loop and exists immediately. So, here is some options on how this can be implemented:

1. Decouple game loop from bevy_winit. Is it possible? If yes, I think it would be the best solution as it allows to simply remove bevy_winit from the server.
2. Compile time option that disables window creation and winit dependency for bevy_winit.
3. We can add a runtime option not to create a window.

@cart Which of the following options would you like more? Do you have any alternative implementation suggestions?

[cart]

Glad you got the initial "headless mode" implemented!

Renderer

I do think that we need to separate the tasks of "logical headless mode that doesn't set up the renderer when enabled" and "completely removing wgpu from the tree". The first two solutions you suggest for the renderer don't solve the "remove wgpu from the tree" problem because users still code to wgpu symbols in those scenarios. The problem here is that wgpu is a part of our public API. If we want to disable the dependency entirely at compile time, we need user code to either:

1. Never touch wgpu directly (and we would instead build an abstraction layer on top of wgpu that users interact with)
2. Accommodate the wgpu feature at compile time. I can think of two options here. Either by convention wgpu render logic and other logic are completely separate crates, with wgpu crates being optional, or engine developers and users would agree to only use wgpu logic behind a "wgpu" cargo feature.

Neither (1) or (2) are actually options in my opinion. In (1) we'd just be re-inventing the wgpu api. We've already agreed that wgpu is our user-facing gpu api and I'm not reverting that decision. (2) involves foisting too much responsibility on Bevy users to manually draw hard compile-time lines between render logic and app logic. Unity / Godot / Unreal / etc don't force users to do this. We can't either. So what can we do? Did we design ourselves into a hole?

Nope! We can actually accomplish (1), we just need to shift our perspective a bit. The goal of removing wgpu from the tree is to remove cruft that isn't needed by servers. With the constraints I have established above, I am basically asserting that headless apps need to be compiled with a gpu api if they are coding to something that uses bevy_render (things like Meshes, Images, Sprites, etc). That is not cruft, its a part of the public Bevy api. Note that bevy_render is already an optional dependency. If you aren't writing code that spawns "rendered things", you can disable it. If you are spawning "rendered things" wgpu is our public gpu api and that needs to be usable whenever someone pulls in bevy_render as a dependency. What we don't need are the gpu backends like Vulkan, Metal, OpenGL, etc. We only need the actual users facing api. Internally wgpu can either fail when called or use a "null" backend (we might want to support both). We just need a way to tell wgpu to compile without these backends. Fortunately wgpu-hal backends are already expressed as cargo features, so making this configurable shouldn't be too hard.

To solve the other "logical headless mode that doesn't set up the renderer when enabled" problem, we can either:

A. Keep user / engine code as it is and implement a "null backend" in wgpu that doesn't fail when used like a gpu api. This would likely still incur some overhead as we'd still be running the renderer, it just wouldn't do anything. We probably don't want that in most headless scenarios.
B. Check a run-time "headless mode" flag that ensures we dont run the renderer or try to init the render app.

I think we should scope this issue to solving (B). Your idea to return Option when users call sub_app is reasonable (we actually already have a get_sub_app api that does this, so the solution would be to remove the panic-ing api to force users to contend with the Option). But it will still fail in headless scenarios the second users try to unwrap that Option. Maybe thats acceptable (and it does seem like a reasonable short term solution), but we should also discuss ways to make this line stricter / harder for users to mess up. Maybe we should have separate "RenderPlugins" that we can just avoid initializing at all. Combine that with deferred plugin init (ex: bevyengine/rfcs#33) and we could choose to just not initialize render app plugins when headless mode is enabled. This will require design work and I can't just give you an easy answer about what that should look like.

Window

We can't really decouple the game loop from bevy_winit because winit needs to own the loop. We could shim that out, but we'd effectively be re-inventing the winit api. I'm heavily considering ripping out the existing "bevy window layer" in favor of treating winit as our public window api (much like the bevy_render2 + wgpu changes). In that case we would want something like a "null window backend" if the goal is to remove "backend cruft".

Fortunately we've already abstracted out "app runners". You can already just disable the optional bevy_winit dependency and define a custom app runner: https://github.com/bevyengine/bevy/blob/main/examples/app/custom_loop.rs. Custom runners can just choose to ignore window creation events, so this should satisfy "headless" needs already.

[Shatur]

Thank you a lot for your input! The suggested design is so elegant, I like it. Will start working on it.