How did you added your own asset to the Carla ?

[ronyshaji]

I have a doubt regarding the addition of your own asset. Did you edited the carla source or added your own asset to it so that it appears after build ?

[LeeYiyuan]

We created a dynamic mesh asset that was burned into the compiled binaries. Then, the CARLA plugin was modified to allow us to spawn PNG images (map tiles) as instances of these dynamic mesh assets, by displaying the images on the surface of the tiles.

[ronyshaji]

SO you write the c++ code and compiled while build ?

[LeeYiyuan]

Yes -- we modified CARLA's plugin in UE to include the meshes. You may refer to the source code here.

[ronyshaji]

So if you modified the CARLA plugin from UE, why we still need to do this ? I mean copy from your CustomAssets folder to the carla after build ?

Copy <summit_root>/CustomAssets/M_Tile.uasset to <summit_root>/Unreal/CarlaUE4/Content/Carla/Static/GenericMaterials/Ground/M_Tile.uasset

[LeeYiyuan]

The file defines the dynamic mesh tile asset, which the plugin creates instances of. Thus, both need to be present.

[ronyshaji]

Well...is nativising the blueprint works ? I mean nativize my blueprint and build the simulator without copying .uasset file ?

[LeeYiyuan]

The compiled simulator can't load anything that it does not have access to when it was compiled. Hence, if you exclude a blueprint and compile, you will not be able to use it in the simulator.

[ronyshaji]

@LeeYiyuan So if i need to include a blueprint from unreal engine to the Summit(CARLA), you have to write it in c++ and compile while building, is that right ?

[LeeYiyuan]

You will have to, depending on how you intend to access your blueprint, do the following:

1. include the blueprint in the resources folder,
2. load the blueprint from the UE plugin for SUMMIT (DynamicMeshActor.cpp:148),
3. in the UE plugin, create a hook to spawn an actor from the blueprint (CarlaServer.cpp:291, CarlaServer:301), which will be invoked from the C++ LibCarla library,
4. modify C++ LibCarla library to include functions to call the UE plugin hook in step 3 (Client.h:99, 101). With this you can spawn the blueprint from any C++ program which uses C++ LibCarla;
5. modify the Python binding of the C++ LibCarla library to expose the newly added function in step 4 (World.cpp:151-175). With this you can spawn the blueprint from any Python program which imports libcarla.

Steps 1, 2, 3 require rebuilding the plugin and subsequently the compiled binaries. Step 4 requires rebuilding the C++ LibCarla library. Step 5 requires rebuilding the Python bindings.

[ronyshaji]

@LeeYiyuan For the step 2, you mean i have to replace UMaterial* Material = Cast<UMaterial>(StaticLoadObject( UMaterial::StaticClass(), NULL, TEXT("/Game/Carla/Static/GenericMaterials/Ground/M_Tile"))); with my own blueprint path ?

[LeeYiyuan]

That's the general idea, but the exact requirements depends on what kind of blueprint you are using. We are using a UMaterial and it will not work if your blueprint does not match. Please refer UE's official documentation and the community forum on the specific codes needed to get this to work.

[ronyshaji]

Thanks for the information. In my case the blueprint contains procedural mesh and i think it is different to summit case. Also the DynamicMeshActor.cpp is created entirely in unreal ? I didnot see any carla headers in that, is that the case ?

[LeeYiyuan]

The DynamicMeshActor is in fact a procedural mesh, so perhaps it might help you to get started on what you have in mind. Yes, it is created entirely in unreal, and is part of the UE plugin for SUMMIT/CARLA. The UE plugin is loaded when the compiled simulator binaries run. On the other hand, the carla headers are part of LibCarla, which is an entirely separate modue that resides outside of the UE plugin, but communicates with the UE plugin via RPC calls.

[ronyshaji]

I already have my blueprints (c++ and .h files) ready. So i guess when i follow your steps mentioned above, i can able to access the blueprint in carla as other build in carla contents, right ?

[LeeYiyuan]

Yup, in that case you will need steps 3, 4, 5 depending on where you want to access the blueprint from.