How to train and inference on custom video dataset

[yejian97]

hello, could you describe how to train and inference on custom video dataset? Which part need to be modified.

[MaverickRen]

If you want to train an LDM with your own dataset, you need to modify 'LDM/ldm/datasets/calvin_dataset.py'. If you need to train your own AR Transformer, you need to modify 'VideoWorld/falcon/datasets/calvin_dataset.py'. You need to make the output of your data loading function consistent with that of CALVINDataset or GoDataset.