Unable to get the same resolution

[SamitM1]

test1.mp4

When i try to enhance the video above from open-sora the target resolution is always set to a max of 1152, 2048

2024-10-14 01:59:38,941 - video_to_video - INFO - input frames length: 102
2024-10-14 01:59:38,941 - video_to_video - INFO - input fps: 24.0
2024-10-14 01:59:38,941 - video_to_video - INFO - target_fps: 24.0
2024-10-14 01:59:39,703 - video_to_video - INFO - input resolution: (720, 1280)
2024-10-14 01:59:39,704 - video_to_video - INFO - target resolution: (1152, 2048)
2024-10-14 01:59:39,704 - video_to_video - INFO - noise augmentation: 250
2024-10-14 01:59:39,704 - video_to_video - INFO - scale s is set to: 8
2024-10-14 01:59:39,834 - video_to_video - INFO - video_data shape: torch.Size([102, 3, 1152, 2048])

I thought the max was 2k or 2560x1440 like the example you provided. If i try to manually set the target height and width I get a memory error with a 80gb H100

[hejingwenhejingwen]

there's resolution restrictions in

VEnhancer/enhance_a_video.py

Line 64 in 80ffaa3

target_h, target_w = adjust_resolution(h, w, up_scale)

you can comment it to obtain your target resolution.

[SamitM1]

Thanks @hejingwenhejingwen
So does that mean i can upscale 720x480 to 2560x1440 if i comment that?

[hejingwenhejingwen]

Thanks @hejingwenhejingwen So does that mean i can upscale 720x480 to 2560x1440 if i comment that?

it depends on your 'up_scale'.

[SamitM1]

What do i need to set the up_scale to in order to get my 720x480 video to 2560x1440

[hejingwenhejingwen]

What do i need to set the up_scale to in order to get my 720x480 video to 2560x1440

up_scale = 1440/480 or 2560/720
Note that your input and output resolutions have different aspect ratios, so you can set up_scale=3, and use black borders for augmenting the resolution to 2560x1440.

[SamitM1]

Will the black bars be present in the outputted video and we can just crop it out?

And if we were to set the upscale to 2560/720 without any black bars then would it stretch things or mess with the resolution of the outputted video

[hejingwenhejingwen]

I mean you obtain 2160x1440 by setting up_scale=3. After venhancer, you add black borders to make it 2560x1440.
Better not have black borders during inference, as it will increase the computational cost.

If you directly set the target resolution with different aspect ratio, the video will be stretched.