Video-Bench.github.io/index.html at main · Video-Bench/Video-Bench.github.io · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>Video-Bench: Human-Aligned Video Generation Benchmark</title>
    <style>
        body {
            font-family: Arial, sans-serif;
            margin: 0;
            padding: 0;
            background-color: #f4f4f4;
        }
        header {
            background-color: #333;
            color: #fff;
            text-align: center;
            padding: 20px;
        }
        header h1 {
            font-size: 2.5em;
            margin: 0;
        }
        header h2 {
            font-size: 1.5em;
            margin: 5px 0;
        }
        .affiliations {
            font-size: 1em;
            color: #ddd;
            margin-top: 10px;
        }
        .author-info {
            font-size: 1.2em;
            color: #f0f0f0;
        }
        .author-info sup {
            font-size: 0.8em;
            color: #88aaff;
        }
        section {
            margin: 20px;
            padding: 20px;
            background-color: white;
            border-radius: 8px;
            box-shadow: 0px 0px 10px rgba(0, 0, 0, 0.1);
        }
        h2 {
            color: #333;
        }
        p {
            font-size: 1.1em;
            color: #555;
        }
        ul {
            list-style: none;
            padding: 0;
        }
        ul li {
            padding: 8px;
        }
        a {
            color: #007bff;
            text-decoration: none;
        }
        a:hover {
            text-decoration: underline;
        }
        .table-container {
            margin: 20px 0;
        }
        table {
            width: 100%;
            border-collapse: collapse;
            margin-bottom: 20px;
        }
        table, th, td {
            border: 1px solid #ddd;
        }
        th, td {
            padding: 10px;
            text-align: center;
        }
        th {
            background-color: #f4f4f4;
        }
        .image-container {
            text-align: center;
            margin: 20px 0;
        }
        .image-container img {
            max-width: 100%;
            height: auto;
            border-radius: 8px;
        }
        .video-container {
            text-align: center;
            margin-top: 20px;
        }
        .video-container iframe {
            width: 100%;
            max-width: 800px;
            height: 450px;
            border-radius: 8px;
        }
    </style>
</head>
<body>

<header>
    <h1>Video-Bench: Human-Aligned Video Generation Benchmark</h1>
    <div class="author-info">
        <p>by <strong>Hui Han<sup>1</sup>, Siyuan Li<sup>1</sup>, Jiaqi Chen<sup>2,3,4</sup>, Yiwen Yuan<sup>5</sup>, Yuling Wu<sup>6</sup>, Chak Tou Leong<sup>5</sup>, Hanwen Du<sup>7</sup>, Junchen Fu<sup>8</sup>, Youhua Li<sup>9</sup>, Jie Zhang<sup>4</sup>, Chi Zhang<sup>10</sup>, Li-jia Li<sup>11</sup>, Yongxin Ni<sup>12</sup></strong></p>
    </div>
    <div class="affiliations">
        <p>1. Shanghai Jiao Tong University, 2. Stanford University, 3. Fellou AI, 4. Fudan University, 5. Carnegie Mellon University, 6. Hong Kong Polytechnic University, 7. Soochow University, 8. University of Glasgow, 9. City University of Hong Kong, 10. Westlake University, 11. LiveX AI, 12. National University of Singapore</p>
    </div>
</header>

<section>
    <div class="image-container">
        <img src="dimension-old.png" alt="Head Image" /> <!-- Ensure this is the correct path to your image -->
    </div>

    <h2>Project Overview</h2>
    <p>Video generation assessment is critical for ensuring generative models produce visually realistic, high-quality videos aligned with human expectations. Current video generation benchmarks are limited in aligning with human judgment. To address this, <strong>Video-Bench</strong> is introduced—a comprehensive benchmark incorporating large language models (LLMs) to evaluate video generation quality. The framework includes automated multimodal LLM evaluation, improving the alignment with human preferences. Experimental results show that Video-Bench significantly outperforms previous methods and provides more objective and accurate insights into generated video quality.</p>

    <h2>Main Results</h2>

    <h3>Video-Bench Leaderboard</h3>
    Higher scores indicate better performance. The best score in each dimension is highlighted in bold. ``Avg Rank" is the average rank of multiple dimensions, the lower the better.
    <div class="table-container">
        <table>
            <thead>
                <tr>
                    <th>Model</th>
                    <th>Video Quality</th>
                    <th>Video-Condition Alignment</th>
                    <th>Overall Rank</th>
                </tr>
            </thead>
            <tbody>
                <tr>
                    <td>Gen3</td>
                    <td>4.66</td>
                    <td>4.38</td>
                    <td>1</td>
                </tr>
                <tr>
                    <td>CogVideoX</td>
                    <td>3.84</td>
                    <td>4.62</td>
                    <td>2</td>
                </tr>
                <tr>
                    <td>VideoCrafter2</td>
                    <td>4.08</td>
                    <td>4.18</td>
                    <td>3</td>
                </tr>
                <tr>
                    <td>Kling</td>
                    <td>4.26</td>
                    <td>4.07</td>
                    <td>4</td>
                </tr>
                <tr>
                    <td>Show-1</td>
                    <td>3.30</td>
                    <td>4.21</td>
                    <td>5</td>
                </tr>
                <tr>
                    <td>LaVie</td>
                    <td>3.00</td>
                    <td>3.71</td>
                    <td>6</td>
                </tr>
                <tr>
                    <td>PiKa-Beta</td>
                    <td>3.76</td>
                    <td>2.60</td>
                    <td>7</td>
                </tr>
            </tbody>
        </table>
    </div>

    <h3>Human Preference Alignment Scores</h3>
    This score is measured by Spearman's rank correlation coefficient. Higher score indicates better performance. The best score in each dimension is highlighted in bold. In practice, ComBench$^{*}$~\cite{sun2024t2v} is a reproduction version on our benchmark metrics.
    <div class="table-container">
        <table>
            <thead>
                <tr>
                    <th>Entities</th>
                    <th>Video Quality</th>
                    <th>Video-Condition Alignment</th>
                    <th>Average Score</th>
                </tr>
            </thead>
            <tbody>
                <tr>
                    <td>HU - HU</td>
                    <td>0.63</td>
                    <td>0.47</td>
                    <td>0.52</td>
                </tr>
                <tr>
                    <td>HU - GPT</td>
                    <td>0.51</td>
                    <td>0.47</td>
                    <td>0.41</td>
                </tr>
                <tr>
                    <td>HU - HA</td>
                    <td>0.61</td>
                    <td>0.50</td>
                    <td>0.50</td>
                </tr>
            </tbody>
        </table>
    </div>

    <h2>Project Video Demonstration</h2>
    <div class="video-container">
        <iframe src="https://www.youtube.com/embed/BMvgyWbWPFg" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
    </div>

    <h2>GitHub Repository</h2>
    <p>For more details, visit the official repository: <a href="https://github.com/Video-Bench/Video-Bench.git" target="_blank">Video-Bench GitHub Repository</a></p>

</section>

</body>
</html>