-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathsearch.xml
More file actions
174 lines (149 loc) · 53.1 KB
/
search.xml
File metadata and controls
174 lines (149 loc) · 53.1 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title>文章导读：Motivation for and Evaluation of the First Tensor Processing Unit</title>
<url>/2024/12/10/paper-TPU/</url>
<content><![CDATA[<h1 id="相关资料"><a href="#相关资料" class="headerlink" title="相关资料"></a>相关资料</h1><hr>
<p><strong><a href="https://ieeexplore.ieee.org/document/8358031">https://ieeexplore.ieee.org/document/8358031</a></strong></p>
<h1 id="TPU介绍"><a href="#TPU介绍" class="headerlink" title="TPU介绍"></a>TPU介绍</h1><hr>
<p>Tensor Processing Unit (TPU) 是Google提出的一种专门设计的硬件加速器，这种DSA(特定领域架构)芯片展现出惊人的运算性能与能效比，自2015年以来在Google的数据中心广泛使用，服务于全球数十亿用户。</p>
<p>TPU硬件如下图所示，主要包含如下模块</p>
<ul>
<li>256x256 的 INT8 矩阵乘法单元 (MACs)</li>
<li>片上FIFO队列：读取 DRAM 的权重值</li>
<li>统一缓存: 存储计算中间结果，可作为矩阵单元的输入</li>
<li>可编程 DMA 控制器：将数据从 CPU 端送入统一缓存</li>
</ul>
<p><img src="/2024/12/10/paper-TPU/1.png"></p>
<h1 id="TPU使用的技术"><a href="#TPU使用的技术" class="headerlink" title="TPU使用的技术"></a>TPU使用的技术</h1><hr>
<p>为了缓解计算开销与访存开销的严重失衡，软件层面我们会在高速缓存中充分复用数据，减少低速内存访问次数；硬件层面 Google 使用了一种名为 systolic execution（脉动阵列）的技术，同样可以减少访存次数。</p>
<p>传统的计算方式（左图）是数据每计算一次就要存储一次，而下一次要调取计算结果的时候也要从存储器里面重新获得这个数据，往复循环。那么在脉动结构中，单一 PE 被替换成了一串 PE。数据在经手所有 PE 计算之后才会被存储，由于矩阵加乘计算需要大量的数据复用，这种数据计算流程大量地减少了数据被访问的次数，从而实现了更高的效率。</p>
<p><img src="/2024/12/10/paper-TPU/2.png"></p>
<p>使用脉动阵列计算矩阵C=AxB时，需要将A、B的数据分多次输入阵列，如图所示。</p>
<p><img src="/2024/12/10/paper-TPU/3.jpg"></p>
<h1 id="TPU的优势"><a href="#TPU的优势" class="headerlink" title="TPU的优势"></a>TPU的优势</h1><hr>
<h3 id="优势："><a href="#优势：" class="headerlink" title="优势："></a>优势：</h3><ul>
<li>控制单元少 可以放更多缓存</li>
<li>峰值性能高 功耗低</li>
<li>设计简单（微结构特征简单）</li>
</ul>
]]></content>
<categories>
<category>论文导读</category>
</categories>
</entry>
<entry>
<title>看论文相关工具</title>
<url>/2024/12/13/read-paper-tools/</url>
<content><![CDATA[<h1 id="基于英文看论文"><a href="#基于英文看论文" class="headerlink" title="基于英文看论文"></a>基于英文看论文</h1><hr>
<p>啃英文论文虽然花时间，但也有好处：</p>
<ul>
<li><p>熟悉英文表达方式 </p>
</li>
<li><p>慢下来思考文章逻辑 </p>
</li>
<li><p>熟悉专有名词</p>
</li>
</ul>
<figure class="highlight tex"><table><tr><td class="code"><pre><span class="line">工具：ReadPaper</span><br><span class="line">位置：https://readpaper.com/new</span><br></pre></td></tr></table></figure>
<p>可以选择 软件 / 网页版，软件效果如下图所示：<br><img src="/2024/12/13/read-paper-tools/1.jpg"></p>
<h1 id="基于中文看论文"><a href="#基于中文看论文" class="headerlink" title="基于中文看论文"></a>基于中文看论文</h1><hr>
<ul>
<li><p>好处：减少语言壁垒 专注于内容</p>
</li>
<li><p>坏处：减少提高英语能力机会</p>
</li>
</ul>
<figure class="highlight tex"><table><tr><td class="code"><pre><span class="line">工具：yiyibook</span><br><span class="line">位置：https://yiyibooks.cn/</span><br></pre></td></tr></table></figure>
<p>该工具效果如下图所示：<br><img src="/2024/12/13/read-paper-tools/2.jpg"></p>
<h1 id="根据论文看论文-（快速了解领域信息）"><a href="#根据论文看论文-（快速了解领域信息）" class="headerlink" title="根据论文看论文 （快速了解领域信息）"></a>根据论文看论文 （快速了解领域信息）</h1><hr>
<figure class="highlight tex"><table><tr><td class="code"><pre><span class="line">工具：connected paper</span><br><span class="line">位置：https://www.connectedpapers.com/</span><br><span class="line">Edge浏览器可利用无痕模式反复使用 </span><br></pre></td></tr></table></figure>
<p><img src="/2024/12/13/read-paper-tools/3.jpg"></p>
<h1 id="根据作者看论文（了解作者长期研究脉络）"><a href="#根据作者看论文（了解作者长期研究脉络）" class="headerlink" title="根据作者看论文（了解作者长期研究脉络）"></a>根据作者看论文（了解作者长期研究脉络）</h1><hr>
<figure class="highlight tex"><table><tr><td class="code"><pre><span class="line">工具：dblp</span><br><span class="line">位置：https://dblp.org/</span><br></pre></td></tr></table></figure>
<p>这是我老板的相关paper</p>
<p><img src="/2024/12/13/read-paper-tools/4.jpg"></p>
<h1 id="根据论文看代码"><a href="#根据论文看代码" class="headerlink" title="根据论文看代码"></a>根据论文看代码</h1><hr>
<figure class="highlight tex"><table><tr><td class="code"><pre><span class="line">工具：paper with code</span><br><span class="line">位置：https://paperswithcode.com/ </span><br></pre></td></tr></table></figure>
<p><img src="/2024/12/13/read-paper-tools/5.jpg"></p>
]]></content>
<categories>
<category>工具</category>
</categories>
</entry>
<entry>
<title>先导杯比赛回顾</title>
<url>/2024/12/05/review/</url>
<content><![CDATA[<h1 id="竞赛结果"><a href="#竞赛结果" class="headerlink" title="竞赛结果"></a>竞赛结果</h1><hr>
<p>本次先导杯竞赛我带领DeepOptimized团队获得<strong>国赛三等奖</strong>, 复赛rank为 <strong>5/16</strong>。</p>
<p>赛程期间DeepOptimized团队共完成 <strong>20+</strong> 次代码迭代，在中科曙光DCU K100-AI平台上高度优化了卷积运算，实现了较直接卷积<strong>30-40</strong>倍的性能提升</p>
<p><img src="/2024/12/05/review/award.jpg" alt="award"></p>
<p><img src="/2024/12/05/review/image-20241205160808477.png" alt="image-20241205160808477"></p>
<h1 id="任务介绍"><a href="#任务介绍" class="headerlink" title="任务介绍"></a>任务介绍</h1><hr>
<p>本届先导杯分为 <strong>“多模态大模型基础卷积算子优化”</strong> 和 “大语言模型llama3 8B推理性能的软硬件协同优化” 赛道。</p>
<p>“多模态大模型基础卷积算子优化”赛道中共有6组测试样例, 6组测试样例特征各不相同，需要针对测试样例编写算子实现高性能运算。</p>
<p><code>测试样例1： K = 27 无法被2整除 测试样例4: C*R*S >> M N 测试样例6： K = 4 维度过小</code></p>
<p>DeepOptimized团队针对大赛提出的6个测试样例的访存特征，总结赛题难点如下：</p>
<ul>
<li><p>线程束调度空间受限</p>
</li>
<li><p>NCHW布局空间连续性不佳</p>
</li>
<li><p>矩阵维度高度不均衡</p>
</li>
</ul>
<h3 id="线程束调度空间受限"><a href="#线程束调度空间受限" class="headerlink" title="线程束调度空间受限"></a>线程束调度空间受限</h3><p><strong><code>K100-AI的访存时延隐藏能力受限于活跃线程束的数量。常规方法根据(M, N)维度划分线程束，但在特定参数配置下，该划分方法无法满足K100-AI的调度需求。</code></strong></p>
<h3 id="NCHW布局空间连续性不佳"><a href="#NCHW布局空间连续性不佳" class="headerlink" title="NCHW布局空间连续性不佳"></a>NCHW布局空间连续性不佳</h3><p><strong><code>在NCHW格式中，同一通道的所有像素数据在内存中是连续存储的，而在卷积操作中，通常需要访问多个通道的数据。GPU在加载数据到缓存时可能需要频繁地进行内存访问，导致缓存利用率降低，进而影响整体计算性能。</code></strong></p>
<h3 id="矩阵维度高度不均衡"><a href="#矩阵维度高度不均衡" class="headerlink" title="矩阵维度高度不均衡"></a>矩阵维度高度不均衡</h3><p><strong><code>L2缓存命中率高度依赖于wave（同时运行的线程块）对输入矩阵的重复读取。当输出矩阵的维度高度不均衡时，基于轮转的线程块调度策略会导致wave覆盖的区域呈现出极度扁平的形态，从而减少了对矩阵B的重复读取机会。导致L2缓存命中率降低。</code></strong></p>
<p><img src="/2024/12/05/review/image-20241205160602875.png" alt="image-20241205160602875"></p>
<p><img src="/2024/12/05/review/image-20241205161110512.png" alt="image-20241205161110512"></p>
<h1 id="卷积转化"><a href="#卷积转化" class="headerlink" title="卷积转化"></a>卷积转化</h1><hr>
<p>直接卷积运算小规模内积的深层嵌套循环无法充分利用 K100-AI丰富的计算资源，这些嵌套的卷积循环通常会被展开并转化为 <strong>GEMM(通用矩阵乘)</strong> 的形式来提高计算效率。</p>
<p><img src="/2024/12/05/review/image-20241205162024854.png" alt="image-20241205162024854"></p>
<p>由于K100-AI的Tensor core具有强大的浮点处理能力，在这类加速设备上进行 GEMM 运算时，更容易表现出内存访问的瓶颈。</p>
<p><img src="/2024/12/05/review/image-20241205162055238.png" alt="image-20241205162055238"></p>
<h1 id="优化策略"><a href="#优化策略" class="headerlink" title="优化策略"></a>优化策略</h1><hr>
<ul>
<li><p>NCHW_2_NHWC 数据排布转换的高性能实现</p>
</li>
<li><p>多批次展开的 Im2col 方法</p>
</li>
<li><p>Swizzle 线程块映射</p>
</li>
<li><p>基于 atnomicAdd 的 Split K 策略</p>
</li>
<li><p>Epilogue 写回优化</p>
</li>
<li><p>Padding 解决 bankconflict</p>
</li>
</ul>
<h3 id="NCHW-2-NHWC-数据排布转换的高性能实现"><a href="#NCHW-2-NHWC-数据排布转换的高性能实现" class="headerlink" title="NCHW_2_NHWC 数据排布转换的高性能实现"></a>NCHW_2_NHWC 数据排布转换的高性能实现</h3><p>NCHW到NHWC的数据排布转换等价于N次(C, HW)到(HW, C)的transpose操作。</p>
<p>数据排布转换能否高性能实现取决于transpose算子的性能表现。</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">extern</span> <span class="string">"C"</span> <span class="function">__global__ <span class="type">void</span> <span class="title">transpose_64x64_kernel</span><span class="params">(</span></span></span><br><span class="line"><span class="params"><span class="function"> _Float16* A, </span></span></span><br><span class="line"><span class="params"><span class="function"> _Float16* At,</span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> M, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> N)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="type">uint32_t</span> col_idx = blockIdx.x * blockDim.x + threadIdx.x;</span><br><span class="line"> <span class="type">uint32_t</span> row_idx = blockIdx.y * blockDim.y + threadIdx.y;</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">if</span> (col_idx >= (N >> <span class="number">2</span>) <span class="keyword">or</span> row_idx >= (M >> <span class="number">2</span>)) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="type">uint32_t</span> offset = (row_idx * N + col_idx) << <span class="number">2</span>; <span class="comment">// * 4</span></span><br><span class="line"> _Float16_4 src_row_0, src_row_1, src_row_2, src_row_3;</span><br><span class="line"></span><br><span class="line"> _Float16_4* input_4 = <span class="built_in">reinterpret_cast</span><_Float16_4*>(A + offset);</span><br><span class="line"></span><br><span class="line"> _Float16_4 src_row0 = input_4[<span class="number">0</span>]; <span class="comment">// 第一行</span></span><br><span class="line"> _Float16_4 src_row1 = input_4[N >> <span class="number">2</span>]; <span class="comment">// 第二行</span></span><br><span class="line"> _Float16_4 src_row2 = input_4[N >> <span class="number">1</span>]; <span class="comment">// 第三行</span></span><br><span class="line"> _Float16_4 src_row3 = input_4[(N >> <span class="number">2</span>) * <span class="number">3</span>]; <span class="comment">// 第四行</span></span><br><span class="line"></span><br><span class="line"> _Float16_4 dst_row0 = {src_row0[<span class="number">0</span>], src_row1[<span class="number">0</span>], src_row2[<span class="number">0</span>], src_row3[<span class="number">0</span>]};</span><br><span class="line"> _Float16_4 dst_row1 = {src_row0[<span class="number">1</span>], src_row1[<span class="number">1</span>], src_row2[<span class="number">1</span>], src_row3[<span class="number">1</span>]};</span><br><span class="line"> _Float16_4 dst_row2 = {src_row0[<span class="number">2</span>], src_row1[<span class="number">2</span>], src_row2[<span class="number">2</span>], src_row3[<span class="number">2</span>]};</span><br><span class="line"> _Float16_4 dst_row3 = {src_row0[<span class="number">3</span>], src_row1[<span class="number">3</span>], src_row2[<span class="number">3</span>], src_row3[<span class="number">3</span>]};</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">// 将4行写入</span></span><br><span class="line"> offset = (col_idx * M + row_idx) << <span class="number">2</span>;</span><br><span class="line"> _Float16_4* dst_4 = <span class="built_in">reinterpret_cast</span><_Float16_4*>(At + offset);</span><br><span class="line"></span><br><span class="line"> dst_4[<span class="number">0</span>] = dst_row0;</span><br><span class="line"> dst_4[M >> <span class="number">2</span>] = dst_row1; <span class="comment">// 第二行</span></span><br><span class="line"> dst_4[M >> <span class="number">1</span>] = dst_row2; <span class="comment">// 第三行</span></span><br><span class="line"> dst_4[(M >> <span class="number">2</span>) * <span class="number">3</span>] = dst_row3; <span class="comment">// 第四行</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="comment">// call function</span></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">launch_transpose_64x64_kernel</span><span class="params">(</span></span></span><br><span class="line"><span class="params"><span class="function"> _Float16* A, </span></span></span><br><span class="line"><span class="params"><span class="function"> _Float16* At, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> M, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> K)</span> </span>{</span><br><span class="line"> <span class="function">dim3 <span class="title">block</span><span class="params">(<span class="number">16</span>, <span class="number">16</span>)</span></span>;</span><br><span class="line"> <span class="function">dim3 <span class="title">grid</span><span class="params">((K >> <span class="number">2</span>) / <span class="number">16</span>, (M >> <span class="number">2</span>) / <span class="number">16</span>)</span></span>;</span><br><span class="line"> transpose_64x64_kernel<<<grid, block>>> (A, At, M, K);</span><br><span class="line"></span><br><span class="line"> hipError_t error = <span class="built_in">hipGetLastError</span>();</span><br><span class="line"> <span class="keyword">if</span> (error != hipSuccess)</span><br><span class="line"> {</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"HIP kernel launch failed with error: %s\n"</span>, </span><br><span class="line"> <span class="built_in">hipGetErrorString</span>(error));</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">launch_nchw_to_nhwc_kernel</span><span class="params">(</span></span></span><br><span class="line"><span class="params"><span class="function"> _Float16* input, </span></span></span><br><span class="line"><span class="params"><span class="function"> _Float16* output, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> N, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> C, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> H, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> W)</span> </span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="type">uint32_t</span> span = C * H * W;</span><br><span class="line"> <span class="keyword">for</span>(<span class="type">int</span> i = <span class="number">0</span>; i < N; i++) {</span><br><span class="line"> <span class="built_in">launch_transpose_64x64_kernel</span>(</span><br><span class="line"> input + i * span,</span><br><span class="line"> output + i * span,</span><br><span class="line"> C,</span><br><span class="line"> H * W);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<h3 id="多批次展开的-Im2col-方法"><a href="#多批次展开的-Im2col-方法" class="headerlink" title="多批次展开的 Im2col 方法"></a>多批次展开的 Im2col 方法</h3><p>基于简单的图优化原理，Im2col方法可以展开多个批次的输入数据，扩大线程束调度空间，利用调度机制隐藏访存延迟。</p>
<p>具体来说Gemm运算每次需要处理(k, CRS) x (CRS, expand_col * oh * ow)的数据，在最后通过reshape算子调整输出矩阵排布方式。</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="type">void</span> <span class="title">mma_32x32x16_run</span><span class="params">(mykernelParamType * param)</span> </span>{</span><br><span class="line"> <span class="built_in">UNROLL_PARAM</span>(param);</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> M = k;</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> N = n * outh * outw;</span><br><span class="line"> <span class="type">unsigned</span> <span class="type">int</span> K = c * r * s;</span><br><span class="line"></span><br><span class="line"> <span class="built_in">launch_im2col_2D_kernel</span>(param->pin, n, c, h, w, r, s, h, w, param->expand_row, param->expand_col, param->data_col_device);</span><br><span class="line"> <span class="built_in">launch_transpose_64x64_kernel</span>(param->pweight, param->pweight_trans, M, K);</span><br><span class="line"> <span class="built_in">launch_gemm_32x32x16_fp16</span>(param, param->expand_row, param->expand_col);</span><br><span class="line"> <span class="built_in">launch_reshape_2D_32x32_kernel</span>(param->output_gemm_device, param->pout, n, k, outh, outw, param->expand_row, param- >expand_col);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>im2col的高性能实现如下图所示：</p>
<figure class="highlight c++"><table><tr><td class="code"><pre><span class="line"><span class="keyword">extern</span> <span class="string">"C"</span> __global__ __launch_bounds__(<span class="number">1024</span>) <span class="function"><span class="type">void</span> <span class="title">im2col_2D_kernel</span><span class="params">(</span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">const</span> _Float16* data_im, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> n, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> channels, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> height, <span class="type">int</span> width,</span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> kernel_h, <span class="type">int</span> kernel_w, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> pad_h, <span class="type">int</span> pad_w, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> stride_h, <span class="type">int</span> stride_w, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> span_expand_row, <span class="type">int</span> span_unfold_crs, <span class="type">int</span> span_expand_col, <span class="type">int</span> expand_row, <span class="type">int</span> expand_col,</span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> output_h, <span class="type">int</span> output_w, </span></span></span><br><span class="line"><span class="params"><span class="function"> _Float16* data_col)</span> </span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="type">int</span> index = (blockIdx.x * blockDim.x + threadIdx.x) * <span class="number">2</span>;</span><br><span class="line"> <span class="type">int</span> crs = blockIdx.y * <span class="number">3</span>; </span><br><span class="line"> <span class="type">int</span> c = crs / (kernel_h * kernel_w); </span><br><span class="line"> <span class="type">int</span> kh = (crs % (kernel_h * kernel_w)) / kernel_w; </span><br><span class="line"></span><br><span class="line"> _Float16 smem[<span class="number">10</span>];</span><br><span class="line"> <span class="built_in">memset</span>(smem, <span class="number">0</span> ,<span class="built_in">sizeof</span>(smem));</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (index < span_unfold_crs) {</span><br><span class="line"> <span class="type">int</span> c_b = index / (output_h * output_w); </span><br><span class="line"> <span class="type">int</span> r_b = blockIdx.z;</span><br><span class="line"> <span class="type">int</span> b = r_b * expand_col + c_b;</span><br><span class="line"> <span class="type">int</span> oh = (index % (output_h * output_w)) / output_w; </span><br><span class="line"> <span class="type">int</span> ow = (index % (output_h * output_w)) % output_w; </span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> im_row = kh - pad_h + oh * stride_h;</span><br><span class="line"> <span class="type">int</span> im_col = ow * stride_w - pad_w;</span><br><span class="line"></span><br><span class="line"> <span class="type">int</span> offset_col = r_b * span_expand_row + crs * span_unfold_crs + c_b * span_expand_col + oh * output_w + ow; </span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (im_row >= <span class="number">0</span> && im_row < height){</span><br><span class="line"> <span class="keyword">if</span>(im_col >= <span class="number">0</span>)</span><br><span class="line"> smem[<span class="number">0</span>] = data_im[(b * channels + c) * height * width + im_row * width + im_col];</span><br><span class="line"> *(<span class="type">int</span> *)(&smem[<span class="number">1</span>]) = *(<span class="type">int</span> *)(&data_im[(b * channels + c) * height * width + im_row * width + im_col + <span class="number">1</span>]);</span><br><span class="line"> <span class="keyword">if</span>(im_col + <span class="number">3</span> < width)</span><br><span class="line"> smem[<span class="number">3</span>] = data_im[(b * channels + c) * height * width + im_row * width + im_col + <span class="number">3</span>];</span><br><span class="line"> }</span><br><span class="line"> *(<span class="type">int</span> *)(&data_col[offset_col]) = *(<span class="type">int</span> *)(&smem[<span class="number">0</span>]);</span><br><span class="line"> *(<span class="type">int</span> *)(&data_col[offset_col + span_unfold_crs]) = *(<span class="type">int</span> *)(&smem[<span class="number">1</span>]);</span><br><span class="line"> *(<span class="type">int</span> *)(&data_col[offset_col + span_unfold_crs + span_unfold_crs]) = *(<span class="type">int</span> *)(&smem[<span class="number">2</span>]);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">void</span> <span class="title">launch_im2col_2D_kernel</span><span class="params">(</span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">const</span> _Float16* data_im_device, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> n, <span class="type">int</span> channels, <span class="type">int</span> height, <span class="type">int</span> width, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> kernel_h, <span class="type">int</span> kernel_w, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> output_h, <span class="type">int</span> output_w,</span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">int</span> expand_row, <span class="type">int</span> expand_col,</span></span></span><br><span class="line"><span class="params"><span class="function"> _Float16* data_col_device)</span> </span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="type">int</span> span_expand_col = output_h * output_w;</span><br><span class="line"> <span class="type">int</span> span_unfold_crs = expand_col * span_expand_col;</span><br><span class="line"> <span class="type">int</span> span_expand_row = channels * kernel_h * kernel_w * span_unfold_crs;</span><br><span class="line"> </span><br><span class="line"> <span class="function">dim3 <span class="title">blockSize</span><span class="params">(<span class="number">512</span>)</span></span>; </span><br><span class="line"> <span class="function">dim3 <span class="title">gridSize</span><span class="params">((expand_col * output_h * output_w + blockSize.x - <span class="number">1</span>) / <span class="number">1024</span>, channels * kernel_h * kernel_w / <span class="number">3</span>, n / expand_col)</span></span>;</span><br><span class="line"></span><br><span class="line"> im2col_2D_kernel<<<gridSize, blockSize>>>(data_im_device,</span><br><span class="line"> n, channels, height, width, kernel_h, kernel_w, <span class="number">1</span>,<span class="number">1</span>,<span class="number">1</span>,<span class="number">1</span>,</span><br><span class="line"> span_expand_row, span_unfold_crs, span_expand_col, expand_row, expand_col,</span><br><span class="line"> output_h, output_w,</span><br><span class="line"> data_col_device);</span><br><span class="line"> hipError_t error = <span class="built_in">hipGetLastError</span>();</span><br><span class="line"> <span class="keyword">if</span> (error != hipSuccess) {</span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"HIP kernel launch failed with error: %s\n"</span>, <span class="built_in">hipGetErrorString</span>(error));</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<h3 id="Swizzle-线程块映射"><a href="#Swizzle-线程块映射" class="headerlink" title="Swizzle 线程块映射"></a>Swizzle 线程块映射</h3><p>L2命中率取决于同时运行的线程块的覆盖区域，通过Swizzle方法映射线程块可以缓解因维度不均衡导致的数据复用率低的问题。</p>
<p>相较于cutlass的方形映射方式，我们的映射方式通过设置SWIZZLE_X和SWIZZLE_y可以更加灵活地映射线程块。</p>
<figure class="highlight cpp"><table><tr><td class="code"><pre><span class="line"><span class="keyword">extern</span> <span class="string">"C"</span> __global__ __launch_bounds__(<span class="number">1024</span>) <span class="function"><span class="type">void</span> </span></span><br><span class="line"><span class="function"> <span class="title">Gemm_128x128x16_double_buffering_f16</span><span class="params">(</span></span></span><br><span class="line"><span class="params"><span class="function"> _Float16 *__restrict__ A, </span></span></span><br><span class="line"><span class="params"><span class="function"> _Float16 *__restrict__ B, </span></span></span><br><span class="line"><span class="params"><span class="function"> _Float16 *C,</span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">const</span> <span class="type">int</span> M, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">const</span> <span class="type">int</span> N, </span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="type">const</span> <span class="type">int</span> K)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"></span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> warp_id = threadIdx.x / warpSize;</span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> lane_id = threadIdx.x % warpSize;</span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> K_tiles = K / K_tile;</span><br><span class="line"> </span><br><span class="line"><span class="comment">//swizzle begin</span></span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> BLOCK_TILE_NUM = <span class="built_in">div_ceil</span>(N, N_tile);</span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> SWIZZLE_X = <span class="number">2</span>;</span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> SWIZZLE_Y = <span class="number">2</span>;</span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> swizzle_block_idx = blockIdx.x / (SWIZZLE_X * SWIZZLE_Y);</span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> swizzle_idx = blockIdx.x % (SWIZZLE_X * SWIZZLE_Y);</span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> blockIdx_x = (swizzle_block_idx % </span><br><span class="line"> (<span class="built_in">div_ceil</span>(BLOCK_TILE_NUM, SWIZZLE_X)) * SWIZZLE_X + (swizzle_idx % SWIZZLE_X));</span><br><span class="line"> <span class="type">const</span> <span class="type">uint32_t</span> blockIdx_y = (swizzle_block_idx / </span><br><span class="line"> (<span class="built_in">div_ceil</span>(BLOCK_TILE_NUM, SWIZZLE_X)) * SWIZZLE_Y + (swizzle_idx / SWIZZLE_X));</span><br><span class="line"> </span><br><span class="line">}</span><br><span class="line"><span class="comment">/* call function */</span></span><br><span class="line">Gemm_128x128x16_64x32_db_fp16</span><br><span class="line"> <<<<span class="built_in">dim3</span>(<span class="built_in">two_ceil</span>(<span class="built_in">div_ceil</span>(N, N_tile)) * <span class="built_in">two_ceil</span>(<span class="built_in">div_ceil</span>(M, M_tile)), SPLIT_K), block>>></span><br><span class="line"> (A,</span><br><span class="line"> (B + i * span_B),</span><br><span class="line"> (C + i * span_C),</span><br><span class="line"> M, N, K,</span><br><span class="line"> SWIZZLE_X, SWIZZLE_Y);</span><br></pre></td></tr></table></figure>
<h3 id="基于-atnomicAdd-的-Split-K-策略"><a href="#基于-atnomicAdd-的-Split-K-策略" class="headerlink" title="基于 atnomicAdd 的 Split K 策略"></a>基于 atnomicAdd 的 Split K 策略</h3><p>传统Split K策略将计算结果写入连续内存空间并规约累加，用以避免因数据争用造成的结果错误。</p>
<p>使用atnomicAdd原子加可以避免重复访问显存，提高整体性能。</p>
<figure class="highlight cpp"><table><tr><td class="code"><pre><span class="line"><span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">0</span>) + output_col]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">00.</span>x));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">4</span>) + output_col]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">00.</span>y));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">8</span>) + output_col]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">00.</span>z));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">12</span>) + output_col]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">00.</span>w));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">0</span>) + output_col + <span class="number">16</span>]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">10.</span>x));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">4</span>) + output_col + <span class="number">16</span>]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">10.</span>y));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">8</span>) + output_col + <span class="number">16</span>]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">10.</span>z));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">12</span>) + output_col + <span class="number">16</span>]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">10.</span>w));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">16</span>) + output_col]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">01.</span>x));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">20</span>) + output_col]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">01.</span>y));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">24</span>) + output_col]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">01.</span>z));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">28</span>) + output_col]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">01.</span>w));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">16</span>) + output_col + <span class="number">16</span>]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">11.</span>x));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">20</span>) + output_col + <span class="number">16</span>]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">11.</span>y));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">24</span>) + output_col + <span class="number">16</span>]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">11.</span>z));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">28</span>) + output_col + <span class="number">16</span>]), <span class="built_in">static_cast</span><__half>(fragC0_<span class="number">11.</span>w));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">0</span>) + output_col + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">00.</span>x));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">4</span>) + output_col + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">00.</span>y));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">8</span>) + output_col + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">00.</span>z));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">12</span>) + output_col + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">00.</span>w));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">0</span>) + output_col + <span class="number">16</span> + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">10.</span>x));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">4</span>) + output_col + <span class="number">16</span> + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">10.</span>y));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">8</span>) + output_col + <span class="number">16</span> + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">10.</span>z));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">12</span>) + output_col + <span class="number">16</span> + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">10.</span>w));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">16</span>) + output_col + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">01.</span>x));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">20</span>) + output_col + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">01.</span>y));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">24</span>) + output_col + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">01.</span>z));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">28</span>) + output_col + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">01.</span>w));</span><br><span class="line"></span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">16</span>) + output_col + <span class="number">16</span> + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">11.</span>x));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">20</span>) + output_col + <span class="number">16</span> + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">11.</span>y));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">24</span>) + output_col + <span class="number">16</span> + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">11.</span>z));</span><br><span class="line"> <span class="built_in">atomicAdd</span>(<span class="built_in">reinterpret_cast</span><__half*>(&C[M * (output_row + <span class="number">28</span>) + output_col + <span class="number">16</span> + <span class="number">32</span>]), <span class="built_in">static_cast</span><__half>(fragC1_<span class="number">11.</span>w));</span><br><span class="line"></span><br></pre></td></tr></table></figure>
<h3 id="Epilogue-写回优化"><a href="#Epilogue-写回优化" class="headerlink" title="Epilogue 写回优化"></a>Epilogue 写回优化</h3><p>mma指令计算结果对应的显存地址并不连续，无法充分利用K100-AI的带宽资源。</p>
<p>Epilogue通过共享内存重新组织输出形式，有利于合并访存。</p>
<figure class="highlight cpp"><table><tr><td class="code"><pre><span class="line"><span class="comment">// Epilogue</span></span><br><span class="line"> <span class="type">uint32_t</span> ohow = oh * ow;</span><br><span class="line"> <span class="type">uint32_t</span> store_smem_row = (warp_id >> <span class="number">2</span>) * <span class="number">32</span> + (lane_id & <span class="number">15</span>);</span><br><span class="line"> <span class="type">uint32_t</span> store_smem_col = (warp_id % <span class="number">4</span>) * <span class="number">32</span> + (lane_id >> <span class="number">4</span>);</span><br><span class="line"> <span class="type">uint32_t</span> load_smem_row = warp_id * <span class="number">8</span> + lane_id / <span class="number">8</span>;</span><br><span class="line"> <span class="type">uint32_t</span> load_smem_col = (lane_id & <span class="number">7</span>) * <span class="number">16</span>;</span><br><span class="line"> <span class="type">uint32_t</span> output_row = by * M_TILE + load_smem_row;</span><br><span class="line"> <span class="type">uint32_t</span> output_col = bx * N_TILE + load_smem_col;</span><br><span class="line"></span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row, store_smem_col, N_TILE)] = fragC<span class="number">00.</span>x;</span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row, store_smem_col + <span class="number">4</span>, N_TILE)] = fragC<span class="number">00.</span>y;</span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row, store_smem_col + <span class="number">8</span>, N_TILE)] = fragC<span class="number">00.</span>z;</span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row, store_smem_col + <span class="number">12</span>, N_TILE)] = fragC<span class="number">00.</span>w;</span><br><span class="line"></span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row, store_smem_col + <span class="number">16</span>, N_TILE)] = fragC<span class="number">01.</span>x;</span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row, store_smem_col + <span class="number">20</span>, N_TILE)] = fragC<span class="number">01.</span>y;</span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row, store_smem_col + <span class="number">24</span>, N_TILE)] = fragC<span class="number">01.</span>z;</span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row, store_smem_col + <span class="number">28</span>, N_TILE)] = fragC<span class="number">01.</span>w;</span><br><span class="line"></span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row + <span class="number">16</span>, store_smem_col, N_TILE)] = fragC<span class="number">10.</span>x;</span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row + <span class="number">16</span>, store_smem_col + <span class="number">4</span>, N_TILE)] = fragC<span class="number">10.</span>y;</span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row + <span class="number">16</span>, store_smem_col + <span class="number">8</span>, N_TILE)] = fragC<span class="number">10.</span>z;</span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row + <span class="number">16</span>, store_smem_col + <span class="number">12</span>, N_TILE)] = fragC<span class="number">10.</span>w;</span><br><span class="line"></span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row + <span class="number">16</span>, store_smem_col + <span class="number">16</span>, N_TILE)] = fragC<span class="number">11.</span>x;</span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row + <span class="number">16</span>, store_smem_col + <span class="number">20</span>, N_TILE)] = fragC<span class="number">11.</span>y;</span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row + <span class="number">16</span>, store_smem_col + <span class="number">24</span>, N_TILE)] = fragC<span class="number">11.</span>z;</span><br><span class="line"> smem[<span class="built_in">OFFSET</span>(store_smem_row + <span class="number">16</span>, store_smem_col + <span class="number">28</span>, N_TILE)] = fragC<span class="number">11.</span>w;</span><br><span class="line"></span><br><span class="line"> __syncthreads();</span><br><span class="line"></span><br><span class="line"> *(int4*)(&param.pout[output_n + ohow * output_row + output_col + <span class="number">0</span>]) = *(int4*)(&smem[N_TILE * load_smem_row + load_smem_col]);</span><br><span class="line"> *(int4*)(&param.pout[output_n + ohow * output_row + output_col + <span class="number">8</span>]) = *(int4*)(&smem[N_TILE * load_smem_row + load_smem_col + <span class="number">8</span>]);</span><br><span class="line"> __syncthreads();</span><br></pre></td></tr></table></figure>
<h3 id="Padding-解决-bankconflict"><a href="#Padding-解决-bankconflict" class="headerlink" title="Padding 解决 bankconflict"></a>Padding 解决 bankconflict</h3><p>ImplictGEMM每个warp内的线程需要把加载的数据加载到共享内存中，这些数据的写入会出现大量的bankconflict问题，造成性能下降。Padding通过改变数据在共享内存中物理布局，避免了bankconflict问题。</p>
<figure class="highlight cpp"><table><tr><td class="code"><pre><span class="line"><span class="comment">// padding define</span></span><br><span class="line"><span class="type">const</span> <span class="type">uint32_t</span> padding = <span class="number">32</span>;</span><br><span class="line">__shared__ _Float16 smem[((M_tile + padding) * <span class="number">4</span>) * K_tile];</span><br><span class="line"></span><br><span class="line"><span class="comment">// padding use: gmem -> smem</span></span><br><span class="line"><span class="type">uint32_t</span> smem_sel_offset = ((i - <span class="number">1</span>) & <span class="number">1</span>) * <span class="number">2</span> * (M_tile + padding) * K_tile;</span><br><span class="line"><span class="type">uint32_t</span> smem_next_offset = (i & <span class="number">1</span>) * <span class="number">2</span> * (M_tile + padding) * K_tile;</span><br><span class="line"> </span><br><span class="line"><span class="comment">// padding use: load data from smem</span></span><br><span class="line"><span class="type">uint32_t</span> lds_read_A0_offset = (smem_sel_offset + smem_offsetA + (lane_id / <span class="number">4</span>)</span><br><span class="line"> * (M_tile + padding) + (lane_id & <span class="number">3</span>) * <span class="number">8</span>) * <span class="built_in">sizeof</span>(_Float16);</span><br><span class="line"><span class="type">uint32_t</span> lds_read_B0_offset = (smem_sel_offset + smem_offsetB + (lane_id / <span class="number">4</span>) </span><br><span class="line"> * (N_tile + padding) + (lane_id & <span class="number">3</span>) * <span class="number">8</span> + (M_tile + padding) * K_tile) * <span class="built_in">sizeof</span>(_Float16);</span><br></pre></td></tr></table></figure>
<p><img src="/2024/12/05/review/image-20241205163103358.png" alt="image-20241205163103358"></p>
<p><img src="/2024/12/05/review/image-20241205163111787.png" alt="image-20241205163111787"></p>
<h1 id="总结与展望"><a href="#总结与展望" class="headerlink" title="总结与展望"></a>总结与展望</h1><p>本次先导杯竞赛其实是有不少遗憾的，DeepOptimized团队在最后一天确定了高性能实现布局转换的方法，排名从9上升到5。</p>
<p>实验结果表明在大部分测试样例下，隐式卷积性能均远超显式卷积，限于时间对隐式卷积的优化略显粗糙……</p>
]]></content>
<categories>
<category>项目</category>
</categories>
</entry>
</search>