{"id":12999,"date":"2026-10-07T04:57:46","date_gmt":"2026-10-06T20:57:46","guid":{"rendered":"https:\/\/infernews.com\/blog\/prism-a-dynamic-sparse-attention-framework-for-natively-training\/"},"modified":"2026-10-07T05:09:16","modified_gmt":"2026-10-06T21:09:16","slug":"prism-a-dynamic-sparse-attention-framework-for-natively-training","status":"publish","type":"post","link":"https:\/\/infernews.com\/blog\/prism-a-dynamic-sparse-attention-framework-for-natively-training\/","title":{"rendered":"Prism \u4ee4 2K \u5f71\u97f3\u751f\u6210\u8a13\u7df4\u52a0\u901f 2.5 \u500d"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/infernews.com\/blog\/wp-content\/uploads\/2026\/10\/pasted-2936334b28cd.jpg\" alt=\"model architecture\"\/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u9ad8\u89e3\u50cf\u5ea6\u5f71\u97f3\u751f\u6210\u6700\u96e3\u8655\u7406\u7684\u5730\u65b9\uff0c\u662f\u756b\u9762 token \u5927\u589e\u5f8c\uff0cFull Attention \u9700\u8981\u4ed8\u51fa\u5e73\u65b9\u7d1a\u8a08\u7b97\u6210\u672c\uff0c\u8072\u97f3\u8207\u756b\u9762\u4e4b\u9593\u7684\u6709\u6548\u4e92\u52d5\u4ea6\u5bb9\u6613\u88ab\u5927\u91cf\u5197\u9918\u8cc7\u8a0a\u6c96\u6de1\u3002Prism \u5c6c\u65bc Dynamic Sparse Attention \u6846\u67b6\uff0c\u5c08\u9580\u8655\u7406\u539f\u751f 2K Joint Video-Audio Generation Model Training\uff0c\u76ee\u6a19\u662f\u6e1b\u5c11\u8a13\u7df4\u6210\u672c\u800c\u4e0d\u72a7\u7272\u5f71\u97f3\u540c\u6b65\u8207\u7d30\u7bc0\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Prism \u5c07\u5f71\u7247 token \u6309\u6642\u7a7a\u4f4d\u7f6e\u6574\u7406\u6210 macro-zones\uff0c\u518d\u5229\u7528\u5f71\u7247\u7279\u5fb5\u7684 channel variance\uff0c\u4ee5\u53ca audio-to-video cross-attention \u7684 feature norm\uff0c\u5224\u65b7\u6bcf\u500b\u5340\u57df\u7684\u756b\u9762\u8b8a\u5316\u548c\u8072\u97f3\u5f71\u97ff\u7a0b\u5ea6\u3002\u8b8a\u5316\u8f03\u5feb\u6216\u5f71\u97f3\u8026\u5408\u8f03\u5f37\u7684\u4f4d\u7f6e\u6703\u63a1\u7528\u66f4\u7d30\u7684 block \u5206\u5272\uff0c\u4e26\u900f\u904e hybrid block selection \u70ba\u4e0d\u540c query \u52d5\u614b\u6c7a\u5b9a\u7a00\u758f\u7bc4\u570d\uff0c\u4ee4\u6bcf\u500b block \u4fdd\u7559\u8f03\u4e00\u81f4\u7684\u8a9e\u7fa9\u5167\u5bb9\u3002<\/p>\n\n\n<figure class=\"wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"lyte-wrapper\" title=\"[Fudan&amp;amp;Tencent Hunyuan] Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Training\" style=\"width:853px;max-width:100%;margin:5px auto;\"><div class=\"lyMe\" id=\"WYL_XEL2IkSOSQw\" itemprop=\"video\" itemscope itemtype=\"https:\/\/schema.org\/VideoObject\"><div><meta itemprop=\"thumbnailUrl\" content=\"https:\/\/infernews.com\/blog\/wp-content\/plugins\/wp-youtube-lyte\/lyteCache.php?origThumbUrl=https%3A%2F%2Fi.ytimg.com%2Fvi%2FXEL2IkSOSQw%2Fhqdefault.jpg\" \/><meta itemprop=\"embedURL\" content=\"https:\/\/www.youtube.com\/embed\/XEL2IkSOSQw\" \/><meta itemprop=\"duration\" content=\"PT16M2S\" \/><meta itemprop=\"uploadDate\" content=\"2026-10-06T01:49:28Z\" \/><\/div><div id=\"lyte_XEL2IkSOSQw\" data-src=\"https:\/\/infernews.com\/blog\/wp-content\/plugins\/wp-youtube-lyte\/lyteCache.php?origThumbUrl=https%3A%2F%2Fi.ytimg.com%2Fvi%2FXEL2IkSOSQw%2Fhqdefault.jpg\" class=\"pL\"><div class=\"tC\"><div class=\"tT\" itemprop=\"name\">[Fudan&amp;Tencent Hunyuan] Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Training<\/div><\/div><button tabindex=\"0\" class=\"play\"><\/button><div class=\"ctrl\"><div class=\"Lctrl\"><\/div><div class=\"Rctrl\"><\/div><\/div><\/div><noscript><a href=\"https:\/\/youtu.be\/XEL2IkSOSQw\" rel=\"nofollow\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/infernews.com\/blog\/wp-content\/plugins\/wp-youtube-lyte\/lyteCache.php?origThumbUrl=https%3A%2F%2Fi.ytimg.com%2Fvi%2FXEL2IkSOSQw%2F0.jpg\" alt=\"[Fudan&amp;amp;Tencent Hunyuan] Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Training\" width=\"853\" height=\"460\" \/><br \/>Watch this video on YouTube<\/a><\/noscript><meta itemprop=\"description\" content=\"Natively training joint video-audio generation models at higher resolutions empowers them to learn richer visual details and sharper motion dynamics. However, full attention incurs quadratic cost and, as resolution increases, spreads attention over increasingly redundant tokens, diluting learning signals for informative content and disrupting pretrained priors. Existing sparse attention methods either target training-free acceleration or overlook the unique structure of joint video-audio data, where cross-modal interactions are inherently concentrated around sound-producing regions. To address this, we propose Prism, a dynamic sparse attention framework for natively training joint video-audio generation models at 2K. In particular, Prism organizes the token sequence into spatiotemporal macro-zones, enabling the attention structure to adapt to local content. For each zone, it estimates local information structure via video feature variance along the channel and feature norms from the audio-to-video cross-attention, jointly capturing how visual content varies directionally and how strongly audio influences each visual region. Based on these signals, Prism dynamically assigns a tailored block shape to each zone, applying finer partitioning along axes of rapid visual content variation and strong audio-visual coupling. This encourages tokens within each block to remain semantically coherent, allowing block-level features to capture both visual content and joint video-audio interaction patterns. Prism further adopts a hybrid block selection strategy to dynamically determine per-query sparsity. Experiments show that Prism achieves 2.5&#36;\\times&#36; training speedup compared to full attention, while surpassing it in generation quality.\"><\/div><\/div><div class=\"lL\" style=\"max-width:100%;width:853px;margin:5px auto;\"><\/div><figcaption><\/figcaption><\/figure>\n\n\n<p class=\"wp-block-paragraph\">\u540c Full Attention \u76f8\u6bd4\uff0cPrism \u4e0d\u518d\u5c0d\u6240\u6709 token \u9032\u884c\u5bc6\u96c6\u4e92\u76f8\u95dc\u6ce8\uff1b\u540c\u6642\uff0c\u5b83\u4ea6\u4e0d\u662f\u55ae\u7d14\u5957\u7528\u53ea\u70ba\u63a8\u7406\u52a0\u901f\u800c\u8a2d\u7684\u7a00\u758f\u6ce8\u610f\u529b\u3002\u9019\u500b\u53d6\u6368\u91dd\u5c0d\u806f\u5408\u5f71\u97f3\u8cc7\u6599\u7684\u7d50\u69cb\u4f5c\u51fa\u8abf\u6574\uff0c\u4ee3\u50f9\u662f\u8a13\u7df4\u6d41\u7a0b\u9700\u8981\u984d\u5916\u8a08\u7b97\u5340\u57df\u8cc7\u8a0a\u548c\u52d5\u614b\u9078\u64c7\uff0c\u5be6\u4f5c\u8907\u96dc\u5ea6\u6703\u9ad8\u65bc\u6a19\u6e96\u6ce8\u610f\u529b\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u9805\u76ee\u63d0\u4f9b Project Page\u3001Hugging Face \u6a21\u578b\u548c\u793a\u7bc4\u5f71\u7247\uff0c\u53ef\u5148\u900f\u904e\u5408\u6210\u7d50\u679c\u3001\u6a21\u578b\u6bd4\u8f03\u3001\u6d88\u878d\u7814\u7a76\u53ca\u89e3\u6790\u5ea6\u64f4\u5c55\u793a\u7bc4\u7406\u89e3\u6548\u679c\u3002\u63d0\u4f9b\u7684\u8cc7\u6599\u672a\u5217\u51fa\u5b8c\u6574\u672c\u5730\u5b89\u88dd\u6307\u4ee4\u6216\u786c\u4ef6\u8981\u6c42\uff0c\u56e0\u6b64\u66ab\u6642\u8f03\u9069\u5408\u7814\u7a76\u5718\u968a\u6309\u8ad6\u6587\u8207\u6a21\u578b\u8cc7\u6e90\u81ea\u884c\u6574\u7406\u74b0\u5883\uff0c\u800c\u4e0d\u662f\u5373\u6642\u5957\u7528\u5230\u4e00\u822c\u5f71\u97f3\u5de5\u5177\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u8a13\u7df4\u901f\u5ea6\u8f03 Full Attention \u63d0\u5347 2.5 \u500d<\/li>\n\n\n\n<li>\u751f\u6210\u8cea\u7d20\u64da\u5831\u8d85\u8d8a Full Attention<\/li>\n\n\n\n<li>\u540c\u6642\u8003\u616e\u8996\u89ba\u8b8a\u5316\u8207 audio-visual coupling<\/li>\n\n\n\n<li>\u9762\u5411\u539f\u751f 2K \u5f71\u97f3\u751f\u6210\u6a21\u578b\u8a13\u7df4<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Prism \u5c0d\u9700\u8981\u8a13\u7df4\u9ad8\u89e3\u50cf\u5ea6\u6587\u5b57\u8f49\u5f71\u7247\u53ca\u540c\u6b65\u8072\u97f3\u6a21\u578b\u7684\u7814\u7a76\u5718\u968a\u8f03\u6709\u50f9\u503c\uff0c\u5c24\u5176\u9069\u5408\u8a08\u7b97\u8cc7\u6e90\u6602\u8cb4\u3001\u5f71\u97f3\u4e92\u52d5\u96c6\u4e2d\u65bc\u5c40\u90e8\u5340\u57df\u7684\u5de5\u4f5c\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/francis-rings.github.io\/Prism\/\" rel=\"noopener noreferrer\"><strong>\u9805\u76ee\u4e3b\u9801<\/strong><\/a> \u00b7 <a href=\"https:\/\/github.com\/Tencent-Hunyuan\/Prism\" rel=\"noopener noreferrer\"><strong>GitHub<\/strong><\/a> \u00b7 <a href=\"https:\/\/huggingface.co\/FrancisRing\/Prism\/tree\/main\" rel=\"noopener noreferrer\"><strong>\u6a21\u578b<\/strong><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Prism \u5c07\u5f71\u7247\u756b\u9762\u8207\u8072\u97f3\u4e92\u52d5\u7d0d\u5165\u7a00\u758f\u6ce8\u610f\u529b\uff0c\u4ee4\u539f\u751f 2K \u5f71\u97f3\u751f\u6210\u6a21\u578b\u8a13\u7df4\u66f4\u5feb\uff0c\u4e26\u517c\u9867\u751f\u6210\u8cea\u7d20\u3002<\/p>\n","protected":false},"author":8,"featured_media":12998,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"wpai_generated_summary":"","footnotes":""},"categories":[133,172,162,76,119,127,178,179,31,29,197,38,128],"tags":[],"class_list":["post-12999","post","type-post","status-publish","format-standard","hentry","category-133","category-172","category-ai-productions","category-76","category-119","category-127","category-google","category-nvidia","category-video","category-audio","category-framework","category-38","category-128"],"_links":{"self":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/12999","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/comments?post=12999"}],"version-history":[{"count":1,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/12999\/revisions"}],"predecessor-version":[{"id":13001,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/12999\/revisions\/13001"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media\/12998"}],"wp:attachment":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media?parent=12999"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/categories?post=12999"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/tags?post=12999"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}