{"id":13987,"date":"2026-10-26T20:21:36","date_gmt":"2026-10-26T12:21:36","guid":{"rendered":"https:\/\/infernews.com\/blog\/statetree-representation\/"},"modified":"2026-10-26T21:36:13","modified_gmt":"2026-10-26T13:36:13","slug":"statetree-representation","status":"publish","type":"post","link":"https:\/\/infernews.com\/blog\/statetree-representation\/","title":{"rendered":"StateTree-Representation\uff1a\u8b93 LLM \u58d3\u7e2e\u5f8c\u4ecd\u4fdd\u7559 BF16 \u7cbe\u5ea6"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/infernews.com\/blog\/wp-content\/uploads\/2026\/10\/StateTree-Representation-LLM-BF16.jpg\" alt=\"BF16 and statetree answering the same question side by side in vLLM; statetree finishes first with the identical answer\"\/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u5927\u578b\u8a9e\u8a00\u6a21\u578b\u5728\u9577\u4e0a\u4e0b\u6587\u6216\u591a\u8acb\u6c42\u5834\u666f\uff0c\u74f6\u9838\u5f80\u5f80\u4e0d\u662f\u7b97\u529b\uff0c\u800c\u662f\u53cd\u8986\u8b80\u53d6\u6b0a\u91cd\u548c KV cache\u3002StateTree \u4ee5\u7d04 12 bits \u5132\u5b58\u539f\u672c 16-bit \u7684\u8cc7\u6599\uff0c\u5c6c\u65bc\u7d50\u5408\u6a21\u578b\u8868\u793a\u6cd5\u8207\u63a8\u7406\u5f15\u64ce\u7684\u958b\u6e90\u9805\u76ee\uff0c\u8655\u7406\u7684\u662f\u8a18\u61b6\u9ad4\u6d41\u91cf\u3001\u5feb\u53d6\u5bb9\u91cf\u8207\u751f\u6210\u901f\u5ea6\u4e4b\u9593\u7684\u53d6\u6368\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5b83\u628a\u6bcf\u7d44\u6b0a\u91cd\u8868\u793a\u6210\u5171\u4eab\u96e2\u6563\u6c7a\u7b56\u7684 state tree\uff0c\u4e26\u4ee5 fused CUDA kernels \u76f4\u63a5\u5728 vLLM \u548c \ud83e\udd17 Transformers \u4e2d\u89e3\u78bc\uff1bKV cache \u5247\u4ee5\u6bcf\u500b vLLM block 16 tokens\u3001\u6bcf\u500b KV head \u56db\u7d44 16 \u00d7 64 records \u5132\u5b58\u3002\u6b0a\u91cd\u8207 cached key\/value \u90fd\u80fd\u9084\u539f\u81f3 exact BF16 bits\uff0c\u4e0d\u6d89\u53ca quantization\u3001calibration \u6216\u7cbe\u5ea6\u63db\u53d6\u58d3\u7e2e\u7387\u3002<\/p>\n\n\n\n<figure class=\"wp-block-video\"><video height=\"660\" style=\"aspect-ratio: 1200 \/ 660;\" width=\"1200\" autoplay controls loop muted src=\"https:\/\/infernews.com\/blog\/wp-content\/uploads\/2026\/10\/StateTree-1.mp4\" playsinline><\/video><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Qwen3-8B \u5728\u55ae\u5f35 A100-40GB\u3001vLLM 0.32 \u7684\u6e2c\u8a66\u4e2d\uff0c\u6b0a\u91cd\u7531 15.3 GiB \u964d\u81f3 12.8 GiB\uff0cKV cache \u6bcf token \u7531 144 KiB \u964d\u81f3 112.5 KiB\uff1b\u540c\u4e00 GPU \u9810\u7b97\u53ef\u5bb9\u7d0d 1.49 \u500d tokens\uff0cbatch 1 \u751f\u6210\u901f\u5ea6\u7531 75.6 \u63d0\u5347\u81f3 97.2 tok\/s\uff0c\u5b8c\u6574\u56de\u7b54\u5feb 1.10 \u81f3 1.28 \u500d\u3002Decode attention \u76f8\u5c0d FlashAttention-2 \u5247\u9054 1.21 \u81f3 2.46 \u500d\uff0c\u4f46\u7d50\u679c\u6703\u53d7 GPU\u3001\u6279\u6b21\u5927\u5c0f\u548c\u4e0a\u4e0b\u6587\u9577\u5ea6\u5f71\u97ff\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u6e2c\u8a66\u9700\u8981 NVIDIA sm_80 \u6216\u66f4\u65b0\u67b6\u69cb\u3001CUDA 12.6\uff0c\u4ee5\u53ca\u9805\u76ee\u5217\u51fa\u7684 PyTorch\u3001vLLM 0.32 \u6216 Transformers 5.0 \u74b0\u5883\uff1b\u53ef\u5148\u7528 greedy generation\u3001prefix-cache hit \u548c\u9577 prompt chunk prefill \u5c0d\u6bd4 BF16\u3002teacher-forced top-1 token \u4e00\u81f4\u7387\u70ba 97.6% \u81f3 99.8%\uff0c\u5dee\u7570\u96c6\u4e2d\u65bc\u8fd1\u4f3c\u6a5f\u7387\u7684 token\uff0c\u9069\u5408\u91cd\u8996\u5bb9\u91cf\u548c\u541e\u5410\u3001\u53c8\u4e0d\u80fd\u63a5\u53d7\u91cf\u5316\u8aa4\u5dee\u7684\u63a8\u7406\u670d\u52d9\u5718\u968a\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u6b0a\u91cd\u53ca KV cache \u5747\u53ef\u9010 bit \u9084\u539f\u81f3 BF16<\/li>\n\n\n\n<li>\u6b0a\u91cd\u4f54\u7528\u6e1b\u5c11 24%\uff0cKV cache \u4f54\u7528\u6e1b\u5c11 22%<\/li>\n\n\n\n<li>\u652f\u63f4 vLLM fused kernels\u3001paged blocks \u548c prefix caching<\/li>\n\n\n\n<li>\u6e2c\u8a66\u7d50\u679c\u6db5\u84cb\u5c0f batch\u3001\u9577\u4e0a\u4e0b\u6587\u53ca\u591a\u8acb\u6c42\u60c5\u5883<\/li>\n\n\n\n<li>\u9700\u8981\u76f8\u5bb9\u7684 NVIDIA GPU\u3001CUDA \u8207\u6307\u5b9a\u6846\u67b6\u7248\u672c<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/huggingface.co\/papers\/2610.31931\" rel=\"noopener noreferrer\"><strong>\u9805\u76ee\u4e3b\u9801<\/strong><\/a> \u00b7 <a href=\"https:\/\/github.com\/ammarali32\/StateTree-Representation\" rel=\"noopener noreferrer\"><strong>GitHub<\/strong><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>StateTree \u540c\u6642\u58d3\u7e2e\u6a21\u578b\u6b0a\u91cd\u8207 KV cache\uff0c\u6e1b\u5c11\u8a18\u61b6\u9ad4\u4f54\u7528\u4e4b\u9918\uff0c\u751f\u6210\u7d50\u679c\u4ecd\u53ef\u9010 bit \u5c0d\u9f4a BF16\u3002<\/p>\n","protected":false},"author":8,"featured_media":13986,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"wpai_generated_summary":"","wpai_meta_description":"","footnotes":""},"categories":[133,76,185,179,197,152,138],"tags":[],"class_list":["post-13987","post","type-post","status-publish","format-standard","hentry","category-133","category-76","category-qwen","category-nvidia","category-framework","category-python","category-138"],"_links":{"self":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/13987","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/comments?post=13987"}],"version-history":[{"count":2,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/13987\/revisions"}],"predecessor-version":[{"id":13992,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/13987\/revisions\/13992"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media\/13986"}],"wp:attachment":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media?parent=13987"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/categories?post=13987"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/tags?post=13987"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}