{"id":11142,"date":"2026-08-18T02:54:04","date_gmt":"2026-08-17T18:54:04","guid":{"rendered":"https:\/\/infernews.com\/blog\/hrm-text-is-a-1b-text-generation-model-based-on-the-hrm-architecture-strengthene\/"},"modified":"2026-08-18T02:54:04","modified_gmt":"2026-08-17T18:54:04","slug":"hrm-text-is-a-1b-text-generation-model-based-on-the-hrm-architecture-strengthene","status":"publish","type":"post","link":"https:\/\/infernews.com\/blog\/hrm-text-is-a-1b-text-generation-model-based-on-the-hrm-architecture-strengthene\/","title":{"rendered":"HRM-Text \u628a\u57fa\u790e\u6a21\u578b\u9810\u8a13\u7df4\u9580\u6abb\u58d3\u5230\u5343\u7f8e\u5143\u7d1a"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/infernews.com\/blog\/wp-content\/uploads\/2026\/08\/banner-1.png\" alt=\"banner\"><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u6700\u503c\u5f97\u7559\u610f\u7684\u5730\u65b9\uff0c\u4e0d\u662f\u53c8\u591a\u4e00\u500b 1B \u7d1a\u6587\u5b57\u6a21\u578b\uff0c\u800c\u662f\u6709\u4eba\u628a\u300c\u7531\u96f6\u958b\u59cb\u9810\u8a13\u7df4 foundation model\u300d\u6574\u7406\u6210\u4e00\u500b\u53ef\u843d\u5730\u7684\u958b\u6e90\u9805\u76ee\u3002HRM-Text \u5c6c\u65bc\u6a21\u578b\u52a0\u8a13\u7df4\u6846\u67b6\u985e\u578b\uff0c\u8655\u7406\u7684\u662f\u4e2d\u5c0f\u5718\u968a\u60f3\u81ea\u5efa\u6587\u5b57\u751f\u6210\u6a21\u578b\u6642\uff0c\u7b97\u529b\u3001\u6578\u64da\u91cf\u540c\u5de5\u7a0b\u9580\u6abb\u90fd\u904e\u9ad8\u7684\u554f\u984c\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5b83\u63a1\u7528 Hierarchical Recurrent Model\uff08HRM\uff09\u67b6\u69cb\uff0c\u4e26\u52a0\u5165 task completion \u540c latent space reasoning\uff0c\u91cd\u9ede\u4e0d\u662f\u55ae\u7d14\u8ffd\u66f4\u5927\u53c3\u6578\uff0c\u800c\u662f\u7528 hierarchical recurrent architecture\u3001PrefixLM sequence packing\u3001FlashAttention 3 \u540c PyTorch FSDP2\uff0c\u628a\u9810\u8a13\u7df4\u6210\u672c\u58d3\u5230\u50b3\u7d71\u505a\u6cd5\u7684 130 \u81f3 600 \u500d\u66f4\u4f4e\u7b97\u529b\u9700\u6c42\uff0c\u4ee5\u53ca 150 \u81f3 900 \u500d\u66f4\u4f4e\u6578\u64da\u9700\u6c42\u3002\u5462\u500b\u53d6\u6368\u5f88\u9bae\u660e\uff1a\u63db\u4f86\u7684\u4e0d\u662f\u842c\u80fd\u90e8\u7f72\u74b0\u5883\uff0c\u800c\u662f\u4e00\u689d\u504f\u5411\u7814\u7a76\u8207\u8a13\u7df4\u7aef\u3001\u5c0d Hopper-class GPU \u4ecd\u6709\u660e\u78ba\u8981\u6c42\u7684\u8def\u7dda\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u90e8\u7f72\u540c\u6e2c\u8a66\u65b9\u5f0f\u4ea6\u7b97\u5b8c\u6574\u3002\u5132\u5b58\u5eab\u5df2\u5305\u542b training\u3001checkpointing\u3001evaluation\uff0c\u540c checkpoint \u8f49\u6210 Hugging Face Transformers \u683c\u5f0f\u7684\u5de5\u5177\uff0c\u914d\u5408 companion \u7684 <code>data_io<\/code> \u7ba1\u7dda\u6e96\u5099 tokenized data \u5f8c\uff0c\u5c31\u53ef\u4ee5\u6309\u55ae\u7bc0\u9ede\u6216\u591a\u7bc0\u9ede\u65b9\u5f0f\u958b\u8dd1\uff1b\u4e0d\u904e native Transformers \u652f\u63f4\u8981\u7b49\u4e0b\u4e00\u500b release\uff0cnative vLLM \u652f\u63f4\u4ea6\u4ecd\u5728\u9032\u884c\u4e2d\uff0c\u6240\u4ee5\u73fe\u6642\u8f03\u9069\u5408\u60f3\u91cd\u73fe\u8a13\u7df4\u6d41\u7a0b\u3001\u9a57\u8b49\u6210\u672c\u7d50\u69cb\uff0c\u6216\u8005\u7814\u7a76 HRM \u67b6\u69cb\u672c\u8eab\u7684\u5718\u968a\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u53c3\u8003\u8dd1\u5206\u4e0d\u7b97\u4fdd\u5b88\uff1a0.6B \u7248\u672c\u7528 8 \u5f35 H100\u3001\u7d04 50 \u5c0f\u6642\uff0cGSM8k \u6709 77.6%\uff0cMATH 51.2%\uff1b1B \u7248\u672c\u7528 16 \u5f35 H100\u3001\u7d04 46 \u5c0f\u6642\uff0cGSM8k \u63d0\u5347\u5230 84.7%\uff0cMMLU 60.7%\uff0cARC-C 81.9%\u3002\u5462\u5572\u6578\u5b57\u672a\u5fc5\u4ee3\u8868\u5b83\u5df2\u7d93\u9069\u5408\u6240\u6709\u7522\u54c1\u5316\u5834\u666f\uff0c\u4f46\u8db3\u4ee5\u8b49\u660e\u9019\u689d\u8def\u4e0d\u662f\u7d19\u4e0a\u8ac7\u5175\uff0c\u5c24\u5176\u5c0d\u5927\u5b78\u5be6\u9a57\u5ba4\u3001\u570b\u5bb6\u8a9e\u8a00\u6a21\u578b\u8a08\u5283\uff0c\u6216\u8005\u60f3\u70ba\u5c0f\u8a9e\u7a2e\u3001\u81ea\u6709\u8a9e\u6599\u5efa\u7acb base model \u7684\u5718\u968a\uff0c\u6709\u76f8\u7576\u73fe\u5be6\u7684\u5438\u5f15\u529b\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u628a foundation model \u9810\u8a13\u7df4\u6210\u672c\u58d3\u5230\u7d04\u5343\u7f8e\u5143\u7d1a\uff0c\u91cd\u9ede\u5728\u5b8c\u6574\u6d41\u7a0b\u800c\u4e0d\u53ea\u662f\u4e00\u500b\u6a21\u578b\u6b0a\u91cd<\/li>\n<li>\u540c\u6642\u63d0\u4f9b\u8a13\u7df4\u3001\u8a55\u4f30\u3001checkpoint \u8f49\u63db\u5de5\u5177\uff0c\u65b9\u4fbf\u91cd\u73fe\u8207\u5ef6\u4f38\u7814\u7a76<\/li>\n<li>\u76ee\u524d\u8f03\u4f9d\u8cf4 H100 \u7b49 Hopper-class GPU\uff0c\u90e8\u7f72\u5f48\u6027\u672a\u7b97\u5b8c\u5168\u6210\u719f<\/li>\n<li>\u9069\u5408\u7814\u7a76\u6a5f\u69cb\u3001\u8a9e\u8a00\u79d1\u6280\u5718\u968a\uff0c\u540c\u8981\u81ea\u5efa\u57fa\u790e\u6a21\u578b\u80fd\u529b\u7684\u9805\u76ee<\/li>\n<li>\u539f\u751f Transformers \u8207 vLLM \u652f\u63f4\u4ecd\u5728\u88dc\u9f4a\uff0c\u77ed\u671f\u5167\u8f03\u504f\u8a13\u7df4\u7aef\u4f7f\u7528<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/www.foundationmodels.dk\/\" rel=\"noopener noreferrer\"><strong>\u9805\u76ee\u4e3b\u9801<\/strong><\/a> \u00b7 <a href=\"https:\/\/github.com\/schneiderkamplab\/HRM-Text\" rel=\"noopener noreferrer\"><strong>GitHub<\/strong><\/a> \u00b7 <a href=\"https:\/\/huggingface.co\/danish-foundation-models\" rel=\"noopener noreferrer\"><strong>\u6a21\u578b<\/strong><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u60f3\u81ea\u5df1\u9810\u8a13\u7df4\u6587\u5b57\u6a21\u578b\uff0c\u901a\u5e38\u5148\u88ab\u7b97\u529b\u540c\u6578\u64da\u6210\u672c\u52f8\u9000\u3002HRM-Text\u5617\u8a66\u628a\u5462\u9053\u9580\u6abb\u5927\u5e45\u58d3\u4f4e\uff0c\u9023\u5b8c\u6574\u8a13\u7df4\u6d41\u7a0b\u90fd\u4e00\u4f75\u958b\u653e\u3002<\/p>\n","protected":false},"author":8,"featured_media":11141,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ai_generated_summary":"","footnotes":""},"categories":[133,152,119,76,127,199],"tags":[],"class_list":["post-11142","post","type-post","status-publish","format-standard","hentry","category-133","category-python","category-119","category-76","category-127","category-dataset-"],"_links":{"self":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/11142","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/comments?post=11142"}],"version-history":[{"count":0,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/11142\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media\/11141"}],"wp:attachment":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media?parent=11142"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/categories?post=11142"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/tags?post=11142"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}