{"id":4106,"date":"2024-12-25T16:56:48","date_gmt":"2024-12-25T08:56:48","guid":{"rendered":"https:\/\/infernews.com\/?p=4106"},"modified":"2024-12-25T17:03:34","modified_gmt":"2024-12-25T09:03:34","slug":"performer%e3%80%81longformer-%e5%92%8c-reformer","status":"publish","type":"post","link":"https:\/\/infernews.com\/blog\/performer%e3%80%81longformer-%e5%92%8c-reformer\/","title":{"rendered":"Performer\u3001Longformer \u548c Reformer ?"},"content":{"rendered":"\n<p>2017 \u5e74\u590f\u5929\uff0c\u4e00\u7fa4 Google Brain \u7814\u7a76\u4eba\u54e1\u6084\u6084\u767c\u8868\u4e86\u4e00\u7bc7\u5c07\u6c38\u9060\u6539\u8b8a\u4eba\u5de5\u667a\u6167\u767c\u5c55\u8ecc\u8de1\u7684\u8ad6\u6587\u3002\u9019\u4efd &#8220;\u6ce8\u610f\u529b\u5c31\u662f\u4f60\u6240\u9700\u8981\u7684\u4e00\u5207&#8221; (<a href=\"https:\/\/arxiv.org\/pdf\/1706.03762\">Attention Is All You Need)<\/a> \u7684\u5b78\u8853\u8ad6\u6587\u3002\u7576\u6642\u4eba\u5de5\u667a\u6167\u7814\u7a76\u754c\u4e4b\u5916\u5f88\u5c11\u6709\u4eba\u77e5\u9053\u9019\u4e00\u9ede\uff0c\u4f46\u9019\u7bc7\u8ad6\u6587\u5c07\u70ba\u4f60\u4eca\u5929\u807d\u8aaa\u904e\u7684\u5e7e\u4e4e\u6240\u6709\u4e3b\u8981\u751f\u6210\u5f0f\u4eba\u5de5\u667a\u6167\u6a21\u578b\u5960\u5b9a\u57fa\u790e\uff0c\u5f9e OpenAI \u7684 GPT \u5230 Meta \u7684 LLaMA \u8b8a\u9ad4\u3001BERT\u3001 Claude\u3001Bard \u7b49\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"ededec\" data-has-transparency=\"false\" style=\"--dominant-color: #ededec;\" loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"587\" src=\"\/blog\/wp-content\/uploads\/2024\/12\/2024070141-1024x587.png\" alt=\"\" class=\"wp-image-4107 not-transparent\" srcset=\"https:\/\/infernews.com\/blog\/wp-content\/uploads\/2024\/12\/2024070141-1024x587.png 1024w, https:\/\/infernews.com\/blog\/wp-content\/uploads\/2024\/12\/2024070141-300x172.png 300w, https:\/\/infernews.com\/blog\/wp-content\/uploads\/2024\/12\/2024070141-768x440.png 768w, https:\/\/infernews.com\/blog\/wp-content\/uploads\/2024\/12\/2024070141-640x367.png 640w, https:\/\/infernews.com\/blog\/wp-content\/uploads\/2024\/12\/2024070141.png 1030w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>\u65b0\u7684\u67b6\u69cb\u65bc 2020 \u5e74\u5df2\u7d93\u51fa\u73fe\uff0c\u4f8b\u5982 <a href=\"https:\/\/arxiv.org\/abs\/2009.06732\" target=\"_blank\" rel=\"noreferrer noopener\">Performer\u3001Longformer \u548c Reformer<\/a>\uff0c\u65e8\u5728\u63d0\u9ad8\u8d85\u9577\u5e8f\u5217\u7684\u6ce8\u610f\u529b\u6548\u7387\u3002\u76ee\u524d\u5176\u4ed6\u4eba\u6b63\u5728\u5617\u8a66\u6df7\u5408\u65b9\u6cd5\uff0c\u5c07 Transformer \u5340\u584a\u8207\u5176\u4ed6\u5c08\u7528\u5c64\u7d50\u5408\u3002\u9019\u500b\u9818\u57df\u7d55\u975e\u505c\u6eef\u4e0d\u524d\u3002\u5c55\u671b\u672a\u4f86\uff0c\u6bcf\u4e00\u9805\u65b0\u63d0\u6848\u90fd\u5c07\u53d7\u5230\u5be9\u67e5\u3001\u8208\u596e\uff0c\u751a\u81f3\u6050\u61fc\u3002<\/p>\n\n\n\n<p><a href=\"http:\/\/www.uml.org.cn\/ai\/202407014.asp\" target=\"_blank\" rel=\"noreferrer noopener\">\u4e00\u6587\u641e\u61c2 Transformer<\/a>\uff08\u7e3d\u9ad4\u67b6\u69cb &amp; \u4e09\u7a2e\u6ce8\u610f\u529b\u5c64\uff09<\/p>\n","protected":false},"excerpt":{"rendered":"<p>2017 \u5e74\u590f\u5929\uff0c\u4e00\u7fa4 Google Brain \u7814\u7a76\u4eba\u54e1\u6084\u6084\u767c\u8868\u4e86\u4e00\u7bc7\u5c07\u6c38\u9060\u6539\u8b8a\u4eba\u5de5\u667a\u6167\u767c\u5c55\u8ecc\u8de1\u7684\u8ad6\u6587\u3002\u9019\u4efd &#8220;\u6ce8\u610f\u529b\u5c31\u662f\u4f60\u6240\u9700\u8981\u7684\u4e00\u5207&#8221; (Attention Is All You Need) \u7684\u5b78\u8853\u8ad6\u6587\u3002\u7576\u6642\u4eba\u5de5\u667a\u6167\u7814\u7a76\u754c\u4e4b\u5916\u5f88\u5c11\u6709\u4eba\u77e5\u9053\u9019\u4e00\u9ede\uff0c\u4f46\u9019\u7bc7\u8ad6\u6587\u5c07\u70ba\u4f60\u4eca\u5929\u807d\u8aaa\u904e\u7684\u5e7e\u4e4e\u6240\u6709\u4e3b\u8981\u751f\u6210\u5f0f\u4eba\u5de5\u667a\u6167\u6a21\u578b\u5960\u5b9a\u57fa\u790e\uff0c\u5f9e OpenAI \u7684 GPT \u5230 Meta \u7684 LLaMA \u8b8a\u9ad4\u3001BERT\u3001 Claude\u3001Bard \u7b49\u3002 \u65b0\u7684\u67b6\u69cb\u65bc 2020 \u5e74\u5df2\u7d93\u51fa\u73fe\uff0c\u4f8b\u5982 Performer\u3001Longformer \u548c Reformer\uff0c\u65e8\u5728\u63d0\u9ad8\u8d85\u9577\u5e8f\u5217\u7684\u6ce8\u610f\u529b\u6548\u7387\u3002\u76ee\u524d\u5176\u4ed6\u4eba\u6b63\u5728\u5617\u8a66\u6df7\u5408\u65b9\u6cd5\uff0c\u5c07 Transformer \u5340\u584a\u8207\u5176\u4ed6\u5c08\u7528\u5c64\u7d50\u5408\u3002\u9019\u500b\u9818\u57df\u7d55\u975e\u505c\u6eef\u4e0d\u524d\u3002\u5c55\u671b\u672a\u4f86\uff0c\u6bcf\u4e00\u9805\u65b0\u63d0\u6848\u90fd\u5c07\u53d7\u5230\u5be9\u67e5\u3001\u8208\u596e\uff0c\u751a\u81f3\u6050\u61fc\u3002 \u4e00\u6587\u641e\u61c2 Transformer\uff08\u7e3d\u9ad4\u67b6\u69cb &amp; \u4e09\u7a2e\u6ce8\u610f\u529b\u5c64\uff09<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"googlesitekit_rrm_CAowvqSiDA:productID":"","footnotes":""},"categories":[23],"tags":[],"class_list":["post-4106","post","type-post","status-publish","format-standard","hentry","category-news"],"_links":{"self":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/4106","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/comments?post=4106"}],"version-history":[{"count":0,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/posts\/4106\/revisions"}],"wp:attachment":[{"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/media?parent=4106"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/categories?post=4106"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infernews.com\/blog\/wp-json\/wp\/v2\/tags?post=4106"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}