{"id":3890,"date":"2021-06-10T19:24:06","date_gmt":"2021-06-10T11:24:06","guid":{"rendered":"http:\/\/www.sniper97.cn\/?p=3890"},"modified":"2021-06-10T19:24:06","modified_gmt":"2021-06-10T11:24:06","slug":"%e3%80%90%e6%b7%b1%e5%ba%a6%e5%ad%a6%e4%b9%a0%e7%ac%94%e8%ae%b0%e3%80%91bleu%e4%b8%8erouge","status":"publish","type":"post","link":"http:\/\/www.sniper97.cn\/index.php\/note\/deep-learning\/base\/3890\/","title":{"rendered":"\u3010\u6df1\u5ea6\u5b66\u4e60\u7b14\u8bb0\u3011BLEU\u4e0eROUGE"},"content":{"rendered":"\n<p>\u672c\u7ae0\u7528\u6765\u8bf4\u660e\u4e00\u4e9b\u6df1\u5ea6\u5b66\u4e60\u7ffb\u8bd1\uff08\u6587\u672c\u751f\u6210\uff09\u4efb\u52a1\u4e2d\u7684\u51e0\u4e2a\u6307\u6807\u3002<\/p>\n\n\n<p>BLEU\u548cROUGE\u53ef\u4ee5\u8bf4\u662f\u57fa\u672c\u4e0a\u4e00\u6478\u4e00\u6837\u4e86\uff0c\u533a\u522b\u5c31\u662f<strong>BLEU\u53ea\u8ba1\u7b97\u51c6\u786e\u7387<\/strong>\uff0c\u800c<strong>ROUGE\u53ea\u8ba1\u7b97\u53ec\u56de\u7387<\/strong>\u3002<\/p>\n\n\n<p>\u4e3a\u4e86\u8bf4\u660e\u8fd9\u4e24\u79cd\u8bc4\u4ef7\u6307\u6807\uff0c\u6211\u4eec\u9996\u5148\u5f15\u5165\u4e24\u6bb5\u6587\u5b57\uff1a<\/p>\n\n\n<ul><li>\u8fd9\u82b1\u513f\u5f00\u7684\u771f\u7f8e\u4e3d\u554a<\/li><li>\u8fd9\u5f00\u7684\u82b1\u513f\u771f\u7f8e\u4e3d\u554a<\/li><\/ul>\n\n\n<p>\u5176\u4e2d\u7b2c\u4e8c\u4e2a\u4e3a\u9884\u6d4b\u3002<\/p>\n\n\n<h2 class=\"wp-block-heading\">BLEU<\/h2>\n\n\n<p>\u8fd9\u4e2a\u7b97\u662f\u6bd4\u8f83\u53e4\u8001\u7684\u8bc4\u4ef7\u6307\u6807\u4e86\uff0c\u4e3b\u8981\u8861\u91cf\u6587\u672c\u7ffb\u8bd1\u7684\u51c6\u786e\u7387\u3002<\/p>\n\n\n<p>\u9996\u5148\u8ba1\u7b97BLEU-1\uff0c\u6211\u4eec\u9700\u8981\u5c06\u53e5\u5b50\u62c6\u5f00<\/p>\n\n\n<ul><li>\u8fd9 \u82b1 \u513f \u5f00 \u7684 \u771f \u7f8e \u4e3d \u554a<\/li><li>\u8fd9 \u5f00 \u7684 \u82b1 \u513f \u771f \u7f8e \u4e3d \u554a<\/li><\/ul>\n\n\n<p>\u5bf9\u4e8e<strong>\u9884\u6d4b\u7ed3\u679c<\/strong>\uff0c9\u4e2a\u5b57\u5168\u90e8\u90fd\u5728<strong>\u771f\u5b9e\u7ed3\u679c<\/strong>\u4e2d\u51fa\u73b0\uff0c\u56e0\u6b64BLEU-1\u4e3a9\/9\uff0c\u540c\u7406\uff1a<\/p>\n\n\n<ul><li>\u8fd9\u82b1 \u82b1\u513f \u513f\u5f00 \u5f00\u7684 \u7684\u771f \u771f\u7f8e \u7f8e\u4e3d \u4e3d\u554a<\/li><li>\u8fd9\u5f00 \u5f00\u7684 \u7684\u82b1 \u82b1\u513f \u513f\u771f \u771f\u7f8e \u7f8e\u4e3d \u4e3d\u554a<\/li><\/ul>\n\n\n<p>\u5bf9\u4e8eBLEU-2\uff0c\u9884\u6d4b\u7ed3\u679c\u4e2d\u67094\u4e2a\u6ca1\u6709\u5728\u771f\u5b9e\u7ed3\u679c\u4e2d\u51fa\u73b0\uff0c\u5206\u522b\u4e3a\u201c\u8fd9\u5f00 \u7684\u82b1 \u513f\u771f\u201d\uff0c\u56e0\u6b64BLEU-2\u4e3a5\/8\u3002<\/p>\n\n\n<p>\u540c\u7406\u6211\u4eec\u53ef\u77e5BLEU-3\uff0cBLEU-4\u5747\u4e3a2\/7\uff0c1\/6\u3002<\/p>\n\n\n<p><strong>\u53ef\u4ee5\u770b\u51fa\uff0c\u4f4e\u9636n-gram\u53ef\u4ee5\u8861\u91cf\u51c6\u786e\u6027\uff0c\u800c\u9ad8\u9636n-gram\u53ef\u4ee5\u8861\u91cf\u6700\u7ec8\u53e5\u5b50\u7684\u6d41\u7545\u6027\u3002<\/strong><\/p>\n\n\n<p>\u5f53\u7136\uff0c\u6700\u7ec8\u7684BLEU\u5e76\u4e0d\u662f\u4e0a\u8ff0\u7684\u503c\u3002<\/p>\n\n\n<p>\u6211\u4eec\u53ef\u4ee5\u53d1\u73b0\uff1a\u5982\u679c\u6a21\u578b\u603b\u662f\u9884\u6d4b\u5f88\u77ed\u7684\u5e8f\u5217\uff0c\u90a3\u4e48\u5c31\u53ef\u4ee5\u83b7\u5f97\u8f83\u9ad8\u7684BLEU\u5206\u6570\uff0c\u6bd4\u5982\uff1a<\/p>\n\n\n<ul><li>\u4eca\u5929\u662f\u4e2a\u597d\u65e5\u5b50\uff0c\u56e0\u4e3a\u4eca\u5929\u662f\u4e2d\u534e\u4eba\u6c11\u5171\u548c\u56fd\u7684\u56fd\u5e86\u8282\u3002<\/li><li>\u4e2d\u534e\u4eba\u6c11\u5171\u548c\u56fd<\/li><\/ul>\n\n\n<p>\u5bf9\u4e8e\u4e0b\u9762\u7684\u9884\u6d4b\u53e5\uff0cBLUE-1\uff0c2\uff0c3\uff0c4\u5747\u4e3a1\u3002\u8fd9\u663e\u7136\u4e0d\u662f\u6211\u4eec\u60f3\u8981\u7684\u7ed3\u679c\u3002<\/p>\n\n\n<p>\u56e0\u6b64\u5728BLUE\u4e2d\u8fd8\u5f15\u5165\u4e86\u4e00\u4e2a\u60e9\u7f5a\u9879\uff0c\u4ee5\u60e9\u7f5a\u8fd9\u79cd\u9884\u6d4b\u53e5\u957f\u5ea6\u5c0f\u4e8e\u771f\u5b9e\u53e5\u7684\u60c5\u51b5\u3002<\/p>\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"564\" height=\"240\" src=\"\/wp-content\/uploads\/2021\/08\/image.png\" alt=\"\" class=\"wp-image-3893\" srcset=\"http:\/\/www.sniper97.cn\/wp-content\/uploads\/2021\/08\/image.png 564w, http:\/\/www.sniper97.cn\/wp-content\/uploads\/2021\/08\/image-300x128.png 300w\" sizes=\"(max-width: 564px) 100vw, 564px\" \/><\/figure><\/div>\n\n\n<pre class=\"wp-block-preformatted\">if hyp_len &gt; closest_ref_len:\n    return 1\n# If hypothesis is empty, brevity penalty = 0 should result in BLEU = 0.0\nelif hyp_len == 0:\n    return 0\nelse:\n    return math.exp(1 - closest_ref_len \/ hyp_len)<\/pre>\n\n\n<p>\u6211\u4eec\u5c06BLEU-1\uff0c2\uff0c3\uff0c4\u5206\u522b\u53d6\u5bf9\u6570\u7136\u540e\u6c42\u5e73\u5747,\u7136\u540e\u53d6\u6307\u6570\uff0c\u5c31\u662f\u6700\u7ec8\u7684BLEU\u3002<\/p>\n\n\n<pre class=\"wp-block-preformatted\">import math<br \/>from nltk.translate.bleu_score import sentence_bleu<br \/><br \/>log_avg = (math.log(9 \/ 9) + math.log(5 \/ 8) + math.log(2 \/ 7) + math.log(1 \/ 6)) \/ 4<br \/>bp = 1  # l_pred == l_true  bp=1<br \/>print(math.exp(log_avg))  # 0.41535092372063953<br \/><br \/>r = sentence_bleu(<br \/>    references=[list('\u8fd9\u82b1\u513f\u5f00\u7684\u771f\u7f8e\u4e3d\u554a')],<br \/>    hypothesis=list('\u8fd9\u5f00\u7684\u82b1\u513f\u771f\u7f8e\u4e3d\u554a'),<br \/>)<br \/>print(r)  # 0.41535092372063953<\/pre>\n\n\n<p>\u6700\u7ec8\uff0c\u53e5\u5b50\u7684BLEU\u5206\u6570\u4e3a0.42\u3002<\/p>\n\n\n<p><strong>\u503c\u5f97\u6ce8\u610f\u7684\u662f\uff0c\u867d\u7136BLEU\u6307\u6807\u589e\u52a0\u4e86\u957f\u5ea6\u60e9\u7f5a\u9879\uff0c\u4f46\u662f\u6548\u679c\u5e76\u6ca1\u6709\u7279\u522b\u597d\uff0c\u56e0\u6b64BLEU\u4f9d\u7136\u504f\u5411\u4e8e\u8f83\u77ed\u7684\u6587\u672c\u8bc4\u4ef7\u3002<\/strong><\/p>\n\n\n<h2 class=\"wp-block-heading\">ROUGE<\/h2>\n\n\n<p><strong>BLEU\u662f\u4e00\u4e2a\u53ea\u770b\u51c6\u786e\u7387\u7684\u6307\u6807\uff0c\u800cROUGE\u662f\u4e00\u4e2a\u53ea\u770b\u53ec\u56de\u7387\u7684\u6307\u6807<\/strong>\uff0c\u6574\u4f53\u6d41\u7a0b\u6765\u770b\uff0c\u5c31\u662fBLEU\u7684\u6bd4\u8f83\u5bf9\u8c61\u53d1\u751f\u4e86\u53d8\u5316\u3002<\/p>\n\n\n<ul><li>\u8fd9 \u82b1 \u513f \u5f00 \u7684 \u771f \u7f8e \u4e3d \u554a<\/li><li>\u8fd9 \u5f00 \u7684 \u82b1 \u513f \u771f \u7f8e \u4e3d \u554a<\/li><\/ul>\n\n\n<p>ROUGE-1\u4e3a1\uff0c\u56e0\u4e3a<strong>\u771f\u5b9e\u6807\u7b7e<\/strong>\u4e2d \u7684\u6bcf\u4e00\u4e2a\u5b57\u90fd\u53ef\u4ee5\u5728<strong>\u9884\u6d4b\u7ed3\u679c<\/strong>\u4e2d\u627e\u5230\u3002<\/p>\n\n\n<ul><li>\u8fd9\u82b1 \u82b1\u513f \u513f\u5f00 \u5f00\u7684 \u7684\u771f \u771f\u7f8e \u7f8e\u4e3d \u4e3d\u554a<\/li><li>\u8fd9\u5f00 \u5f00\u7684 \u7684\u82b1 \u82b1\u513f \u513f\u771f \u771f\u7f8e \u7f8e\u4e3d \u4e3d\u554a<\/li><\/ul>\n\n\n<p>ROUGE-2\u4e3a5\/8\uff0c\u56e0\u4e3a\u771f\u5b9e\u6807\u7b7e\u4e2d\u7684\u201c\u8fd9\u82b1 \u513f\u5f00 \u7684\u771f\u201d\u4e0d\u5728\u9884\u6d4b\u7ed3\u679c\u4e2d\u3002<\/p>\n\n\n<p>\u53ef\u4ee5\u770b\u51fa\uff0c\u76f8\u6bd4BLEU\u53ea\u662f\u6bd4\u8f83\u5bf9\u8c61\u53d8\u4e86\uff0cYX\u53d8\u6210\u4e86XY\u3002<\/p>\n\n\n<p>ROUGE-L\u76f8\u5bf9\u6709\u4e00\u4e9b\u53d8\u5316\uff0cL\u4ee3\u8868LCS\uff0c\u6700\u957f\u516c\u5171\u5b50\u4e32\u3002<\/p>\n\n\n<p>\u516c\u5f0f\u4e3a\uff1a<\/p>\n\n\n<pre class=\"wp-block-preformatted\">r_lcs = llcs \/ m<br \/>p_lcs = llcs \/ n<br \/><br \/>f_lcs = 2.0 * ((p_lcs * r_lcs) \/ (p_lcs + r_lcs + 1e-8))<\/pre>\n\n\n<p>\u5176\u4e2d\uff0cllcs\u4e3a\u6700\u957f\u516c\u5171\u5b50\u4e32\u957f\u5ea6\uff0cm\uff0cn\u5206\u522b\u4e3a\u4e24\u4e2a\u53e5\u5b50\u7684\u957f\u5ea6\u3002<\/p>\n\n\n<p>\u4f8b\u5982\uff1a<\/p>\n\n\n<ul><li>\u8fd9\u82b1\u513f\u5f00\u7684\u771f\u7f8e\u4e3d\u554a<\/li><li>\u8fd9\u5f00\u7684\u82b1\u513f\u771f\u7f8e\u4e3d\u554a<\/li><\/ul>\n\n\n<p>\u7684\u6700\u957f\u516c\u5171\u5b50\u4e32\u957f\u5ea6\u4e3a\u4e3a7\uff08\u8fd9\u82b1\u513f\u771f\u7f8e\u4e3d\u554a\uff08XY\uff09\u6216 \u8fd9\u5f00\u7684\u771f\u7f8e\u4e3d\u554a\uff08YX\uff09\uff09\u3002\u56e0\u6b64r_lcs\/p_lcs \u4e3a 7\/9 f_lcs\u4e3a2*7\/18 = 7\/9.<\/p>\n\n\n<p>\u56e0\u6b64\uff0cROUGE-L\u4e3a7\/9\u3002<\/p>\n\n\n<pre class=\"wp-block-preformatted\">from rouge import Rouge\nrouge = Rouge()\nr = rouge.get_scores(\n    hyps=' '.join(list('\u8fd9\u82b1\u513f\u5f00\u7684\u771f\u7f8e\u4e3d\u554a')),\n    refs=' '.join(list('\u8fd9\u5f00\u7684\u82b1\u513f\u771f\u7f8e\u4e3d\u554a'))\n)\nfor k, v in r[0].items():\n    print(k, v['r'])\n#output\nrouge-1 1.0\nrouge-2 0.625\nrouge-l 0.7777777777777778<\/pre>\n\n\n<p><strong>\u7531\u4e8eROUGE\u5e76\u4e0d\u68c0\u67e5\u8fde\u8d2f\u6027\uff0c\u56e0\u6b64ROUGE\u5bf9\u4e8e\u8bed\u4e49\u8fde\u8d2f\u6027\u8f83\u5dee\u7684\u6a21\u578b\u4ee5\u53ca\u4efb\u52a1\uff08\u6bd4\u5982\u4f7f\u7528\u673a\u5668\u5b66\u4e60\u8fdb\u884c\u6587\u672c\u7ffb\u8bd1\uff09\u4e0d\u592a\u53cb\u597d\uff0c\u5bf9\u4e8e\u6df1\u5ea6\u5b66\u4e60\u7c7b\u7684\u4efb\u52a1\uff08\u8fd9\u7c7b\u6a21\u578b\u751f\u6210\u7ed3\u679c\u5f80\u5f80\u6709\u8f83\u597d\u7684\u8bed\u4e49\u8fde\u8d2f\u6027\uff09\u8f83\u4e3a\u53cb\u597d\u3002<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u672c\u7ae0\u7528\u6765\u8bf4\u660e\u4e00\u4e9b\u6df1\u5ea6\u5b66\u4e60\u7ffb\u8bd1\uff08\u6587\u672c\u751f\u6210\uff09\u4efb\u52a1\u4e2d\u7684\u51e0\u4e2a\u6307\u6807\u3002 BLEU\u548cROUGE\u53ef\u4ee5\u8bf4\u662f\u57fa\u672c\u4e0a\u4e00\u6478\u4e00 [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[8,9],"tags":[33,37,39],"views":9058,"_links":{"self":[{"href":"http:\/\/www.sniper97.cn\/index.php\/wp-json\/wp\/v2\/posts\/3890"}],"collection":[{"href":"http:\/\/www.sniper97.cn\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.sniper97.cn\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.sniper97.cn\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.sniper97.cn\/index.php\/wp-json\/wp\/v2\/comments?post=3890"}],"version-history":[{"count":0,"href":"http:\/\/www.sniper97.cn\/index.php\/wp-json\/wp\/v2\/posts\/3890\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.sniper97.cn\/index.php\/wp-json\/wp\/v2\/media?parent=3890"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.sniper97.cn\/index.php\/wp-json\/wp\/v2\/categories?post=3890"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.sniper97.cn\/index.php\/wp-json\/wp\/v2\/tags?post=3890"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}