Paper reading (2020)

このセミナーについて (About this seminar)

担当者が見つけた面白い研究を紹介するセミナーです．以下の内容で構成されます． In this seminar, presenters share interesting studies that they found. The seminar consists of the following elements.

  1. 論文紹介 (Paper presentation): 面白いと思う論文（基本的にLong Paper）を選んで，その内容を著者の代わりになったつもりで発表する． A presenter reads a paper (long paper is strongly preffered) that he or she finds interesting, and explains the content as if he or she was an author of the paper.
  2. 著者プレゼンの上映と解説 (Watching talk videos): 担当者は面白いと思う研究の発表動画を選び，その論文の内容を把握する．担当者の補足説明を聞きながら，参加者みんなで発表動画を鑑賞する． Finding an interesting video presentation, a presenter reads and understands the content of the paper. Attendees watch the presentation video of the author of the paper. The presenter is expected to explain supplementary information about the research.

前者 (1) はプレゼンテーションの練習を兼ねています．後者 (2) は「よい」プレゼンテーションを鑑賞しながら，英語でのプレゼンテーションやディスカッションに慣れることを狙っています． The former (1) aims at practicing presentations. The latter (2) aims at improving listening and discussion skills in English as we enjoy good presentations.

発表者は一人目は (1) を，二人目は (1) か (2) のどちらかを担当します． The 1st presentator take (1). The 2nd one can select (1) or (2).

発表時間 (Presentation Time)

  • Presentation: 15 ~ 20 minnutes
  • QA: 5 ~ 10 minutes

日時 (Date and time)

  • Wednesday 12:45~

参加者 (Attendee)

  • 全員

:exclamation: 発表を登録するときは，著者名，発表年，タイトル，会議名／ジャーナル名，（巻，号，ページ番号など）を必ず記入してください． :exclamation: Please include author names, publication year, title, conference/journal name, (volume and page numbers) when you add an entry for presentation.

Tips

  • リモートミーティングでビデオプレゼンテーションを行う場合は，画面およびオーディオを共有してください．
  • When you stream the video presentation via remote meeting, please share the both screen and system audio.
    • For Zoom: You don’t have to do nothing special. Just share your screen.
    • For Microsoft Teams: Please activate Share the system audio feature.
      • Note: Currently this feature is not available on Mac (as of Sep. 2020).

今後の予定 (Planned Seminars)

2020-10-28(Wed)12:45~

  • Niwa
    • MPNet: Masked and Permuted Pre-training for Language Understanding
      • Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, Tie-Yan Liu (NeurIPS2020)
      • paper, code
  • Okazaki:
    • Joshua Maynez, Shashi Narayan, Bernd Bohnet, Ryan McDonald. 2020. On Faithfulness and Factuality in Abstractive Summarization. ACL. paper data
      • 用語説明
        • hallucination: 幻覚（原文に含まれない情報が要約に含まれること）
        • faithfulness: hallucinationを全く含まない要約
        • factuality: faithfulness + hallucinationを含むものの事実に基づく要約
      • まとめ
        • 抽象型の要約器はどのくらいhallucinationを起こしているのか
          • 70%の短文要約でhallucinationが発生
        • hallucinationが起こるとき、入力の情報を間違えて出力してしまう（intrinsic hallucination）のか、入力に書かれていな情報を出力してしまう（extrinsic hallucination）のか？
          • 大部分は入力に書かれていない情報を出力してしまうこと（extrinsic hallucination）によって起こり、そのうちの90%は事実に関する間違いを含む
        • 湧き出しを定量化する自動評価尺度はあるのか？
          • 要約の忠実性・事実性に関して、ROUGEとBERTScoreは人間の判断との相関が低い
          • 含意関係に基づく評価尺度の方が人間の判断との相関が高い

2020-11-04(Wed)12:45~

  • Ueki
  • Koyama

2020-11-11(Wed)12:45~

  • Mizuki
    • VOITA, Elena; TITOV, Ivan. Information-Theoretic Probing with Minimum Description Length. To Appear: EMNLP 2020. 2020.
    • paper, code, blog post, presentation
  • takase

2020-11-18(Wed)12:45~

  • maruyama
  • iida

    2020-11-25(Wed)12:45~

  • taniguchi
  • niwa

2020-12-2(Wed)12:45~

  • miyazaki
  • wahira

2020-12-9(Wed)12:45~

  • nakamura
  • hiraoka

    2020-12-16(Wed)12:45~

  • nobori
  • wiem

    2020-12-23(Wed)12:45~

  • wahira
  • yang

過去の記録 (Past Seminars)

2020-04-15(Wed) 16:50~

  • Hiraoka
    • Using Similarity Measures to Select Pretraining Data for NER
      • Xiang Dai, Sarvnaz Karimi, Ben Hachey, Cecile Paris (NAACL2019)
      • paper, slides
  • Nobori
    • Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets
      • Mor Geva, Yoav Goldberg, Jonathan Berant (EMNLP2019)
      • paper, video

2020-04-22(Wed) 16:50~

  • Niwa
    • Unified Language Model Pre-training for Natural Language Understanding and Generation
      • Li Dong, Nan Yang et al.(NeurIPS2019)
      • paper, slide
  • Wiem
    • Commonsense Knowledge Mining from Pretrained Models
    • Joshua Feldman, Joe Davison, Alexander M. Rush (EMNLP2019)
    • paper, video, summary

      2020-04-29(Wed) 17:15~

  • National Holiday

2020-05-06(Wed) 17:15~

  • Ma
    • Knowledge Enhanced Contextual Word Representations
    • Matthew E. Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, Noah A. Smith
    • paper, slides
  • Takase: Nikita Kitaev, Lukasz Kaiser, Anselm Levskaya. Reformer: The Efficient Transformer. ICLR 2020. paper, slide, author presentation.

2020-05-13(Wed) 17:15~

  • Ueki:
    • Kevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. Manning.
    • ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators ICLR2020 paper, video,slide
  • Nakamura
    • Zuchao Li, Rui Wang, Kehai Chen, Masso Utiyama, Eiichiro Sumita, Zhuosheng Zhang, Hai Zhao. Data-dependent Gaussian Prior Objective for Language Generation. ICLR 2020
    • video, paper

2020-05-20(Wed) 17:15~

  • Wiem
    • Rewarding Coreference Resolvers for Being Consistent with World Knowledge
    • Rahul Aralikatte, Heather Lent, Ana Valeria Gonzalez, Daniel Hershcovich, Chen Qiu, Anders Sandholm, Michael Ringaard, Anders Søgaard (EMNLP-IJCNLP 2019)
    • paper, slides
  • Hiraoka
    • Sweta Agrawal, Marine Carpuat. EMNLP 2019
    • Controlling Text Complexity in Neural Machine Translation
    • video, paper

2020-05-27(Wed) 17:15~

  • Koyama
    • Neural Grammatical Error Correction Systems with Unsupervised Pre-training on Synthetic Data
      • Roman Grundkiewicz Marcin Junczys-Dowmunt Kenneth Heafield (ACL 2019)
      • paper slide
  • Niwa
    • Entity Recognition at First Sight: Improving NER with Eye Movement Information
      • Nora Hollenstein and Ce Zhang(NAACL 2019)
      • paper, video

2020-06-03(Wed) 17:15~

  • Yang : Image Captioning: Transforming Objects into Words Simao Herdade, Armin Kappeler, Kofi Boakye, Joao Soares (https://arxiv.org/abs/1906.05963) Image_captioning_trsnforming_objects_into_words.pdf (13.6 MB)

  • Ueki

    • A Study of Non-autoregressive Model for Sequence Generation slide paper

    • Yi Ren, Jinglin Liu, Xu Tan, Zhou Zhao, Sheng Zhao, Tie-Yan Liu(ACL2020)

2020-06-10(Wed) 17:15~

  • Mizuki
    • Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi. WINOGRANDE: An Adversarial Winograd Schema Challenge at Scale. In: AAAI 2020. 2020.
    • paper, slide
  • Ma
    • Wenhan Xiong, Jingfei Du, William Yang Wang, Veselin Stoyanov. Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model. In ICLR 2020.
    • paper, video, slides

2020-06-17(Wed) 17:15~

  • Nobori: Jose Camacho-Collados, Luis Espinosa-Anke and Steven Schockaert. Relational Word Embeddings (ACL 2019) paper, slide

  • Sasazawa: Xingxing Zhang, Furu Wei and Ming Zhou. HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization. (ACL 2019) paper viedo

2020-06-24(Wed) 17:15~

  • Okazaki: Yuntian Deng and Alexander M. Rush. 2020. Cascaded Text Generation with Markov Transformers. arXiv:2006.01112. Slides (PDF), Slides (pptx)
  • Iida: Wei-Cheng Chang, Felix X. Yu, Yin-Wen Chang, Yiming Yang, Sanjiv Kumar ICLR2020. Pre-training Tasks for Embedding-based Large-scale Retrieval
  • Nakamura
    • On the Word Alignment from Neural Machine Translation
      • Xintong Li, Guanlin Li, Lemao Liu, Max Meng, Shuming Shi(ACL2019)
      • paper, slide
  • Maruyama
    • Unifying Human and Statistical Evaluation for Natural Language Generation (NAACL 2019)
      • Tatsunori B. Hashimoto, Hugh Zhang, Percy Liang
      • video, paper

2020-07-08(Wed) 17:15~

  • Sasazawa: Ming Zhong, Pengfei Liu, Yiran Chen, Danqing Wang, Xipeng Qiu, Xuanjing Huang. Extractive Summarization as Text Matching. acl2020. paper slide

  • Taniguchi: Tao Gui, Qi Zhang, Jingjing Gong, Minlong Peng, Di Liang, Keyu Ding, Xuanjing Huang. Transferring from Formal Newswire Domain with Hypernet for Twitter POS Tagging. EMNLP2018. paper, slide

2020-07-15(Wed) 17:15~

  • Nobori

  • Takase (emergency presentation): Cohen, E. and Beck, C., Empirical Analysis of Beam Search Performance Degradation in Neural Sequence Models. ICML 2019. Paper, (annotated) author slide

  • Miyazaki: Gonçalo M. Correia, Vlad Niculae, André F. T. Martins, “Adaptively Sparse Transformers”, EMNLP-IJCNLP 2019 paper video

2020-07-22(Wed) 17:15~

  • Ma
    • Generalizing Natural Language Analysis through Span-relation Representations
      • Zhengbao Jiang, Wei Xu, Jun Araki, Graham Neubig (ACL2020)
      • paper, slides
  • Nobori
    • Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus
      • Hongyu Gong, Suma Bhat, Lingfei Wu, Jinjun Xiong, Wen-mei Hwu(NAACL 2019)
      • paper, video

2020-07-29(Wed) 17:15~

  • Koyama
    • Masked Language Model Scoring
      • Julian Salazar, Davis Liang, Toan Q. Nguyen, Katrin Kirchhoff (ACL2020)
      • paper slide
  • Ueki
    • Norm-Based Curriculum Learning for Neural Machine Translation
      • Xuebo Liu, Houtim Lai, Derek F. Wong, Lidia S. Chao (ACL 2020)
      • paper, slide

2020-08-05(Wed) 17:15~

  • Wiem
    • Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
      • ACL2020 Best Overall Paper
      • Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, Sameer Singh
      • paper, slides
  • Niwa
    • Improved Natural Language Generation via Loss Truncation
      • Daniel Kang and Tatsunori B. Hashimoto (ACL2020)
      • paper, video, supplemental material

(Summer vacation)

2020-09-02(Wed) 17:15~

  • Iida
    • Gino Brunner, Yang Liu, Dami´an Pascual, Oliver Richter, Massimiliano Ciaramita, Roger Wattenhofer: ON IDENTIFIABILITY IN TRANSFORMERS. In ICLR2020, 2020
    • paper, video, slides
  • Mizuki
    • BRAŽINSKAS, Arthur; LAPATA, Mirella; TITOV, Ivan. Unsupervised Opinion Summarization as Copycat-Review Generation. In: ACL 2020. 2020.
    • paper, video (requires conference account), summary (in Japanese)

2020-09-09(Wed)17:15~

  • Maruyama
    • e-SNLI: Natural Language Inference with Natural Language Explanations
      • Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. NeurIPS 2018
      • paper, slide
  • Hiraoka
    • Dice Loss for Data-imbalanced NLP Tasks
      • videoがなかったのでまた今度（最先端NLPで読まれる気もする）
    • Human Attention Maps for Text Classification: Do Humans and Neural Networks Focus on the Same Words?
      • Cansu Sen, Thomas Hartvigsen, Biao Yin, Xiangnan Kong, Elke Rundensteiner. ACL2020.
      • paper, video

2020-09-16(Wed)17:15~

  • Taniguchi
    • Exploring Contextual Word-level Style Relevance for Unsupervised Style Transfer
      • Chulun Zhou, Liangyu Chen, Jiachen Liu, Xinyan Xiao, Jinsong Su, Sheng Guo, Hua Wu, ACL2020
      • paper slide
  • Sasazawa:
    • Don’t Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference
      • Yonatan Belinkov, Adam Poliak, Stuart Shieber, Benjamin Van Durme, Alexander Rush.
      • ACL 2019
      • paper video

2020-09-23(Wed)17:15~

  • Miyazaki
    • Learning Source Phrase Representations for Neural Machine Translation
      • Hongfei Xu, Josef van Genabith, Deyi Xiong, Qiuhui Liu, Jingyi Zhang, ACL 2020
      • paper slides
  • Nakamura
    • Unsupervised Paraphrasing by Simulated Annealing
      • Xianggen Liu, Lili Mou, Fandong Meng, Hao Zhou, Jie Zhou, and Sen Song. ACL2020
      • paper, video

2020-09-30(Wed)17:15~

  • Sangwhan
    • In Neural Machine Translation, What Does Transfer Learning Transfer?
      • Alham Fikri Aji, Nikolay Bogoychev, Kenneth Heafield, Rico Sennrich (ACL2020)
      • Paper, Slides
  • Wahira

2020-10-07(Wed)12:45~

  • Takase
    • Improving Transformer Models by Reordering their Sublayers
      • Ofir Press, Noah A. Smith, Omer Levy. ACL 2020
      • Paper, Slide
  • Wiem
    • Contrastive Self-Supervised Learning for Commonsense Reasoning
    • Tassilo Klein, Moin Nabi (ACL2020)
    • paper video

2020-10-14(Wed)12:45~

  • Hiraoka
  • Ma
    • Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks
      • Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A. Smith (ACL2020)
      • paper, video

2020-10-21(Wed)12:45~

  • Nobori
    • CoLAKE: Contextualized Language and Knowledge Embedding *Tianxiang Sun, Yunfan Shao, Xipeng Qiu, Qipeng Guo, Yaru Hu, Xuanjing Huang, Zheng Zhang(coling 2020) *Paper,Slide
  • Sangwhan
    • Fast and Accurate Deep Bidirectional Language Representationsfor Unsupervised Learning
      • Joongbo Shin, Yoonhyung Lee, Seunghyun Yoon, Kyomin Jung (ACL 2020)
      • Paper, Video