Paper reading (2021)

このセミナーについて (About this seminar)

担当者が見つけた面白い研究を紹介するセミナーです．以下の内容で構成されます． In this seminar, presenters share interesting studies that they found. The seminar consists of the following elements.

  1. 論文紹介 (Paper presentation): 面白いと思う論文（基本的にLong Paper）を選んで，その内容を著者の代わりになったつもりで発表する． A presenter reads a paper (long paper is strongly preffered) that he or she finds interesting, and explains the content as if he or she was an author of the paper.
  2. 著者プレゼンの上映と解説 (Watching talk videos): 担当者は面白いと思う研究の発表動画を選び，その論文の内容を把握する．担当者の補足説明を聞きながら，参加者みんなで発表動画を鑑賞する． Finding an interesting video presentation, a presenter reads and understands the content of the paper. Attendees watch the presentation video of the author of the paper. The presenter is expected to explain supplementary information about the research.

前者 (1) はプレゼンテーションの練習を兼ねています．後者 (2) は「よい」プレゼンテーションを鑑賞しながら，英語でのプレゼンテーションやディスカッションに慣れることを狙っています． The former (1) aims at practicing presentations. The latter (2) aims at improving listening and discussion skills in English as we enjoy good presentations.

発表者は一人目は (1) を，二人目は (1) か (2) のどちらかを担当します． The 1st presentator take (1). The 2nd one can select (1) or (2).

座長(chair)は、セミナーの司会進行をおこないます。 chair person host the seminar. 座長の仕事を参考にして，円滑に議論が進むように心がけてください． Please contribute to active discussion refering to Chair’s Job.

発表時間 (Presentation Time)

  • Presentation: 15 ~ 20 minnutes
  • QA: 5 ~ 10 minutes

日時 (Date and time)

  • 1Q: 12:30~ (Wed)
  • 2Q: 18:05~ (Wed)

参加者 (Attendee)

  • 全員

:exclamation: 発表を登録するときは，著者名，発表年，タイトル，会議名／ジャーナル名，（巻，号，ページ番号など）を必ず記入してください． :exclamation: Please include author names, publication year, title, conference/journal name, (volume and page numbers) when you add an entry for presentation.

Tips

  • リモートミーティングでビデオプレゼンテーションを行う場合は，画面およびオーディオを共有してください．
  • When you stream the video presentation via remote meeting, please share the both screen and system audio.
    • For Zoom: You don’t have to do nothing special. Just share your screen.
    • For Microsoft Teams: Please activate Share the system audio feature.

今後の予定 (Planned Seminars)

2021-10-29(Fri) 12:30~

  • presenter
    • Liu Yang
    • Niwa
      • Towards Robustness Against Natural Language Word Substitutions (ICLR 2021)
        • Xinshuai Dong, Anh Tuan Luu, Rongrong Ji, Hong Liu.
        • paper, video
  • chair
    • Maeda

2021-11-05(Fri) 12:30~

  • presenter
    • Maruyama
    • Kuo
  • chair
    • Loem

2021-11-12(Fri) 12:30~

  • presenter
    • Hiraoka
    • Sangwhan
  • chair
    • Ishikawa

2021-11-19(Fri) 12:30~

  • presenter
    • Erick
    • Koyama
  • chair
    • Ueki

2021-11-26(Fri) 12:30~

  • presenter
    • Iida
    • Ueki
  • chair
    • Maruyama

2021-12-03(Fri) 12:30~

  • presenter
    • Mizuki
      • Muhammad Khalifa, Hady Elsahar, Marc Dymetman. A Distributional Approach to Controlled Text Generation. ICLR 2021. 2021.
      • paper, slide in Japanese (at SNLP 2021)
    • Muraoka
  • chair
    • Koyama

      過去の記録 (Past Seminars)

2021-04-14(Wed) 12:30~

  • presenter
    • Koyama
      • Zhaohong Wan, et al. Improving Grammatical Error Correction with Data Augmentation by Editing Latent Representation (COLING 2020)
      • paper, slide
    • Mizuki
      • Tom B. Brown, et al. Language Models are Few-Shot Learners. In: NeurIPS 2020. 2020. (best paper)
      • paper, video, supplementary material
      • Those who are interested in the theoretical background of the effectiveness, I recommend following paper.
        • Saunshi. N, et al. A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks. To Appear: ICLR 2021. 2020. paper, video

2021-04-21(Wed) 12:30~

  • presenter
    • Muraoka
      • Candidate 1: Hao Tan, and Mohit Bansal. Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision. (EMNLP2020)
        • paper -> I will introduce it somewhere else in the near future :)
      • Candidate 2: Irwan Bello. LambdaNetworks: Modeling long-range Interactions without Attention. (ICLR2021, spotlight)
    • Yang
      • CapWAP: Captioning with a Purpose
        Adam Fisch, Kenton Lee, Ming-Wei Chang, Jonathan Clark, Regina Barzilay Video, Paper

2021-04-28(Wed) 12:30~

  • presenter
    • Taniguchi
      • Non-Autoregressive Text Generation with Pre-trained Language Models Yixuan Su, Deng Cai, Yan Wang, David Vandyke, Simon Baker, Piji Li, Nigel Collier EACL2021
      • paper, slide
    • Niwa
      • How Can We Know What Language Models Know? (TACL 2020)
        • Zhengbao Jiang, Frank F. Xu, Jun Araki, Graham Neubig
        • paper, video

2021-05-05(Wed)

  • national holiday

2021-05-12(Wed) 12:30~

  • presenter
    • Maruyama
      • Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs. In NeurIPS 2020. Slide Paper
    • Liu
      • Martin Schmitt, Sahand Sharifzadeh, Volker Tresp And Hinrich Schütze. An Unsupervised Joint System for Text Generation from Knowledge Graphs and Semantic Parsing. In EMNLP 2020. Video. Paper.

2021-05-19(Wed) 12:30~

  • presenter
    • Wiem
      • UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark [AAAI2021] paper, slides
    • Kuo
      • Joint Constrained Learning for Event-Event Relation Extraction. In EMNLP2020. paper, video

2021-05-26(Wed) 12:30~

  • presenter

    ### 2021-06-02(Wed) 12:30~

  • Presenter:
    • Erick:
      • Nikolay Malkin et all. GPT Perdetry Test: Generating new meanings for new words. In: NAACL 2021.
      • Paper, Slide
  • Sangwhan
    • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
    • Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby
    • In ICLR2021.
    • Paper Slides

2021-06-09(Wed) 12:30~

  • Iida
    • Yixing Fan, Jiafeng Guo, Xinyu Ma, Ruqing Zhang, Yanyan Lan, and Xueqi Cheng A Linguistic Study on Relevance Modeling in Information Retrieval WWW2021
    • slides, paper
  • Koyama
    • Dhruv Kumar, Lili Mou, Lukasz Golab, Olga Vechtomova. Iterative Edit-Based Unsupervised Sentence Simplification. ACL 2020.
    • paper, slide

2021-06-16(Wed) 18:05~

  • presenter
    • Mizuki
      • SAUNSHI, Nikunj; MALLADI, Sadhika; ARORA, Sanjeev. A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks. In: ICLR 2021. 2021.
      • paper, slide, video (for the interested readers)
    • Muraoka
      • Hao Tan, and Mohit Bansal. Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision. (EMNLP2020)
      • video, paper
  • chair
    • Wiem

2021-06-23(Wed) 18:05~

  • presenter
    • Kaneko
      • Cunxiao Du, Zhaopeng Tu, Jing Jiang. Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation (ICML 2021)
      • paper, slides
    • Yoshikawa
      • Evaluation Examples Are Not Equally Informative: How Should That Change NLP Leaderboards?
      • Pedro Rodriguez, Joe Barrow, Alexander Hoyle, John P. Lalor, Robin Jia and Jordan Boyd-Graber (ACL-IJCNLP2021)
      • paper, slides
  • chair
    • Koyama

### 2021-06-30(Wed) 18:05~

  • presenter
    • Takase
      • Are Pre-trained Convolutions Better than Pre-trained Transformers?
        • Yi Tay, Mostafa Dehghani, Jai Gupta, Dara Bahri, Vamsi Aribandi, Zhen Qin, Donald Metzler. ACL 2021.
        • paper, slide
    • Vijay
      • Hooks in the Headline: Learning to Generate Headlines with Controlled Styles
        • Di Jin, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, Peter Szolovits. ACL 2020
        • paper, slide
  • chair
    • Ueki

2021-07-07(Wed) 18:05~

  • presenter
    • Liu
      • CoDA: Contrast-enhanced and Diverse-promoting Data Augmentation for Natural Language Understanding. (ICLR 2021)
        • Yanru Qu, Dinghan Shen, Yelong Shen, Sandra Sajeev, Jiawei Han, Weizhu Chen
        • Paper. Slide
    • Yang
  • chair
    • Kuo

2021-07-14(Wed) 18:05~

  • presenter
    • Kuo
      • Automatic Detection of Generated Text isEasiest when Humans are Fooled (ACL 2020)
        • Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, Douglas Eck
        • Paper, Slides(revised)
    • Taniguchi
      • ZeroShotCeres: Zero-Shot Relation Extraction from Semi-Structured Webpages (ACL 2020)
        • Colin Lockard, Prashant Shiralkar, Xin Luna Dong, Hannaneh Hajishirzi
        • Paper, Video
  • chair
    • Maruyama

2021-07-21(Wed) 18:05~

  • presenter
    • Niwa
      • Glancing Transformer for Non-Autoregressive Neural Machine Translation (ACL2021)
        • Lihua Qian, Hao Zhou, Yu Bao, Mingxuan Wang, Lin Qiu, Weinan Zhang, Yong Yu, Lei Li
      • paper, paper (one column ver.), slide
    • Maruyama
      • Ziheng Wang, Jeremy Wohlwend, Tao Lei. Structured Pruning of Large Language Models (emnlp2020)
      • paper, video
  • chair
    • Taniguchi

2021-07-28(Wed) 18:05~

  • presenter
    • Maeda
    • Ueki
      • Meta Back-Translation (ICLR2021)
      • Hieu Pham,, Xinyi Wang, Yiming Yang, Graham Neubig
      • paper, slide
  • chair
    • Toyama

2021-08-04(Wed) 18:05~

2021-09-01(Wed) 18:05~

2021-09-08(Wed) 18:05~

  • presenter
    • Ma
      • Examining the Inductive Bias of Neural Language Models with Artificial Languages (ACL2021)
      • Jennifer C. White, Ryan Cotterell
      • paper
      • slides
    • Sangwhan
      • HTLM: Hyper-Text Pre-Training and Prompting of Language Models
      • Armen Aghajanyan, Dmytro Okhonko, Mike Lewis, Mandar Joshi, Hu Xu, Gargi Ghosh, Luke Zettlemoyer
      • paper
      • slides
  • chair
    • Loem

      2021-09-15(Wed) 18:05~

  • presenter
    • Erick
      • UnNatural Language Inference.
      • Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau and Adina Williams. In ACL-IJCNLP (2021)
      • paper, slides
    • Takase
      • Efficient Content-Based Sparse Attention with Routing Transformers
        • Aurko Roy, Mohammad Saffar, Ashish Vaswani, David Grangier. TACL 2021.
        • paper, slide
  • chair
    • Ueki

2021-09-22(Wed) 18:05~

  • presenter
    • Maeda
      • Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
      • Conference : ACL2021
      • Author : Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, Maosong Sun
      • [slides] [paper]
    • Wiem
  • chair
    • Koyama

2021-09-29(Wed) 18:05~

  • presenter
    • Toyama
      • Armen Aghajanyan et al., Intrinsic Dimensionality Explains the effectiveness of Language Model Fine-Tuning (ACL2021)
      • paper, slide
    • Mizuki
  • chair
    • Maruyama

2021-10-08(Fri) 12:30~

2021-10-15(Fri) 12:30~

  • presenter
    • Taniguchi
      • Daniel Grießhaber, Johannes Maucher
      • Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning
      • COLING 2020
      • paper, slide
    • Yang Liu (video presentation)
  • chair
    • Kuo