論文輪読（2019年度）

このセミナーについて (About this seminar)

担当者が見つけた面白い研究を紹介するセミナーです．以下の内容で構成されます． In this seminar, presenters share interesting studies that they found. The seminar consists of the following elements.

  1. 論文紹介 (Paper presentation): 面白いと思う論文を選んで，その内容を著者の代わりになったつもりで発表する． A presenter reads a paper that he or she finds interesting, and explains the content as if he or she was an author of the paper.
  2. 著者プレゼンの上映と解説 (Watching talk videos): 担当者は面白いと思う研究の発表動画を選び，その論文の内容を把握する．担当者の補足説明を聞きながら，参加者みんなで発表動画を鑑賞する． Finding an interesting video presentation, a presenter reads and understands the content of the paper. Attendees watch the presentation video of the author of the paper. The presenter is expected to explain supplementary information about the research.

前者 (1) はプレゼンテーションの練習を兼ねています．後者 (2) は「よい」プレゼンテーションを鑑賞しながら，英語でのプレゼンテーションやディスカッションに慣れることを狙っています． The former (1) aims at practicing presentations. The latter (2) aims at improving listening and discussion skills in English as we enjoy good presentations.

日時 (Date and time)

  • Wednesday 16:50~

参加者 (Attendee)

  • 全員

今後の予定 (Planned schedule)

2020-02-26 (Wed) 16:50

Canceled due to the Titech entrance examination

2020-03-04 (Wed) 16:50

  • Takase: Jesse Dodge, Suchin Gururangan, Dallas Card, Roy Schwartz, Noah A. Smith. Show Your Work: Improved Reporting of Experimental Results. EMNLP 2019. paper, slide

  • Koyama : EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing. ACL2019. paper video

2020-03-11 (Wed) 16:50

  • Okazaki: Junxian He, Jiatao Gu, Jiajun Shen, Marc’Aurelio Ranzato. 2020. Revisiting Self-Training for Neural Sequence Generation. to appear in ICLR 2020. (paper) (slides (648.0 kB))

  • Okazaki: Simon Peyton Jones (Microsoft Research Cambridge). How to Write a Great Research Paper. (materials)

過去の記録 (Past seminars)

2019-04-03 (Wed) 16:50

  • sasazawa:A Deep Neural Network Sentence Level Classification Method with Context Information.Xingyi Song, Johann Petrak, Angus Roberts.EMNLP 2018:(paper,slide)
  • matsumaru: SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference. Rowan Zellers, Yonatan Bisk, Roy Schwartz, Yejin Choi. EMNLP 2018. (paper, video)

2019-04-10 (Wed) 16:50

  • ema: Linguistically-Informed Self-Attention for Semantic Role Labeling. Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, Andrew McCallum. EMNLP 2018. [slides] [paper] [video]

2019-04-17 (Wed) 16:50

  • Erick: Attention Based Natural Language Grounding by Navigating Virtual Environment. Akilesh B, Abhishek Sinha, Mausoom Sarkar, Balaji Krishnamurthy [paper] [slides]
  • Okazaki: Jie Lei, Licheng Yu, Mohit Bansal, and Tamara L. Berg. TVQA: Localized, Compositional Video Question Answering. EMNLP 2018. [paper] [video]

2019-04-24 (Wed) 16:50

  • mizuki: KUMAR, Sachin; TSVETKOV, Yulia. Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs. In: ICLR 2019. 2019. paper, slides
  • yang: Xianda Zhou, William Yang Wang. Mojitalk: Generating emotional responses at scale. ACL 2018 video, Paper

2019-05-08 (Wed) 16:50

  • Wan: Nan Wang, Xibin Zhao, Yu Jiang, Yue Gao, Iterative Metric Learning for Imbalance Data Classification IJCAI2018 slide,paper
  • niwa : Rashkin, Hannah, et al. “Modeling naive psychology of characters in simple commonsense stories.” ACL2018 paper, video

2019-05-15 (Wed) 16:50

  • okazaki
  • ema: Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter. Lijun Wu, Xu Tan, Di He, Fei Tian, Tao Qin, Jianhuang Lai, Tie-Yan Liu. EMNLP 2018. [paper][video]

2019-05-22 (Wed) 16:50

  • yang: Text Understanding from Scratch. Xiang Zhang, Yann LeCun paper , slides
  • sasazawa: Rumor Detection on Twitter with Tree-structured Recursive Neural Networks.Jing Ma, Wei Gao, Kam-Fai Wong.ACL 2018. (video, paper)

2019-05-29 (Wed) 16:50

Cancelled due to the talk in Tokyo univ.

2019-06-05 (Wed) 16:50

  • matsumaru: Attention is not Explanation. Sarthak Jain and Byron C. Wallace. NAACL 2019. paper, slides
  • Erick: Localizing Moments in Video with Temporal Language. Hendricks et al 2018. EMNLP 2018 paper, talk

2019-06-12 (Wed) 16:50

  • ema: Choosing Transfer Languages for Cross-Lingual Learning. Lin et al. ACL 2019. [paper] [slides]
  • Takase Mizuki: Discovering Discrete Latent Topics with Neural Variational Inference. Yishu Miao et al. ICML 2017. paper, talk, summary

2019-06-19 (Wed) 16:50

  • Nakamura: A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss, [paper](https://aclweb.org/anthology/P18-1013), [talk](https://vimeo.com/showcase/5391494/video/285800568)

2019-06-26 (Wed) 16:50

  • niwa: Text Generation with Exemplar-based Adaptive Decoding. Hao Peng et al. NAACL2019. paper, slides
  • takase: Alexey Romanov, Anna Rumshisky, Anna Rogers, David Donahue. Adversarial Decomposition of Text Representation. NAACL 2019. paper, slide
  • Sangwhan: Simplified Abugidas. Ding et al. ACL 2018. paper, talk

2019-07-03 (Wed) 16:50

  • sasazawa: Han Guo,Ramakanth Pasunuru,Mohit Bansal. Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation .ACL 2018 paper slide
  • Hiraoka: Learning Neural Templates for Text Generation. EMNLP2018. paper, video

2019-07-10 (Wed) 16:50

  • Erick: Dense Procedure Captioning in Narrated Instructional Videos. ACL 2019 paper, pdf

2019-07-17 (Wed) 16:50

  • Hiraoka: Misspelling Oblivious Word Embeddings, NAACL 2019, paper, slides
  • Ueki:Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers(ACL2018) paper,video

2019-07-24 (Wed) 16:50

2019-07-31 (Wed) 16:50

  • Nakamura: Jointly Extracting and Compressing Documents with Summary State Representations, NACCL2019 paper, pdf
  • Nobori:Did the Model Understand the Question? paper, video

2019-08-07 (Wed) 16:50

  • Nobori : Asking the Right Question: Inferring Advice-Seeking Intentions from Personal Narratives (paper,slide)
  • Koyama:Fluency Boost Learning and Inference for Neural Grammatical Error Correction (paper,video)
    • additional experiments are refered to in
      • Reaching Human-Level Performance in Automatic Grammatical Error Correction: An Empirical Study (paper)

2019-09-04 (Wed) 16:50

  • Koyama Corpora Generation for Grammatical Error Correction NAACL 2019 paper slide

  • Mizuki Takase Erick: Phrase-Based & Neural Unsupervised Statistical Machine Translation Emnlp 2018 video, paper

2019-09-11 (Wed) 16:50

  • Fukushima: Ankur Bapna et al., Non-Parametric Adaptation for Neural Machine Translation, NAACL 2019 paper, slide

  • Okazaki Takase: Jiaji Huang, Yi Li, Wei Ping, Liang Huang. Large Margin Neural Language Model. EMNLP 2018. paper, video

2019-09-18 (Wed) 16:50

  • Ueki:Consistency by Agreement in Zero-shot Neural Machine Translation NAACL 2019: paper,slide

  • Matsumaru: Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, Tie-Yan Liu. MASS: Masked Sequence to Sequence Pre-training for Language Generation. ICML 2019. paper, video, slide by author

2019-09-25 (Wed) 16:50

  • Mizuki: Fabio Petroni et al. Language Models as Knowledge Bases?. To appear: EMNLP 2019. 2019. paper, slide

  • Erick Okazaki: Xiang Li, Luke Vilnis, Dongxu Zhang, Michael Boratko, Andrew McCallum. Smoothing the Geometry of Probabilistic Box Embeddings. ICLR 2019. paper, video

2019-10-02 (Wed) 16:50

  • Okazaki: Yau-Shian Wang, Hung-Yi Lee, Yun-Nung Chen. Tree Transformer: Integrating Tree Structures into Self-Attention. to appear in EMNLP 2019. paper slides (3.5 MB)

  • Yang: Caglayan, Ozan, Pranava Madhyastha, Lucia Specia, and Loïc Barrault. “Probing the Need for Visual Context in Multimodal Machine Translation.” arXiv preprint arXiv:1903.08678 (2019).paper, video, starts from 39:18.

2019-10-09 (Wed) 16:50

  • Hiraoka
    • ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
    • Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, Radu Soricut. ICLR2020 under-review.
    • paper, slides
  • Niwa : Reinald Kim Amplayo, Seonjae Lim, Seung-won Hwang. Entity Commonsense Representation for Neural Abstractive Summarization. NAACL 2018. paper, video, github

2019-10-16 (Wed) 16:50

  • Sangwhan: Extreme Language Model Compression with Optimal Subwords and Shared Projections. Sanqiang Zhao, Raghav Gupta, Yang Song, Denny Zhou. ICLR2020 under-review. Paper, Slides An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models. Chronopoulou A. et al. Proceedings of NAACL-HLT 2019, pages 2089–2095. Paper, Slides (I’ll revisit this one of these days, don’t worry.)
  • Fukushima: Jingjing Xu et al,. Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach. ACL 2019. paper, video, presentation

2019-10-23 (Wed) 16:50

  • Erick: Mix-review: Alleviate Forgetting in the Pretrain-Finetune Framework for Neural Language Generation Models. In review for ICLR 2020. paper, slides.

  • Nakamura: Tianxiao Shen · Myle Ott · Michael Auli · Marc’Aurelio Ranzato, Mixture Models for Diverse Machine Translation: Tricks of the Trade, ICML 2019, paper, video

2019-10-30 (Wed) 16:50

  • Matsumaru: Neural Text Summarization: A Critical Evaluation. Wojciech Kryściński, Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, Richard Socher. To appear in EMNLP 2019. paper, slides, summary (ja)

  • Nobori: Robert McHardy, Heike Adel and Roman Klinger, Adversarial Training for Satire Detection: Controlling for Confounding Variables,NAACL 2019. paper, video

2019-11-06 (Wed) 16:50

  • Koyama slide
    • Levenshtein Transformer. paper
      • Jiatao Gu, Changhan Wang, Jake Zhao. NeurIPS 2019.
  • Sasazawa: Di Jin, Peter Szolovits. Hierarchical Neural Networks for Sequential Sentence Classification in Medical Scientific Abstracts. EMNLP 2018. paper, video

2019-11-13 (Wed) 16:50

  • Fukushima
  • Mizuki: FRANKLE, Jonathan; CARBIN, Michael. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. In: ICLR 2019. 2018. paper, video
    • ICLR 2019 Best Paper Awards

2019-11-20 (Wed) 16:50

  • Fukushima
    • Kenton Lee, Luheng He, Mike Lewis, and Luke S. Zettlemoyer. End-to-end neural coreference resolution. EMNLP 2017. paper, slide
  • Koyama
    • Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Data.
      • paper, video
      • Wei Zhao, Liang Wang, Kewei Shen, Ruoyu Jia, Jingming Liu. NAACL 2019.
      • More detailed explanation about how to noise the monolingual data & application for text summrisation is in this paper.

2019-11-27 (Wed) 16:50

  • Nakamura: Generating Sentences from Disentangled Syntactic and Semantic Spaces
    • Yu Bao, Hao Zhou, Shujian Huang, Lei Li, Lili Mou, Olga Vechtomova, Xinyu Dai, Jiajun Chen. ACL 2019
    • slide, paper
  • Ueki
    • Selective Attention for Context-aware Neural Machine Translation
      • paper, video
      • Sameen Maruf,Andre F. T. Martins ,Gholamreza Haffari
      • NACCL 2019

### 2019-12-04 (Wed) 16:50

  • Takase: Yong Cheng, Lu Jiang, Wolfgang Macherey. Robust Neural Machine Translation with Doubly Adversarial Inputs. ACL 2019. paper, slide.
  • Ma: Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf, Hannaneh Hajishirzi. A General Framework for Information Extraction using Dynamic Span Graphs. NAACL 2019:(paper,video)

2019-12-11 (Wed) 16:50

  • Yang: Zhao, S., Sharma, P., Levinboim, T., & Soricut, R. (2019). Informative Image Captioning with External Sources of Information paper, Informative Image Captioning with External Sources of Information.pdf (9.4 MB)

  • Wiem: Linguistic Knowledge and Transferability of Contextual Representations. Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, Noah A. Smith (NAACL 2019) paper video

2019-12-18 (Wed) 16:50

  • Niwa: Allen, Carl, Ivana Balazevic, and Timothy Hospedales. “What the vec? towards probabilistically grounded embeddings.” Advances in Neural Information Processing Systems. 2019.paper, slide

  • Takase: Hao Peng, Roy Schwartz, Sam Thomson, and Noah A. Smith. Rational Recurrences. EMNLP 2018. paper, video

    • Summary: This paper argues that some RNNs can be interpreted as weighted finite state automata (WFSAs) but the authors suppose that widely used RNNs such as Elman net, GRU, and LSTM cannot.

2019-12-25 (Wed) 16:50

  • Sasazawa: Binxuan Huang, Kathleen M. Carley. A Hierarchical Location Prediction Neural Network for Twitter User Geolocation. EMNLP 2019. paper slide summary(en)
  • Sangwhan: An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models. Chronopoulou A. et al. Proceedings of NAACL-HLT 2019, pages 2089–2095. Paper, Video (As promised earlier, revisiting.)

2020-01-08 (Wed) 16:50

  • Nobori Yue Wang , Jing Li, Irwin King, Michael R. Lyu, Shuming Shi Microblog Hashtag Generation via Encoding Conversation Contexts. NAACL2019. paper, slide

  • Hiraoka Abhijit Mahabal, Jason Baldridge, Burcu Karagol Ayan, Vincent Perot, Dan Roth Text Classification with Few Examples using Controlled Generalization. NAACL2019. paper, video

2020-01-15 (Wed) 16:50

Canceled due to JNLP’s deadline

2020-01-22 (Wed) 16:50

  • Ma: Alexei Baevski, Sergey Edunov, Yinhan Liu, Luke Zettlemoyer, Michael Auli. Cloze-driven Pretraining of Self-attention Networks. In EMNLP-IJCNLP 2019. paper, slides

  • Mizuki: BARRON, Jonathan T. A general and adaptive robust loss function. In: CVPR 2019. paper, video, slides

2020-01-29 (Wed) 16:50

  • Wiem: Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, Yejin Choi. COMET : Commonsense Transformers for Automatic Knowledge Graph Construction paper, slides Note: Please follow on the projector because there are animations that do not show on the PDF

  • Fukushima: Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, Ivan Titov. Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned. ACL 2019. paper, video, supplement

    • Note: If you are not familiar with Transformer, please take a brief look at this slide for explaining Transformer.

2020-02-05 (Wed) 16:50

  • Nakamura:
    • Unsupervised Pivot Translation for Distant Languages, Yichong Leng, Xu Tan, Tao Qin, Xiang-Yang Li, Tie-Yan Liu, ACL 2019, paper, slides
  • Yang: Incorporating Emoji Descriptions Improves Tweet Classification Abhishek Singh, Eduardo Blanco and Wei Jin, NAACL 2019, [Video]:https://vimeo.com/354236787 [paper]: https://www.aclweb.org/anthology/N19-1214.pdf

2020-02-12 (Wed) 16:50

  • Sasazawa: Ruizhe Li, Chenghua Lin, Matthew Collinson, Xiao Li, Guanyi Chen. A Dual-Attention Hierarchical Recurrent Neural Network for Dialogue Act Classification. CoNLL 2019. paper slide

  • Erick: Wen Zhang, Yang Feng, Fandong Meng, Di You, Qun Liu. Bridging the Gap between Training and Inference for Neural Machine Translation. ACL 2019 Paper, Video ,Video transcript

2020-02-19 (Wed) 16:50

  • Ueki
    • Ouroboros: On Accelerating Training of Transformer-Based Language Models
    • Qian Yang, Zhouyuan Huo, Wenlin Wang, Heng Huang, Lawrence Carin
    • NeurIPS 2019
    • paper, slide
  • Matsumaru: A Simple Theoretical Model of Importance for Summarization. Maxime Peyrard. ACL 2019. paper, video