Winter 2018 CS291A Syllabus

• 01/16 Introduction, logistics, NLP, and deep learning.
• 01/18 Tips for a successful class project
• 01/23 NLP Tasks
• 01/25 Word embeddings
  •  : Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space, Neelakantan et al., EMNLP 2014
  • Arya: Glove: Global Vectors for Word Representation, J Pennington, R Socher, CD Manning - EMNLP, 2014
  •  : AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes, Rothe and Schutze, ACL 2015
• 01/30 Neural network basics (Project proposal due to Grader: Ke Ni < ke00@ucsb.edu> , HW1 out)
  •  : Learning representations by back-propagating errors, Nature, 1986
  • Dan : An overview of gradient descent optimization algorithms, Sebastian Ruder, Arxiv 2016
  • Vivek : Dropout: A simple way to prevent neural networks from overfitting (2014), N. Srivastava et al., JMLR 2014
• 02/01 Recursive Neural Networks
  •  : Semantic Compositionality through Recursive Matrix-Vector Spaces, Socher et al., EMNLP 2012
  •  : Parsing with Compositional Vector Grammars, Socher et al., ACL 2013
  •  : Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank, Socher et al., EMNLP 2013
• 02/06 RNNs
  • Lukas : A Learning Algorithm for Continually Running Fully Recurrent Neural Networks, Ronald J. Williams and David Zipser, 1989
  • Yifu : Recurrent neural network based language model
  •  : Generating Sequences With Recurrent Neural Networks, Alex Graves, 2013 arxiv
• 02/08 LSTMs/GRUs
  • Liu : Long short term memory, S. Hochreiter and J. Schmidhuber, Neural Computation, 1997
  •  : On the Properties of Neural Machine Translation: Encoder–Decoder Approaches, Cho et al., 2014
  •  : Gated Feedback Recurrent Neural Networks, Chung et al., ICML 2015
• 02/13 Sequence-to-sequence models and neural machine translation (HW1 due and HW2 out)
  •  : Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation, Cho et al., EMNLP 2014
  • Yanju : Sequence to Sequence Learning with Neural Networks, Sutskever et al., NIPS 2014
  •  : Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models, Luong and Manning, ACL 2016
• 02/15 Attention mechanisms
  • Jing : NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE, Bahdanau et al., ICLR 2015
  •  : Teaching Machines to Read and Comprehend, NIPS 2015
  •  : End-to-end memory networks, NIPS 2015
• 02/20 Convolutional Neural Networks (Mid-term report due to Grader: Ke Ni <ke00@ucsb.edu>)
  •  : Natural Language Processing (Almost) from Scratch, Collobert et al., JMLR 2011
  • Jiawei : A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification, Zhang and Wallace, Arxiv 2015
  •  : Convolutional Neural Network Architectures for Matching Natural Language Sentences, Hu et al., NIPS 2014
• 02/22 Language and vision
  •  : Show and Tell: A Neural Image Caption Generator, CVPR 2015
  • Xiyou : Deep Visual-Semantic Alignments for Generating Image Descriptions, Andrej Karpathy and Li Fei-Fei, CVPR 2015
  •  : Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books, Zhu et al., ICCV 2015
• 02/27 Deep Reinforcement Learning 1 (HW2 due)
  •  : Deep Reinforcement Learning for Dialogue Generation, Li et al., EMNLP 2016
  •  : Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning, Narasimh et al., EMNLP 2016
  •  : Deep Reinforcement Learning with a Natural Language Action Space, He et al., ACL 2016
• 03/01 Deep Reinforcement Learning 2
  • Trevor : Playing Atari with Deep Reinforcement Learning, Mnih et al., NIPS workshop 2013
  •  : Continuous control with deep reinforcement learning, Lillicrap et al, ICLR 2016
  • Chani : Mastering the game of Go with deep neural networks and tree search (2016), D. Silver et al., Nature
• 03/06 Unsupervised Learning
  • Hongmin : Generative Adversarial Nets, Goodfellow et al., NIPS 2014
  •  : Auto-encoding variational Bayes, Kingma and Welling, ICLR 2014
  • Pushkar : Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, Redford et al., 2015
• 03/08 Project: final presentation (1)
• 03/13 Project: final presentation (2)
• 03/15 Project: final presentation (3)
• 03/23 23:59PM PT Project Final Report Due. Grader: Ke Ni <ke00@ucsb.edu>