Spring 2017 CS292F Syllabus

• 04/04 Introduction, logistics, NLP, and deep learning.
• 04/06 Tips for a successful class project
• 04/11 Word embeddings
  • A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning, Collobert and Weston, ICML 2008
  • Distributed Representations of Words and Phrases and their Compositionality, T Mikolov, I Sutskever, K Chen, GS Corrado, J Dean, NIPS 2013
  • Glove: Global Vectors for Word Representation, J Pennington, R Socher, CD Manning - EMNLP, 2014
• 04/13 Knowledge base embeddings
  • A three-way model for collective learning on multi-relational data, M Nickel, V Tresp, HP Kriegel, ICML 2011
  • Translating embeddings for modeling multi-relational data, A Bordes, N Usunier, A Garcia-Duran, NIPS 2013
  • A Review of Relational Machine Learning for Knowledge Graphs, Nichel et al., Proceedings of the IEEE
• 04/18 Neural network basics (Project proposal due, HW1 out)
• 04/20 Neural networks language models
  • A Neural Probabilistic Language Model, Bengio, JMLR 2003
• 04/25 RNNs
  • Recurrent neural network based language model
  • Generating Sequences With Recurrent Neural Networks, Alex Graves, 2013 arxiv
• 04/27 LSTMs/GRUs
  • Long short term memory, S. Hochreiter and J. Schmidhuber, Neural Computation, 1997
  • On the Properties of Neural Machine Translation: Encoder–Decoder Approaches, Cho et al., 2014
• 05/02 Sequence-to-sequence models and neural machine translation (HW1 due and HW2 out)
  • Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation, Cho et al., EMNLP 2014
  • Sequence to Sequence Learning with Neural Networks, Sutskever et al., NIPS 2014
• 05/04 Attention mechanisms
  • NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE, Bahdanau et al., ICLR 2015
  • Teaching Machines to Read and Comprehend, NIPS 2015
• 05/09 Project: mid-term presentation (1)
• 05/11 Project: mid-term presentation (2)
• 05/16 Convolutional Neural Networks (HW2 due)
  • Natural Language Processing (Almost) from Scratch, Collobert et al., JMLR 2011
  • Convolutional Neural Networks for Sentence Classification, Yoon Kim, EMNLP 2014
• 05/18 Language and vision
  • Show and Tell: A Neural Image Caption Generator, CVPR 2015
  • Deep Visual-Semantic Alignments for Generating Image Descriptions, Andrej Karpathy and Li Fei-Fei, CVPR 2015
• 05/23 Speech recognition and understanding
  • Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, Hinton et al., 2012 IEEE Signal Proc. Magazine
  • Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition, Dahl et al., 2012 IEEE TASLP
• 05/25 Information Extraction
  • Lample, Guillaume, et al. "Neural Architectures for Named Entity Recognition." Proceedings of NAACL-HLT. 2016.
  • Miwa, Makoto, and Mohit Bansal. "End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures.
• 05/30 Summarization
  • A neural attention model for abstractive sentence summarization EMNLP 2015
• 06/01 Question answering
• 06/06 Project: final presentation (1)
• 06/08 Project: final presentation (2)