Blog posts

2018

Generative Adversarial Network (GAN) Overview

5 minute read

Published:

Generative Adversarial Network (GAN) is a class of generative models. Basically generative models aim to model the underlying distribution of some data. Some models have the specific density function for the data distribution, while other models can only generate learned distribution via an implicit density function. GAN is one of the latter models.

Task-Oriented Dialog Agents Review

3 minute read

Published:

Task-oriented dialog agents are designed for a particular task and set up to have short conversations to get information from the user to help complete the task. Apple Siri and Google Assistant are common examples. They are different from chit-chat agents in that their primary goal is to help the users complete the task. Often, they are accompanied with a knowledge base about the domain.

Text Summarization Overview

4 minute read

Published:

Automatic text summarization is the task of producing a concise and fluent summary while preserving key information content and overall meaning. With the explosion of Internet, people are overwhelmed by the amount of information and documents on it. Automatic text summarization have applications such as preview of ducuments in search engine, and headline generations/post summarizations in news websites.

SenticNet 5 - Discovering Conceptual Primitives for Sentiment Analysis by Means of Context Embeddings

4 minute read

Published:

This paper is published on AAAI 2018. This authors couple symbolic and sub-symbolic AI to automatically discover conceptual primitives from text and link them to commonsense concepts and named entities in a new three-level knowledge representation for sentiment analysis. In particular, they employ RNN to infer primitives by lexical substitution and use them for grounding common and commonsense knowledge by means of multi-dimensional scaling.

On the Convergence of Adam and Beyond

9 minute read

Published:

The paper On the Convergence of Adam and Beyond was awarded as the best paper on ICLR 2018. The authors empirically observed that several popular gradient based stochastic optimization algorithms such as Adam (Kingma and Ba, 2014) and RMSProp (Tieleman and Hinton, 2012), fail to converge to an optimal solution in convex settings (or a critical point in nonconvex settings). This paper identifies one cause of this convergence issue and proposes a fix to it through convergence analysis.