Recent Publications on Visualizing Deep Neural Networks
The visualization research community has published a number of papers on the topic of visualizing and understanding deep neural networks. This article provides an overview of these publications.
LSTMVis provides a pretty neat visualization tool to study the hidden states in RNNs. It visualizes the change of hidden states over time as parallel coordinates. Users can observe the shape of hidden states, select a range of inputs and a threshold on hidden state values as the formulation of a hypothesis. The tool then matches inputs with similar hidden states patterns to support refinement of hypothesis. External linguistic information can be overlaid on the matched inputs to validate hypothesis. This tool is great for exploration in the sense that it lets users to freely formulate, refine and validate hypothesis with minimum restriction. It is open sourced and hosted online, go and play! RNNVis also focuses on hidden states of RNNs. It establishes direct connections between hidden states and semantic information. It provides information not just about individual states, but the distributed relationship among states.
CNNVis focuses on convolutional networks. It proposed a DAG formulation, a matrix packing algorithm and edge bundling techniques to effectively visualize large CNNs. It preserves the global structure of a CNN and provides feature and activation information in the hidden layers. Demo is online. For those who are interested in feature visualization of image-based models, it is more thoroughly explored in in this Distill publication.
Dimensionality reduction is a popular and effective technique to reveal facets of the strcuture of the hidden representations learned by deep networks. This paper employs t-SNE to study (1) the relationship between learned representations of observations (2) the relationship between artificial neurons. It exposes interesting patterns of the learned representations of a model as well as how the representations evolve during training. Google’s Embedding Projector provides a platform to easily plugin data, model and dimensionality reduction techniques to visualize 2-D or 3-D projections. A more in-depth exploration of t-SNE can be found in this Distill publication.
Computational graph provides effective external and internal images to anchor and navigate through the structure of complex models. TensorBoard employs various techniques to enable standard graph layouts for complex models. The hierarchical nature of deep models allows overviews by showing the highest level of the hierarchies. Each level can be expanded to see its nested sub-levels. ActiVis also chooses computational graph to display model architectures. It further supports selecting regions in the computational graph to explore their activations of single instance or subsets as inputs.
Most of these papers are forms of design study. They first identify problems in the deep learning domain that are suitable for a visualization solution, then design and implement commensurate visual analysis tools. Right now, each paper characterizes a different set of problems in the domain. I think the community will benefit from synthesizing these problems into a standard set of shared recognition. Later design study can use the set as a reference to either facilitate identifying problems or add new elements to the set. Other types of work can focus on (1) how to transform these problems into visualization tasks (2) invent techniques and design systems to tackle these tasks.
Here,each paper has different goals and proposes different approaches. Depending on what your research goal is, some of them can serve as reference and inspiration.
(More material will be added to this post)