5. Visualization

Since the original data and decomposition are three-dimensional volumes, showing only a time-course or a planar slice of a volume would not give an accurate view of the data or results. Therefore, it is important to visualize the information in a form that allows all the information to be analyzed easily. This is not simple since the amount of data is huge, easily making the visualization cluttered or too slow to be of any value.

Fortunately, interpreting volumetric data is quite natural for humans and specifically in medical sciences, doctors are accustomed to using images, such as ordinary x-rays. The decomposition also gives multi-modal information consisting of the volumetric functional patterns and the related activation time-courses. Visualizing the information in a clear and natural way is very important for fast and easy interpretation.

Due to their efficiency and possible interest to many other analysis environments, the visualization tools created during the work are considered as a possible future toolbox to be published on its own. Open questions related to the release of such a toolbox concern mainly portability to other environments and interoperability with existing tools.

5.2 Showing Brain Activation

The whole point of the analysis is to locate activated areas in the brain. Quite naturally, the aim of the visualization must be to allow the user to accurately see these locations. This can be problematic, as the activation patterns can be of any shape and size. Additionally, due to their smoothing preprocessing, the structural information in the functional volumes is very poor. Therefore, it is best to overlay the functional patterns on top of a structural MRI, if available. This allows the user to interactively study the content of the volumes, comparing the brain activation to known regions in the structural image. The interactive nature of the visualization adds constraints to the system in terms of usability. For example, updating the images on the screen has to be fast enough.

The estimated independent components have a time-course in addition to the spatial pattern. The temporal information is as important in the analysis of fMRI data as the spatial, because the time-courses reveal how the activation patterns are related to the stimuli. Additionally, the information acquired from the multiple runs, related to the grouping of the estimates and their nature, can be very helpful during the interpretation. For example, it is easier to focus on the most relevant results by spotting the most consistent, or reliable, components.

5.2.1 Spatial Information

Part of the interactive visualization interface is shown in Figure 5.1(a), with an example of activation pattern under study. The interface shows the volumes simultaneously from three orthogonal directions, aligned with the main axes of the volume. The three slices are linked together and the yellow cross-hairs pinpoint the current location, which lies in the intersection of the three slices. The user can move the current location freely to any point in the volume. A structural MRI is shown as a template in grayscale and the functional volume is overlaid on top of it in color.

The coloring is based on a smooth gradient, that is, a lookup table for a range of smoothly interpolated values, which makes stronger activation show up in brighter (hotter) colors. The color gradient can be seen in Figure 5.1(b), which shows the histogram of the activation volume. As the active regions are sparsely distributed, most of the volume is, in fact, noise. Therefore, the main lobe of the histogram can be considered as noise, or the inactive region. The volume is always shown so that the tail of the histogram with the most energy, or mass, is considered to be the positive extreme. This effectively fixes the sign ambiguity of ICA. However, there are cases where the histogram is almost symmetric, or nearly all of the energy is in the main lobe. This may suggest that, in such cases, there is no significant focal activation in the volume. For example, some artifacts seem to produce such a volume.

**Figure 5.1:** First partial view of the interactive user interface. (a) The volumes from three orthogonal directions with the cross-hairs pinpointing the current location. (b) The histogram of the activation volume. The structural volume is drawn using grayscale values and the amount of activation with a color gradient, where brighter (hotter) colors mean stronger activation.
$\includegraphics[height=6cm]{images/visualize_activation.eps}$ (a)	$\includegraphics[height=6cm]{images/visualize_histogram.eps}$ (b)

The gradient used for coloring the activation pattern is fitted to the range from the main lobe to the more powerful extreme. The lower end of the gradient is fully transparent and the higher end fully opaque. As mentioned, the color also changes smoothly from darker (colder) to brighter (hotter) values. For example, it is very easy to see that a strongly activated area is located at the back of the brain on the right side, most probably related to processing visual information.

Additionally, the whole spatial patterns produced by ICA are, by definition, independent of each other and thresholding the activation patterns is not as crucial as in the traditional fMRI analysis method, explained in Section 2.3.3. When the thresholding does not essentially change the results, it is easier to overlay the activation pattern on the structural image using a smooth level of transparency. This makes it easy to see how noisy the volume is or if there are many separate areas of activation. For example, often the two hemispheres of the brain are activated symmetrically, but the dominant side contains stronger activation.

5.2.2 Temporal Information

Interpreting the true nature of the variability from numerical values, like variance and rank, alone is practically impossible. Visualizing the distribution of the grouped components around the representative mean time-course of the group, the magnitude and nature of variability in the group become immediately clear. Figure 5.2 shows an example time-course of a component with the distribution drawn around it. The different quantiles are drawn using different grayscale values. For example, it is very easy to see that this component is quite consistent. The spread is big only at some time points, near sharp transitions, and only the extreme quantiles, that is, lightest shades of gray, seem to have that bigger spread.

**Figure 5.2:** Second partial view of the interface, showing the activation time-course and its variability. The distribution of the group is drawn around the mean time-course using grayscale values for quantiles.
$\includegraphics[width=0.5\textwidth]{images/visualize_variability.eps}$

Naturally, it is also possible to calculate and show the variability from the volume, but calculating the volumetric distribution is very time consuming and may not always be worth the effort. Still, the volume may allow easier interpretation of certain kind of variability, as shown in one of the results from the experiments in Figure 7.5.

5.2.3 Group Information

The components are ranked using a complex measure that produces a meaningful ordering, as explained in Section 4.3.3, but it can be very difficult to understand how the groups really differ. The problem is that the rank is always a compromise between the parameters in the measure. However, in addition to the numerical values that tell how well a group is discriminated, that is, separated from the other groups, the measures can also be visualized for easy interpretation. An example of such visualization is shown in Figure 5.3. The disks show the spread of Euclidean distances with the minimum distance defining the inner radius and the maximum the outer radius. The circles over the disks mark the mean distance. The left disk shows the spread of intra-group distances, measured between the members of the group, and the right disk shows the spread of inter-group distances, or the distances to all other groups.

This type of information is very easy to interpret. For example, the relatively small intra-group disk tells that the group is quite compact, or consistent. Specifically, the mean value is very small, suggesting that there are only a few outliers with higher distances. The inter-group distances also form a rather compact range around a large mean value, meaning that the group is well separated from all the other groups. Additionally, the good discrimination of the group can be seen from the fact that the left disk would fit completely inside the hole in the right disk, with some room to spare.

**Figure 5.3:** Third partial view of the interface, showing the discrimination power of the group. The left disk shows the spread of the Euclidean distances between the members of the group and the right disk between the distances to all other groups, with the black circles marking the mean values of those distances.
$\includegraphics[width=0.5\textwidth]{images/visualize_cluster.eps}$

5.3 Complete User Interface

All the previously discussed multi-modal information is brought together to allow the interactive human interpretation. The complete user interface, for a single component, is shown in Figure 5.4, which combines the parts shown in Figures 5.1, 5.2 and 5.3. The interface includes some numerical properties of the component, shown above and below the time-course. Also, a possible reference time-course is shown as a two color pattern beneath the activation time-course. The bands depict the on-off nature of the stimulus. This enables the user to better see how the component is related to the stimulus.

The numerical information is related to the grouping and the histogram of the activation pattern. The number of estimates in the group and the normalized rank of the component are on the top. The ratio of energy between the upper and lower tail of the histogram, related to skewness, and the amount of energy in the main lobe of the histogram are on the bottom. More examples of the visualization, and related interpretations, are shown in Chapter 7.

**Figure 5.4:** The complete view of the interactive user interface, showing all the information related to a single component.
$\includegraphics[width=0.75\textwidth]{images/visualize_gui.eps}$

Additionally, the user interface offers helpful interactive tools, which make it very user friendly. For example, the functional overlay can be toggled on or off to reveal the structure underneath. The activation pattern can also be viewed without the structural template, and without the artificial coloring, which can be very useful in situations where the component does not contain clear, or focal, activation. Also, the current location can be centered on the strongest activation automatically. All the interactive tools are focused on making the interpretation fast and easy.

5.4 Medical Standards

The medical field has a lot of information on the brain from the increasing number of studies conducted. Some of this information is available in brain atlases, which use a standard convention to localize areas of the brain, such as Talairach coordinates (Talairach and Tournoux, 1988) or the older Broadman's areas. Incorporating such information into the visualization would further increase the usability of the tools and reliability of the interpretations.