2. Brain Imaging

The human brain has been one of the biggest research topics in many sciences, including biomedical, psychology and information theory. Current knowledge of the structure and function of the brain is substantial and growing fast, due to new imaging and analysis techniques. Consider reading Kalat (2003) for a great overview on the human brain. The key concepts needed to understand the thesis are explained next.

2.1.1 Structural Anatomy

The basic anatomical structure of the human brain is depicted in Figure 2.1. It has been studied for a long time, even before there were any ideas on what the brain does or how it functions. Much of the knowledge is based on histological studies, but under extreme cases, the living brain has been studied with crude and invasive methods. Currently, less invasive imaging methods allow the study of the structure and changes in it during the life of a subject, for instance, under a progressing illness.

**Figure 2.1:** Anatomical structure of the human brain. (a) The horizontal view from above and the sagittal views from (b) the side and (c) middle of the brain show the basic structures and divisions, including the four lobes separated by sulci and fissures. Important names and directions are also shown.
$\includegraphics{images/brain_structural_legend.eps}$
$\includegraphics[width=0.32\textwidth]{images/brain_structural_horizontal.eps}$ (a)	$\includegraphics[width=0.33\textwidth]{images/brain_structural_sagittal.eps}$ (b)	$\includegraphics[width=0.29\textwidth]{images/brain_structural_midsagittal.eps}$ (c)

The central nervous system (CNS) is formed by the cortex, brain stem, cerebellum and other connected subcortical areas. The brain stem and other subcortical regions are mainly involved in lower level functions, like automation and primitive signal processing. Higher functions, such as conscious thought, are performed on the cortex, that is, the surface of the brain. Most higher functions, like memory, also rely on support from the subcortical areas.

There are essentially two kinds of tissues, the gray matter and the white matter. The gray matter contains the actual cell bodies of the neurons and most of it is concentrated on the surface, or cortex, of the brain. The connecting structures between the neurons form the white matter. The inside of the brain is mostly white matter, but there are some nuclei, which are small groups of neurons and basically work as junctions along signaling pathways.

The surface is heavily folded to increase its area. The folds are called sulci and separate the surface into small sections, or gyri. Bigger folds that separate larger parts are called fissures, like the longitudinal fissure in Figure 2.1(a), which separates the left and right hemispheres of the brain.

The division of the cortex into the four lobes, as shown in Figure 2.1(b), may seem somewhat arbitrary, but is based on major sulci and fissures, visible on the surface. Additionally, fine details, like the density of neurons and their size and shape, differ between the areas. Naturally, the boundaries are not always so clear in real brains, and can change slightly from one individual to another.

2.1.2 Functional Anatomy

As the gray matter is mostly on the surface of the brain, functional anatomy mainly details areas on the surface. But the connections are also very important. The neuronal configuration is similar throughout the surface, but different inputs and outputs of the peripheral nervous system are connected to different parts of the brain. Thus, different areas of the brain are involved with different kind of information and serve a different purpose. Table 2.1 provides a quick lookup of some of the main details.

Table 2.1: Functional properties of the four lobes of the brain.

Lobe	Input and Output	Other Functions
Frontal	Motor	Memory and Emotions
Temporal	Auditory	Language and Structure
Parietal	Somatosensory	Association and Attention
Occipital	Visual	Pattern and Object Recognition

Figure 2.2(a) shows the location of some of the well known primary processing areas on the cortex more accurately. These areas are mainly connected contralaterally, which means that the areas on the left hemisphere are mainly responsible for signals from the right side of the body. The primary areas are then connected to additional areas nearby on the same hemisphere, or ipsilaterally. The additional areas usually perform more complex functions based on the processing done on the primary areas. The left and right hemispheres of the brain are functionally quite symmetric, but usually each task has a more dominant side. The brain is also adaptive in the sense that sometimes other areas overtake more functionality, when the dominant side suffers an injury.

**Figure 2.2:** Functional areas of the human brain. (a) Primary areas involved in processing different information and (b) areas involved in auditory and speech signal processing.
$\includegraphics{images/brain_functional_legend.eps}$	$\includegraphics[width=0.31\textwidth]{images/brain_functional_primary.eps}$ (a)	$\includegraphics[width=0.31\textwidth]{images/brain_functional_auditory.eps}$ (b)

2.1.3 Auditory Signal Processing

At first, audio processing may seem relatively simple compared to, for example, visual image processing. However, auditory processing is closely linked to understanding spoken language and therefore also to higher functions, such as memory and conscious thinking. Audio processing in the brain has been studied extensively and still remains an interesting topic in current research.

The signal processing actually begins already in the ear and in the thalamus, even before the signals reach the cortex. The early processing is used to form a tonotopic map, based on frequency, on the primary auditory area. Also other processes, like focusing attention, affects how signals arrive at the cortex. Unlike in many other sensory inputs, audio signals from both ears are used together, which makes it possible to detect the direction of the original sound by analyzing the phases of the signals.

The primary auditory area responds to all kinds of sounds, but it is tightly connected with additional areas involved in more complex processing. These areas are shown in Figure 2.2(b) and are involved in, for example, understanding speech and forming sentences. The addional areas are usually more active on the left hemisphere. The precise function of these and other areas is currently studied (c.f., Arnott et al., 2004, Bartels and Zeki, 2004, Calhoun et al., 2001a) and much remains to be discovered.

2.2 Evolution of Imaging Techniques

For a long time, only crude pathological methods existed for studying the brain, and functional studies were virtually impossible. Brain imaging has been developing rapidly during the last decades. Specifically in recent years, it has become also quite noninvasive, allowing the routine imaging of living tissues. For example, Bankman (2000) describes many recent brain imaging techniques and how they are utilized in the biomedical field.

Histological studies, using small tissue samples from pathological analysis, formed the basis of understanding, and motivated the development of methods to study the living brain. The oldest imaging methods use the gamma radiation generated by radioactive decay, commonly known as x-rays, to form images based on differing absorption properties of tissues. Taking x-ray images is very quick and simple, much like using an ordinary pocket camera. But repeated exposure to the radiation can be harmful and the contrast in soft tissues is not very good. Most of us are painfully aware how x-rays are still very widely used, for example, to image bone fractures.

For many years, studying brain functions was possible only by observing the relations between different areas of the brain and different functions when comparing the performance of healthy subjects to that of brain-damaged subjects. For example, aphasia refers to brain-damage that affects language skills.

Electroencephalography (EEG) and magnetoencephalography (MEG) (c.f., Niedermeyer and da Silva, 2004) changed the situation by allowing direct measurements of brain activity. EEG is based on measuring small changes in the electric field caused by neuronal activity, using small sensors attached to the scalp. In MEG, superconducting sensors positioned close to the scalp are used to measure changes in the related magnetic field. EEG and MEG are perhaps the most widely used techniques for functional studies of the brain, such as evoked responses, because they are completely noninvasive and offer very high temporal resolution. However, the inverse problem of finding out the originating location of the brain-waves accurately is very difficult. In some contexts, EEG and MEG are not considered as true imaging methods.

Advances in computer technology made computed tomography (CT) (Andreasen, 1988) possible. Essentially, the idea in CT is to take several x-ray images of a target from many directions and, using a computer, carefully combine the information in each two-dimensional image to form a single three-dimensional image of the target. CT can produce high resolution images, but suffers from the same problems as ordinary x-ray images, and is mainly usable for structural imaging. Additionally, taking many images naturally multiplies the exposure of the subject to the potentially harmful radiation.

Positron emission tomography (PET) (Andreasen, 1988) also produces three-dimensional images, but is based on radioactive markers injected to or inhaled by the subject. The marker substances can be pharmacologically designed to participate in certain metabolic reactions. The fast decaying also improves the image contrast. PET can produce images targeted at specific organs or processes related to illnesses, such as cancer, but the temporal resolution is not very good due to the slowness of the metabolic changes. Additionally, injecting radioactive substances into the body is invasive and potentially harmful. Furthermore, producing the needed beta-decaying substances is very difficult and expensive. Their rapid decaying also makes their usage difficult. Thus, PET is mostly used in studying severe illnesses.

Magnetic resonance imaging (MRI) (Moonen et al., 1990) is a virtually noninvasive method based on nuclear magnetic resonance, which is rather complex and shortly explained in Section 2.3. Because of the noninvasive nature and ability to produce high quality images, MRI has quickly become a popular method. Examples of such MR-images are shown in Figure 2.3. They are scans of a human head, using a setup that produces good contrast between different tissue types, thus revealing the anatomical structure in great detail. The method is also very suitable for functional imaging, allowing a good compromise between spatial and temporal resolution.

**Figure 2.3:** Examples of structural magnetic resonance images. The images show slices of a human head viewed from (a) sagittal, (b) frontal and (c) horizontal directions.
$\includegraphics[height=5.4cm]{images/mri_sagittal.eps}$ (a)	$\includegraphics[height=5.4cm]{images/mri_frontal.eps}$ (b)	$\includegraphics[height=5.4cm]{images/mri_horizontal.eps}$ (c)

Recently, near-infrared spectroscopy (NIRS) or diffuse optical imaging (DOI) (Meek et al., 1995) was proposed as a potential future technique. It is based on the diffusion of laser-light in tissue and blood. Laser based imaging would be fast and not sensitive to electromagnetic interference, allowing much cheaper imaging. The difficulties in using the method include the ability to generate high resolution images and penetrate deep enough into the tissues. In time, NIRS may become yet another widely used imaging technique.

Practically all the methods suffer from interference caused by the surrounding environment, usually in the form of electromagnetic radiation, but each technique has its benefits and may suit certain tasks better than the others. Newer methods are not always developed to replace the older ones, but may rather aim at solving some specific problems. All the methods mentioned before are still in use, and being improved to make them faster or less invasive. The rest of the thesis, however, focuses on the functional form of MRI, which is explained next.

2.3 Functional Magnetic Resonance Imaging

The complex physics of nuclear magnetic phenomena are very interesting, but certainly beyond the scope of this thesis. Therefore, only an overview of the main concepts is given in this section, which also covers the standard processing and analysis methods used in fMRI studies. Consider reading Huettel et al. (2004) for a more thorough introduction.

Nuclear particles, such as protons and neutrons, have a magnetic property called spin, which behaves much like an ordinary dipole-magnet. Usually, the focus of interest is on hydrogen nuclei, because of their abundance in all tissue types and relatively simple spin behavior. The behavior of the nuclear spin of heavier atoms is more complicated, since there are internal interactions between the individual particles. Fundamentally, all MR-imaging is based on the interaction between the imaged tissue, externally applied magnetic fields and carefully synchronized radio frequency pulses.

Under a very strong and uniform magnetic field, the spins try to align parallel to the field either in the same or opposite directions. More precisely, the spins precess around those directions in a minimum energy state. Although the precession is not coherent, all the spins have a characteristic resonance frequency, proportional to the strength of the magnetic field. When a radio pulse in the resonance frequency is emitted, the spins absorb the energy and are forced into coherent precession. After the pulse, the absorbed energy decays in a relaxation process. The process is actually quite complex, for example, due to possible internal interactions in the tissue. However, signals measuring the decay of energy make the imaging possible, and adjusting the properties and timing of the radio pulses produces signals related to different aspects of the relaxation process.

To produce a volumetric (3-dimensional) image, the behavior of the spins is controlled more precisely with two gradient magnetic fields, which are perpendicular to each other. The first is applied in the same direction as the uniform magnetic field, causing the total strength of the field to change slightly along that direction. This makes the resonance frequency of the spins also different along its axis. The other gradient field is similar, but turned on and off repeatedly. As the spins precess faster during the application of the field, the spins along its axis accumulate different phases. The gradient magnetic fields allow focusing on a planar slice, which is defined by the axes of frequency-phase space. Carefully synchronizing the radio pulses with the fields produces signals originating from different parts of the slice. Additionally, the thickness of the slice can be controlled with the bandwidth of the radio pulses.

Using standard signal processing techniques, the measurements can be turned into an image of the focused slice. The full volume is produced by scanning several adjacent slices, one after the other. The image voxels contain a kind of density measure based on the scanning parameters and the properties of the tissue. For example, with certain parameters the image is directly related to the proton density of the tissue.

The scanning is actually very slow since the relaxation process, and adjusting the magnetic fields, requires a certain amount of time. Producing high resolution images, such as the ones in Figure 2.3, can take several minutes. Naturally, the quality of the images is strongly affected by inhomogeneities in the magnetic fields, the internal magnetic interactions and electromagnetic interference from the environment.

Since MRI is virtually noninvasive and is able to produce high quality images, it has quickly become very popular in structural imaging. These properties are crucial also in functional imaging, but the measures used in structural MRI, such as the proton density, are not directly related to neuronal activation. Coincidentally, the scanning parameters can be tuned so that the resulting images provide a measure related to the oxygenation level in the tissue. The measure is based on the differing magnetic properties of oxygenated and deoxygenated hemoglobin molecules, as further explained in Section 2.3.1. The idea in functional MRI is to record a sequence of such images at different time points to allow the local changes in oxygenation level to be analyzed.

Problems arise from the long duration of the scanning. Scanning intervals of several minutes would not allow accurate analysis of the activity in the brain. Additionally, movement of the head and other physiological changes during the long exposures would distort the images. Fortunately, the scanning parameters, mainly the timing of the radio pulses, can be adjusted to allow much faster scanning. However, the spatial resolution suffers greatly from such adjustments. For example, the relaxation process is not allowed to fully complete, resulting in much weaker signals. Therefore, the setup used in fMRI is a careful compromise between fast scanning and high resolution images. Current fMRI scanners are able to produce full head volumes with a time interval of a few seconds, but the spatial resolution is only a fraction of that used in structural imaging.

2.3.1 Measuring Hemodynamic Responses to Stimuli

The detection of changes due to neuronal activation in fMRI is based on the differing magnetic properties of oxygenated (diamagnetic) and deoxygenated (paramagnetic) hemoglobin molecules. Neuronal activation results in a localized change of blood flow and oxygenation levels, which can be measured using suitable scanning parameters. This produces a measure called blood oxygenation level dependent (BOLD) signal (c.f., Ogawa et al., 1992). These vascular or hemodynamic changes are related to the electrical activity of neurons in a complex and delayed way. Local changes in the level of oxygenation reveal the active areas, but it is not possible to completely recover the electrical processes from the vascular ones. The hemodynamic changes are hard to model, but as their nature is somewhat slow and smoothly varying, a Gaussian model is often used. Some newer models of the hemodynamic response function are actually based on measurements from the real brain.

Usually, fMRI studies use a controlled stimulus, like visual patterns or audible beeps, designed to test a specific hypothesis. The simplest way of doing this is to repeat the stimulus several times, with resting periods in between, and scan the fMRI sequence during the whole time. Such experiments should reveal areas of the brain that are always active during the stimulation and inactive during the resting. Sometimes the subject can even be asked to perform a relatively simple mental or motor task during the scanning. Naturally, such tasks should not be allowed to result in head movement.

One example of an fMRI study is shown in Figure 2.4(a). The low resolution and the scanning parameters, optimized for BOLD, make the contrast between different tissue types very poor. Additionally, the fast scanning and low signal-to-noise ratio of the BOLD signal make the image very noisy. Therefore, a high resolution structural MRI is often scanned separately to aid in locating the activation during analysis by super-positioning. The bright areas in the images do not necessarily correspond to the active ones. Careful analysis of the whole sequence is required to detect the activation patterns.

**Figure 2.4:** Examples of functional magnetic resonance images. (a) A scanned slice without processing and (b) after the standard preprocessing has been applied. Note that the images do not show a direct measure of activation.
$\includegraphics[height=6.5cm]{images/fmri_raw.eps}$ (a)	$\includegraphics[height=6.5cm]{images/fmri_smooth.eps}$ (b)

2.3.2 Standard Preprocessing of Images

In addition to the low signal-to-noise ratio and additive noise, seen in Figure 2.4(a), the fMRI measurements are contaminated with artifacts, such as head movement and physiological vascular changes. Thus, the detection and analysis of interesting phenomena is very difficult. To overcome these difficulties, the images need to be preprocessed (c.f., Worsley and Friston, 1995, SPM, 1999). Figure 2.4(b) shows an example slice after preprocessing. The level of noise is clearly reduced and the values are much more continuous. Also, the excess area outside the brain has been removed, and is shown in black. The usual steps of the preprocessing include:

The normalization step is not required for analysis, but is often done to match the functional volumes to the structural ones. Additionally, the normalization makes comparisons between different subjects easier. In practice, this may somewhat distort the brain in the images and individual differences may still remain quite big, so that caution needs to be taken when drawing inter-subject conclusions.

2.3.3 Standard Analysis of fMRI Sequences

The standard way of analyzing an fMRI sequence is to use statistical parametric mapping (SPM) (SPM, 1999), which is based on a general linear model (GLM) (c.f., Worsley and Friston, 1995). Essentially, the analysis reveals the areas of the brain that most probably fit a given hypothesis, which is presented as a reference time-course.

The reference time-course can be approximated using the stimulation pattern and a model of the hemodynamic response. An example of such a reference time-course is shown in Figure 2.5. The depicted pattern is a very simple case of repeated on-off type of stimulus. The stimulation time-course is then convolved with the model of the hemodynamic response, assumed Gaussian in the illustration.

**Figure 2.5:** Illustration of an ideal stimulus. (a) An ideal stimulation time-course comprising of rest and activation periods. (b) The corresponding ideal reference time-course convolved with the hemodynamic response. In reality the detected time-course varies significantly.
$\includegraphics[width=0.48\textwidth]{images/fmri_stimulus.eps}$ (a)	$\includegraphics[width=0.48\textwidth]{images/fmri_reference.eps}$ (b)

The analysis can be considered in two steps. First, the reference time-course is compared to the time-course of each voxel in the fMRI sequence statistically. This produces an image of the probability to fit the given time-course, where the voxels with the highest probabilities are considered to be active. However, the probability image is very noisy and the second step is to segment it into the inactive and active areas. The segmentation is made robust by using a statistical model for the noise, usually assumed Gaussian. The difficulty with this approach is to define a threshold for the probability of activation that produces an accurate segmentation. Choosing a too high value value easily leads to discontinuous or too small areas. On the other hand, a small value may produce big areas that do not accurately locate the activation of interest.

After the spatial activation patterns have been formed, the true activation time-course of each area is formed by taking the mean sequence of all the voxels in the area. Again, if the segmentation is poor, for example, due to an incorrect threshold value, the time-courses are not generated accurately.

There are big problems with such analysis. The accuracy is limited by the ability to approximate the parameters needed for the statistical fitting. Also, small changes to the parameters can change the results severely. Additionally, the stimulation setup has to be simple enough to allow predicting the responses, and forming the reference time-courses, in the first place. Therefore, detecting previously unknown phenomena is extremely hard or close to impossible. These are some of the reasons why current research focuses on more data-driven and adaptive methods, like independent component analysis, explained in Chapter 3.

2.4 Individual and Group Studies

The formulation of general hypotheses is possible only by comparing the results of many subjects. Functional brain studies, like all studies in the biomedical field, are often group studies where many patients are subjected to the same conditions or given the same treatment. However, even under a controlled environment the individual responses change, for example, with attention.

The inherent differences in both individual and group results make the studies difficult. Therefore, it is extremely important that the analysis methods themselves are consistent and reliable. One such consistent method is proposed in Chapter 4.