Each digitised image consists of a matrix of grey values (pixels) distributed on three colour dimensions (most frequently in red, green, blue (rgb)) axes. Thus, the information mapped on this three-dimensional colour space is a function of the grey values in principle. The information acquisition by any viewer (pathologist) can be formulated by a function that transfers the grey value distribution to the analysing pathologist and depends in addition upon the pathologist’s knowledge and interpretation. Interpretation requires in addition the detection and identification of objects, which represent the “external” knowledge. At light microscopy, magnification objects are nuclei and cells, cellular agglutinations arranged to functional units such as vessels, glands, nerves, etc., or intestinal surfaces, or skin. The neighbourhood of these functional units or biological meaningful objects may contain a biological function again, for example goblet cells and ciliated cells in bronchial mucosa.

A more sophisticated analysis directs to the concept of ordered structures in living organisms which seem to be a necessity to stay alive [34, 35, 45–48].

In respect to image analysis, the derivative is the idea that all content-based image information can be derived from three different, not necessarily independent, parameters, namely texture, object, and structure [37, 49, 50]. The principle is demonstrated in Fig. 4.7. Texture is a mapping of pixel-based grey values, which can be transformed by different transformations such as Fourier analysis, regressive analysis comparable to time series analysis, Laplace transformations, local and other filters, etc. [36, 37, 51, 52]. The second and third techniques depend on segmentation and identification of external defined objects and their spatial relationship. The definition of a neighbourhood is mandatory to analyse the structures built by the objects. Voronoi’s neighbourhood condition and the graph theory are often applied to measure structural features [15, 53–58].

The three image information components can be set in order as follows:

Different colour spaces have been defined to present and analyse a digitised image. The most frequently used one is the image presentation in the rgb space. The rgb space takes into account that each colour can be composed by contribution of colours red, green, and blue. The rgb space is suitable to working with fluorescent images because most of the signals are highly colour specific, for example DAPI (blue), FITC (green), or Texas red (red) [15, 27, 59]. The same statement holds true if monochrome light sources (lasers) are used. A transformation of the rgb space into a different colour space that separated the intensity is of advantage if “normal” IHC, especially DAB, is applied which results in a brown detection signal [60]. The HSI colour space (hue, saturation, intensity) is an often used transformation of the rgb colour space. It permits an easy computation of colour-specific intensities [44, 59, 61–63].

Images of “good quality” are mandatory to extract content-based information or to derive a certain diagnosis. However, what is good image quality? The answer depends upon the aim to have “good quality”, or, in other words, what is meant by good quality. The quality measures of images that serve for a certain diagnosis probably differ from those that are created by artists that are designed to express ideas or personal views of specific events or motions. Thus, we can distinguish between subjective and machine-oriented methods to define image quality. Subjective methods analyse colour and intensity (brightness) adjustment to the individual human perception (morning light, sunset, moonlight, etc.). Machine-oriented methods compare image copies with the original ones, and correct potential differences.

Measurements of digitised light microscopy images are aimed to result in reliable, reproducible, and clinically applicable data [44, 59, 63, 64]. The mandatory image properties include:

Some of these requirements can be artificially readjusted. Especially the shading and the grey value range can be corrected to an optimum. An example of automated image quality correction is shown in Fig. 4.8. All these parameters should be continuously measured, corrected, and demonstrated in series as reported by Kayser et al. [65, 66]. Analysis of such series can explain potential inaccurate measurements [16, 59, 65–67].

The segmentation and consecutive identification of objects may be difficult especially in IHC images. Objects that are characterised by closed surfaces (nuclei, cells, glands, vessels, etc.) are usually segmented by algorithms that are based upon a dynamic or fixed threshold, and, in addition, upon edge finding mechanisms and analysis of the convexity of the already segmented surface segments [37, 50, 51, 64, 68, 69]. Such algorithms with included variable thresholds and appropriate feedback mechanisms have been reported to reproducibly segment (and identify) more than 95 % of nuclei in IHC images [39]. Membranes are more difficult to segment, especially in IHC images with only partially stained membranes. The identification of nuclei, the computation of the corresponding gravity centres, and the application of Dirichlet’s (Voronoi’s) tessellation permit the construction of artificial membranes adjusted to the IHC stained membrane fragments [44, 52, 70]. Such an algorithm permits a detailed and reproducible, automated calculation of membrane stains (for example expression of her2_neu) in addition to the crude Her2_neu score [17, 27, 38, 71–73].

Vessels can be segmented in a comparable manner, especially if the inner surface (endothelial cells) is marked by specific antibodies (CD31, CD34, or VEGFR (vascular endothelial growth factor receptor)). The application of appropriate neighbourhood condition and double marker stain (for example Ki67 (MIB2) for proliferation and CD34 for intra-tumour vessels) has been reported to permit a sophisticated insight into the association of tumour cell proliferation and distance from the nearest neighbouring vessel. The situation is shown in Fig. 4.9 and visualises the diffusion density within a malignant tumour [70].

Segmentation of FISH signals is relatively easy due to the sharp and demarcated FISH signals. It is, however, handicapped by the small size of the signals and the thin focus plane which might not contain all FISH signals. Therefore, FISH images should be composed by several focus layers in order to possess and analyse all nuclear signals [71, 72].

Analysis of IHC detectable antigens in the cellular cytoplasm is best performed by stereologic methods, such as point/grid counting, in our opinion. An object is the overlay of an appropriate grid (for details, see [74–77]) and the automated assessment of the hit events (points of interest) which are used to calculate stereologic parameters such as area/volume fraction, size adjusted point (object) density, length/area fraction, etc.

An object remains the principal and visual controllable main feature of any digitised image. Most objects of microscopic images reflect to nuclei, followed by membranes in images derived from IHC. Once the objects have been segmented, they are considered “as a whole” and their features are measured [36, 45, 47, 56, 58, 78]. There does exist an additional method for analysing and characterising digitised images based upon stochastic geometry, so-called image primitives [44, 49, 79–82]. The idea is that each object or characteristic unit of an image can be constructed by a set of primitives that include {points, lines, circles} above and below a certain threshold [49, 50]. They correspond to peaks (holes) and ridges (valleys), and form a so-called group. Similar to the syntactic structure algorithms that are derived from image objects (usually nuclei), these primitives can serve for extraction of image information. The flexible arrangement of thresholds and the threshold-depending clusters and neighbourhood function is probably an advantage of such algorithms [39]. However, detailed reports of IHC application are still missing to our knowledge. An example of the described method is shown in Fig. 4.10.

There exist at least five different techniques to derive information from a histological image [37, 38, 44]: (a) analysis of pixel grey values and their coordination (texture analysis); (b) external definition of objects, segmentation, and object-related feature extraction, (c) analysis of object-associated structures and structure-related feature extraction (after definition of a neighbourhood condition), (d) construction of image primitives and related feature extraction, (e) comparison of the individual image “as a whole” with a “set of standardised and quality (content) evaluated set of images (gold standard set)”, computation of the best fitting (nearest neighbouring) image, and association of the (already computed) corresponding labels (diagnosis, texture, objects, structures, and features) with the image under request. Each of the mentioned algorithms has its advantages and disadvantages. They can be considered to be independent from or only weakly dependent upon each other and, therefore, be combined. Especially, the combination of texture, object, and structure features has been applied successfully to automatically screen histological images and to derive crude diagnoses (normal, inflammatory altered, benign and malignant tumours) [37, 50, 51]. The principal features are based upon grey values and their distribution and include size, surface, form factor, moments, pixel-based entropy, moments, transformation-associated features (derived after application of locally dependent and independent filters/transformations such as Fourier, Prewitt, Laplace, Radon, Hadamard, etc.) [57, 82–85]. All these methods result in a vector that can be statistically analysed in respect to observer-related information (diagnosis, prognosis, response to therapy, etc). The applied statistical methods include neural networks and multivariate analysis [27, 44, 56, 63, 80, 86]. The statistical investigations require a training set (to define the diagnosis-associated limitations of the variables, or to train the neural network, and a test set that confirms the calculated results [27, 87–92]. These results can, in addition, be used to search for and select those image areas that contain the “most significant information”. What is the most significant image information?

Already at a first glance it can be stated that a virtual slide (VS) can be disintegrated into image compartments that might be identical in terms of the visualised objects and structures, or that might differ in between. Such an image consists of image compartments that reflect more to the diagnosis compared to others. The areas are called regions of interest (ROI) and can contain an increased frequency of proliferating cells, a denser vascularisation, or less necrotic areas. They are, in addition, considered to be of increased prognostic significant value.

How can these compartments be automatically detected?

All the above discussed techniques can be applied. The image is either divided into connecting compartments equal in size, and the different compartments are analysed in respect of object density and structure parameters (and, in addition, of texture features (sliding technique)), or clusters of specific texture features dependent upon the minimum spanning tree and derived structural entropy are identified [27, 44, 50]. Both techniques have been reported to detect ROIs with a sensitivity >95 %. An example of the obtained ROI in a biopsy of breast (sclerosing ductal carcinoma) is depicted in Fig. 4.11. Other approaches are based upon stereologic sampling and diffusion maps [84, 86, 93, 94], or by density calculations of similarity (dissimilarity) maps [85].