Integrated Visualization and Analysis of a Multi-scale Biomedical Knowledge Space

M. Pohl and J. Roberts (Editors)

Integrated Visualization and Analysis of a Multi-scale Biomedical Knowledge Space

Asan Agibetov¹, Ricardo Manuel Millán Vaquero²,

Karl-Ingo Friese², Giuseppe Patanè¹, Michela Spagnuolo¹and Franz-Erich Wolter²

1Istituto di Matematica Applicata e Tecnologie Informatiche, Consiglio Nazionale delle Ricerche Genova, Italy

2Welfenlab, Division of Computer Graphics, Leibniz University of Hannover, Germany

Abstract

The study and analysis of relationships in a complex and multi-scale data set is a challenge of information and scientific visualization. This work proposes an integrated visualization to capture all the important aspects of multi-scale data into the same view by leveraging the multi-scale biomedical knowledge encoded into an underly- ing ontology. Ontology supports visualization by providing semantic means to identify relevant items that must be presented to the user. The study and analysis of relationships across the scales are presented as results of queries to the multi-scale biomedical knowledge space. We demonstrate the prototype of the graphical interface of an integrated visualization framework and the knowledge formalization support in an example scenario related to the musculoskeletal diseases.

Categories and Subject Descriptors(according to ACM CCS): H.5.0 [Information Systems]: Information Interfaces and Presentation—General; J.3 [Computer Applications]: Life and Medical Sciences—Health; I.2.4 [Computing Methodologies]: Artificial Intelligence—Knowledge Representation Formalisms and Methods

1 Introduction

This paper investigates visualization methods for the biomedical domain that studies musculoskeletal articulation of the human body and related diseases, focusing on the anatomical district of the human knee. In particular, we are interested in studying pathologies that may have different disease features at different scales (e.g. cellular, molecular, tissue, anatomy and behavior). Consider a pathology that was evidenced as the result of a gait pattern study (behav- ior scale), which might have been caused by the disruption of the macromolecules content during the cellular behavior change (cellular scale). For a complete understanding of this disease, information sources from all different scales and their relationships have to be considered.

In the following, we assume that the information sources that come from different scales, their relationships and data representing or accompanying them, reside in a multi-scale biomedical knowledge space, and for simplicity we are go- ing to refer to it asM. InM, different specialists, such as tissue or biomechanical engineers, contribute with their data and expertise. Sharing the information contained inMis re- quired but it is not an easy task. On the one side, knowledge formalization may be used to specify explicitly a shared con-

ceptualization, for instance using ontologies [SBF98]. On- tologies are a means to identify relevant items in a given do- main and formally define what are the properties or attributes necessary to document them for an effective sharing. On the other side, ontologies alone are not enough. To give a cog- nitively rich and interactive exploration ofM, smart visu- alization means are needed to integrate visualization at the conceptual level with visualization of patient data. Current ontology visualization tools are not sufficient to this aim, hindering the use of ontologies, and formalized semantic de- scription in general, in the biomedical domain.

In this work we propose an integrated visualization en- vironment to provide effective means to visually navigate the knowledge space of a complex domain such as mus- culoskeletal diseases. In this context, ontology serves as knowledge formalization of M and it also registers the suitability of visualization techniques of the patient-specific data. Queries to the ontology determine semantically rele- vant data and information to be visualized together in a sin- gle frame of view, alleviating the search across all the patient specific-data and allowing a local and global understanding.

c The Eurographics Association 2014.

2 Previous work

Multi-scale biomedical visualizationaims at the integration of biomedical data distributed at different spatio-temporal scales and their simultaneous presentation with detailed and global information [MMC^∗12]. Except in the domains of genomics and proteomics, it is mainly covered by Scien- tific Visualization (SciVis), focusing primarily on physical data and providing realistic representations. Advances in multi-scale biomedical visualization have been made dur- ing the last decade, and the level of integration of multi- disciplinary research has been increasing [OGG^∗10]. Recent projects [HCB^∗10] enable collaborative investigation of the human body as a single complex system and demonstrate how multi-scale approaches can solve specific multidisci- plinary biomedical challenges. However, improvements are slow due to the difficulties in the development of scientist- centric visualizations that provide understanding [Joh04].

Fully integrated visualization across all the range of data types has not been achieved yet [GOB^∗10].

An ontology is a knowledge representation, whose role is to define the concepts, relationships, and other distinctions that are relevant for modeling a domain [Gru93]. Applica- tions of ontologies in medicine range from definitions and classifications of common medical terms [SCC97], to ex- plicit specifications that help organizing heterogeneous data and documenting background knowledge for further reuse and integration. Most of the biomedical ontologies [RM03, GSG04,Lan06] are written in Web Ontology Language (OWL) [BKMPS], which has a model-theoretic semantics defined in Description Logics (DL) [BCM^∗10]. The presen- tation of the knowledge encoded in the explicit formaliza- tion may be realized by using ontology visualization tech- niques, which mostly come from the field of Information Vi- sualization (InfoVis) [KHL^∗07]. In the biomedical domain, Treemaps [Shn92] have been applied to the visualization of the Gene Ontology [BDBS04,ABB^∗00]. This method facil- itates the navigation of the ontology but it lacks realistic rep- resentation of each concept. In [KPM^∗08] realistic concept representations of an ontology has been proposed for the vi- sualization of hierarchical neuroanatomical structures of a mouse’ brain.

The traditional distinction of visualization techniques into SciVis and InfoVis delimitates their uses [Rhy03]. The need of overcoming this differentiation led to the trend of propos- ing new visualization classifications [Hag11] and the con- vergence of visualization techniques developed in paral- lel [Hau06]. An example in this direction is the visualiza- tion of the anatomical hierarchy integrated with volumetric data [BVG10]. Indeed, the techniques of these subfields are complementary and can be smoothly integrated in order to represent the features contained inM.

3 Representing the knowledge formalization ofM In order to have a complete understanding of a pathol- ogy with different disease features on different scales, we

need to organize heterogeneous and multi-scale information sources. We also have to take into account different relation- ships between these information sources, as well as data that represent them. To this end, we assume to have an ontol- ogy that encodes such a multi-scale biomedical knowledge, likely a complex ontology, and we want to find an effective means to visualize relevant items to be presented to a specific user within the domainM.

In our approach, relevant items are identified as results of queries to the ontology, exploiting a graph representa- tion of the ontology. More precisely, we represent ontol- ogy as a labelled directed graphG={V,E}, where nodes (V={C,I}) are concepts (C) and instances (I) of concepts, and edgesE={R,is a}, whereRare relations between in- stances and "is a" is a relation between instances and con- cepts. G is labeled with l:V 7→L, that maps nodes to the corresponding labels (L, labels of concepts, individuals and relations). For example, cellular change, loss of biome- chanical function, MRI evidence and alteration in gait pat- tern are instances of a degradation process feature, repre- sented as a graphGas depicted on Figure1. The general

Degradation process feature

Cellular change Loss of biome- chanical function

MRI evidence Alteration in gait pattern

is a is a is a is a

Figure 1: Degradation process feature (DPF).

structure of the ontology focuses on multi-scale degrada- tion process features (DPF ∈C) that may cause one an- other (cause,caused⁻¹∈R), are evidenced (evidenced∈R) by sources of evidence (SOE∈C) which in turn are mea- sured (measured∈R) by different techniques. This essen- tially formalizes propagation of degradation process features in hierarchical pathologies. The ontology supports visualiza- tion, by formalizing relationships between multi-scale data, patients, techniques, user profiles and relevant visualization items (e.g. spatio-temporal scale, visualization suitability).

Succinctly the multi-scale biomedical knowledge space is represented byGon Figure2.

4 Methodology of the integrated visualization

The proposed visualization combines a multi-scale approach and InfoVis and SciVis techniques in order to merge and present visually relevant features ofMinto the same view.

User profileWe classify data ofMaccording to theirscale of dataandvisualization suitability(Table 1).Micro-scale, medium-scale and macro-scaleclassiffy data acquired or de- rived from different spatio-temporal ranges.Abstract scale encompasses all the nonspatio-temporal knowledge that can- not be directly implied from one of the previous scales, e.g. anatomical structure or relations between evidences in a multiscalar pathology. The main scale of interest of the user is saved in theuser profile.Visualization suitabilityis the ap-

userm spatempscalep

d p fi soej

techniquek datal

viztect patienth

User profile Spatio-

temporal scale

Degradation process feature

Source of evidence

Technique Data

Visualization

technique Patient

is a is a

is a is a

is a is a

is a is a

interested causes⁻¹=caused

evidenced placed

measured causes⁻¹=caused

obtains

visualized concerns

Figure 2: Graph representation G ofM’s ontology.

Visualization suitability / Scale of Data

SciVis technique InfoVis technique User profile Micro-scale Molecular and his-

tological images, micro-CT

Properties extracted at molecular, cellu- lar and tissue level

Tissue engineer Medium-scale PET, MRI, CT and

segmentations

Radiologist Macro-scale Gait pattern anima-

tion, original video sequence

Gait pattern graphics Biomechanical engineer

Abstract scale - Anatomical struc-

ture, relation among evidences, other derived knowledge

Computer scientist, Generic

Table 1: Data structure and user profiles ofM.

propriateness of a given data set (sub-space) ofMto be rep- resented by using a concrete SciVis (e.g., volume rendering for MRI) or InfoVis technique (e.g., node-link diagram for hierarchies or bar chart for statistical data). The aforemen- tioned parameters, which are encoded in the ontology, allow the positioning of the multi-scale data on the visualization framework in a suitable way according to their properties and user interests.

Integrated visualization framework The visualization scene (Figure 3) consists of three layers: focus, context and background. Data sets from each spatio-temporal scale (i.e.

micro, medium and macro) are positioned on one of these layers, which are mainly distinguished by their z-order.Fo- cusconstitutes the main scale and data sets placed there are visualized on the front level.Contextis placed behind and its data sets are spatially aligned with the previous data to provide context to the data on the focus layer.Backgroundis the last layer, least seen and less important. Their data sets are not directly related to the focus layer data, but they com- plete the general view across all the spatio-temporal ranges (e.g., macro-scale data set in the Figure 3). Data are mainly included in nodes, which allow a consistent representation.

The actual visualization of each node depends on the visual- ization suitability (e.g. graphs, images or 3D content).

The positioning of the nodes depends on the visualization suitability of the data and the current main scale of interest.

Given the main scale, its data representation will be posi-

tioned on the focus layer, represented by a main node. This node contains the most relevant SciVis data for the scale.

InfoVis data and other specific SciVis data sets are visual- ized with subnodes. The integration of the adjacent scales depends on their spatio-temporal proximity. The most adja- cent scale is aligned in the view on the context layer, and the last one is presented on the background layer using the call- out technique. The use of this augmented representation alle- viates the differences in the order of magnitude of data, and allows the direct extraction of information from the context.

The abstract scale is represented by perceptual cues in the re- lations between nodes (labels, arrows, colored lines), enrich- ing understanding, e.g., spatial origin of sources, anatomical structures, hierarchy of evidences in a pathology.

Figure 3: Proposed visualization framework, with micro- scale as main scale.

5 Example scenario

Musculoskeletal diseases are an example of hierarchical pathologies of multi-scale nature. Figure 4represents the articular cartilage degradation during osteoarthritis. Degra- dation features on the cellular scale propagate upwards through molecular, macromolecular, and tissue scales, caus- ing finally the alteration in gait pattern [Gol12,ADC^∗08].

For a complete understanding of the process, data acquired through different techniques across different scales and their relationships have to be considered. In this scenario, the multi-scale biomedical knowledge spaceMbasically con- sists of the features of the degradation process, their re- lationships and the aforementioned information sources as evidences of the features. Ontology querying languages, such as SPARQL (SPARQL Protocol and RDF Query Lan- guage) [SP08], use graph pattern matching techniques to evaluate answers. Evaluation of queries for graph-based structures is completely out of the scope of this paper, we refer to [PAG09] for details. We only present here schemat- ically intuition for graph pattern matching techniques and how the results of these queries may be used to identify se- mantically relevant items to support visualization. For exam- ple the results to the following query, expressed in English as “Given a technique, what is theDPF evidenced by its sources of evidence?” may be obtained by evaluating graph pattern match from query graphQto the Knowledge Base graphKB(see Figure5). The answer set of mappings from

Figure 4: Degradation of articular cartilage during os- teoarthritis. Each DPF is evidenced by sources of evidence and those are measured by certain techniques. Note: this ex- ample is a particular instance ofM’s ontology.

Degradation process feature

Cellular change

Source of evidence Cell death

An./Cat.reactivation Technique Live/Death count

PCR is a

evidenced by

evidenced by is a

measured by

is a

measured by is a

is a KB

Q

?z(DPF) ?y(SOE) ?x(Technique)

evidenced by measured by

Figure 5: Query example.

variables inQto values inKB(depicted in different colors on Figure5) is summarized in the following table:

?x(Technique) ?y(SOE) ?z(DPF)

Live/Death count Cell viability Cellular change PCR An./Cat. reactivation Cellular change Queries to the ontology ofMcan also retrieve all the rele- vant information of a given patient (“Which are all theData which concern a specificpatient?"), and the techniques used (“Which is thetechniquethat obtains specificData?").

VisualizationThe support of the ontology allows for ob- taining all necessary visualization parameters for position- ing the different spatio-temporal data sets on the scene and their proper representation, as visualization suitability and scale of data (“In whichSpatio-temporal scalea givenDPF is placed?", “WhichVisualization techniquevisualizes the given Data?"). Accordingly, the data sets are represented and positioned, as described in Section4. Another important

fact is that ontology can also be queried to get information from the abstract scale, e.g. the relations between the sources of evidences which proof the different cartilage degradation process features (Figure6).

Figure 6: Integrated visualization of patient-specific data set and the cartilage degradation process. The main scale of interest is micro-scale, which shows all its data sets. The relationscausesbetweenSOEsandDPFsare visualized by blue and red arrows, respectively.

6 Conclusion and future work

The next step for the proposed system is to perform the in- teractive exploration of multi-scale data sets [MVRFW14].

This will take benefit from the visualization criteria adopted, such as scene layers or the consistent representation of data sets in nodes. Specifically, the system aims to provide two different view modes. In the2D focus view mode, the user viewport is perpendicular to the layers, and the user can manipulate the nodes, enlarge or hide them and change the view of the data set contained in the node. In the 3D overview mode, the three layers are presented in 3D perspec- tive, which allows the multi-scale navigation by sorting the layers. Another step is to better support collaborative diag- nosis in which different medical specialists work together in the same environment while preserving their habitual way of working. They augmentMwith their findings; discuss their conclusions and continue the knowledge discovery inM.

Acknowledgements

This work was supported from the EU Marie Curie ITN Mul- tiScaleHuman (FP7-PEOPLE-2011-ITN, Grant agreement no.: 289897). The authors would like to thank Marta On- dresik (3B’s Research Group) for providing the description of the example scenario (Figure 4), and all the partners for providing biomedical data sets. The two first authors Asan Agibetov and Ricardo Millán have contributed equally to the writing of the paper.

References

[ABB^∗00] ASHBURNERM., BALLC. A., BLAKEJ. A., BOT- STEIND., BUTLERH., CHERRYJ. M., DAVISA. P., DOLINSKI K., DWIGHTS. S., EPPIGJ. T., HARRISM. A., HILLD. P., ISSEL-TARVERL., KASARSKISA., LEWISS., MATESEJ. C., RICHARDSON J. E., RINGWALDM., RUBIN G. M., SHER- LOCKG.: Gene ontology: tool for the unification of biology.

Nature genetics 25, 1 (May 2000), 25–29. PMID: 10802651 PM- CID: PMC3037419.2

[ADC^∗08] ASTEPHENJ. L., DELUZIOK. J., CALDWELLG. E., DUNBARM. J., HUBLEY-KOZEYC. L.: Gait and neuromuscu- lar pattern changes are associated with differences in knee os- teoarthritis severity levels. Journal of Biomechanics 41, 4 (Jan.

2008), 868–876.3

[BCM^∗10] BAADERF., CALVANESED., MCGUINNESSD. L., NARDID., PATEL-SCHNEIDERP. F.: The Description Logic Handbook: Theory, Implementation and Applications, 2nd ed.

Cambridge University Press, New York, NY, USA, 2010.2 [BDBS04] BAEHRECKEE. H., DANGN., BABARIAK., SHNEI-

DERMANB.: Visualization and analysis of microarray and gene ontology data with treemaps. BMC Bioinformatics 5, 1 (June 2004), 84. PMID: 15222902.2

[BKMPS] BAO J., KENDALL E. F., MCGUINNESS D. L., PATEL-SCHNEIDERP. F.: OWL 2 web ontology language: W3C quick reference.2

[BVG10] BALABANIANJ.-P., VIOLAI., GRÖLLERE.: Inter- active illustrative visualization of hierarchical volume data. In Proceedings of Graphics Interface 2010(Toronto, Ont., Canada, Canada, 2010), GI ’10, Canadian Information Processing Soci- ety, pp. 137–144.2

[GOB^∗10] GEHLENBORG N., O’DONOGHUE S. I., BALIGA N. S., GOESMANN A., HIBBS M. A., KITANO H., KOHLBACHERO., NEUWEGER H., SCHNEIDER R., TENEN- BAUMD.: Visualization of omics data for systems biology.Na- ture methods 7(2010), S56–S68.2

[Gol12] GOLDRINGM. B.: Articular cartilage degradation in os- teoarthritis.HSS Journal 8, 1 (Feb. 2012), 7–9.3

[Gru93] GRUBERT. R.: A translation approach to portable ontol- ogy specifications.Knowledge acquisition 5, 2 (1993), 199–220.

2

[GSG04] GRENONP., SMITHB., GOLDBERGL.: Biodynamic ontology: applying BFO in the biomedical domain. Studies in health technology and informatics 102(2004), 20–38. PMID:

15853262.2

[Hag11] HAGENH. (Ed.):.Scientific Visualization: Interactions, Features, Metaphors (2011), vol. 2 of Dagstuhl Follow-Ups, Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, Germany.

2

[Hau06] HAUSERH.: Generalizing Focus+Context visualization.

InScientific Visualization: The Visual Extraction of Knowledge from Data, Bonneau G.-P., Ertl T., Nielson G. M., (Eds.), Mathe- matics and Visualization. Springer Berlin Heidelberg, Jan. 2006, pp. 305–327.2

[HCB^∗10] HUNTERP., COVENEYP. V., BONOB.D., DIAZV., FENNERJ., FRANGIA. F., HARRISP., HOSER., KOHL P., LAWFORDP., MCCORMACKK., MENDESM., OMHOLT S., QUARTERONIA., SKÁRJ., TEGNERJ., THOMASS. R., TOL- LISI., TSAMARDINOSI., BEEKJ. H. G. M. V., VICECONTI M.: A vision and strategy for the virtual physiological human in 2010 and beyond. Philosophical Transactions of the Royal So- ciety A: Mathematical, Physical and Engineering Sciences 368, 1920 (June 2010), 2595–2614.2

[Joh04] JOHNSONC.: Top scientific visualization research prob- lems. IEEE Computer Graphics and Applications 24, 4 (Aug.

2004), 13 – 17.2

[KHL^∗07] KATIFORIA., HALATSISC., LEPOURAS G., VAS- SILAKIS C., GIANNOPOULOU E.: Ontology visualization methods- a survey.ACM Comput. Surv. 39, 4 (Nov. 2007).2 [KPM^∗08] KUSS A., PROHASKA S., MEYERB., RYBAK J.,

HEGEH.-C.: Ontology-based visualization of hierarchical neu- roanatomical structures. InProceedings of the First Eurograph- ics conference on Visual Computing for Biomedicine (2008), pp. 177–184.2

[Lan06] LANGLOTZC. P.: RadLex: a new method for indexing online educational materials. Radiographics: a review publica- tion of the Radiological Society of North America, Inc 26, 6 (Dec.

2006), 1595–1597. PMID: 17102038.2

[MMC^∗12] MCFARLANE N. J. B., MA X., CLAPWORTHY G. J., BESSIS N., TESTID.: A survey and classification of visualisation in multiscale biomedical applications. InInforma- tion Visualisation (IV), 2012 16th International Conference on (2012), pp. 561–566.2

[MVRFW14] MILLÁNVAQUEROR. M., RZEPECKIJ., FRIESE K.-I., WOLTERF.-E.: Visualization and user interaction meth- ods for multiscale biomedical data. In3D Multiscale Physiologi- cal Human, Magnenat-Thalmann N., Ratib O., Choi H. F., (Eds.).

Springer London, Jan. 2014, pp. 107–133.4

[OGG^∗10] O’DONOGHUES. I., GAVINA. C., GEHLENBORG N., GOODSELLD. S., HÉRICHÉJ. K., NIELSENC. B., NORTH C., OLSONA. J., PROCTERJ. B., SHATTUCKD. W.: Visual- izing biological data – now and in the future. Nature methods 7 (2010), S2–S4.2

[PAG09] PÉREZJ., ARENASM., GUTIERREZC.: Semantics and complexity of sparql. ACM Trans. Database Syst. 34, 3 (Sept.

2009), 16:1–16:45.3

[Rhy03] RHYNET.-M.: Does the difference between informa- tion and scientific visualization really matter? IEEE Computer Graphics and Applications 23, 3 (June 2003), 6 – 8.2 [RM03] ROSSEC., MEJINO,JR. J. L. V.: A reference ontology

for biomedical informatics: the foundational model of anatomy.

J. of Biomedical Informatics 36, 6 (Dec. 2003), 478–500.2 [SBF98] STUDERR., BENJAMINSV., FENSELD.: Knowledge

engineering: Principles and methods. Data & Knowledge Engi- neering 25, 1–2 (Mar. 1998), 161–197.1

[SCC97] SPACKMAN K. A., CAMPBELLK. E., CÔTÉ R. A.:

SNOMED RT: a reference terminology for health care. Pro- ceedings of the AMIA Annual Fall Symposium(1997), 640–644.

PMID: 9357704 PMCID: PMC2233423.2

[Shn92] SHNEIDERMANB.: Tree visualization with tree-maps:

2-d space-filling approach.ACM Trans. Graph. 11, 1 (Jan. 1992), 92–99.2

[SP08] SEABORNEA., PRUD’HOMMEAUXE.: SPARQL Query Language for RDF. W3C recommendation, W3C, Jan- uary 2008. http://www.w3.org/TR/2008/REC-rdf-sparql-query- 20080115/.3