Aim

Multimodal  rendering is an  important requirement of medical imaging systems,  since the correlation  of  images from different  modalities provides  important    clues on   the presence of  pathologies  and dysfunctions. The aim of this research line is to investigate 3D direct volume rendering of multimodal dasets.

We started this research line by analyzing how and where in the rendering pipeline should data fusion be done, how rendering algorithms can be adapted to multimodal datasets and how they can be speed up .  We proposed a new query-driven splatting method based on a decision tree where users can select specific combinations of data.  Rendering is speeded-up by using a  run-length encoding of the data combinations that allows us skipping unselected regions of the data.

More recently, we designed a multimodal rendering method based on 3D texture mapping that supports differents fusions of properties, materials and colors, along with normal-based fusion given a distance map model of a feature surface. We extended this method to support time-varying data. Finally, we showed that  multimodal rendering can be extended to perform structural Focus+Context rendering of pre-classified datasets.

Topics

• Registration of volume models by surface extraction and alignment plus revoxelization
• Categorization and comparative analysis of fusion modalities
  
• Extension of volume rendering algorithm to multimodal datsets
  
• Efficient traversal of a multimodal set using a decision tree
• Design of rendering software for multimodality
• 3D texture mapping of multimodal datasets
• 3D texture-mapping of multimodal tatic and time-varying datasets
• Structural Focus+Context rendering of pre-classified data using 3D multomodal texture mapping
  

Funding

• Project: TIC1999-1230-C02-02 from the Ministerio de Ciencia y Tecnología of the spanish government
• Project: MAT2002-0497-C03-02 from the Ministerio de Ciencia y Tecnología of the spanish government
• Project: Red IM3 from the Fondo de Investigación Sanitaria of the spanish government
• Project: MAT2005-07244-C03 from the Ministerio de Ciencia y Tecnología of the spanish government

Results

• [APT08]: Structural Focus+Context Rendering of biomedical data, P. Abellán, A.Puig and D. Tost, VCBM, 2008
• [AbT08]:Multimodal rendering with 3D texture mapping, P. Abellán, D. Tost,Computers & Graphics, 32, 412-419 2008
• [AGT08]:Time-varying multimodal rendering with 3D-Texture mapping, P. Abellán, S.Grau, D. Tost, GRAPP'08, pp.223-230, 2008
  
• [FPT06]:Decision trees for accelerating unimodal, hybrid and multimodal rendering models, M. Ferré, A. Puig, D. Tost, The Visual Computer, Vo. 22, Nº 3, pp.158-167, 2006
• [FPT04a]: A fast hierarchical traversal strategy for multimodal visualization, M. Ferré, A. Puig, D. Tost, Visualization and Data Analysis 2004, Proceedings of SPIE-IS&T Electronic Imaging, SPIE Vol. 5295, page 1-13, 2004
• [FPT04b]: A framework for fusion methods and rendering techniques of multimodal volume data,  M. Ferré, A. Puig, D. Tost, Computer Animation and Virtual Worlds, Volume 15, page 63-77, 2004
• [FPT04c]: Using a classification tree to speed up rendering on hybrid surface and volume models, M. Ferré, A. Puig, D. Tost, Journal of WSCG, Volume 12, page 105-112, 2004
• [FMP03]: Segmentació d'imatges de Resonància magnètica, M. Ferré, A. Martínez, D. Puig, Technical Report, DEIM-RT-03-005, URV, pages 47, 2003
• [PTF02]: Design of a Multimodal Rendering System, A. Puig, D. Tost, M. Ferré, Vision, Modeling and Visualization 2002, Erlangen, Germany, page 489-496, 2002
  
• [FPT02]: Rendering Techniques for Multimodal Data, M. Ferré, A. Puig, D. Tost, SIACG 2002, Guimaraes, Portugal, page 305-313, 2002
• [TPF02a]: Multimodal rendering of SPECT and MR data of the brain, D. Tost, A. Puig, M. Ferré, Terceres Jornades de Recerca en Enginyeria Biomèdica, Xarxa Temàtica en Enginyeria Biomèdica, page 19-25,2002
• [TPF02b]: Visual Clues in Multimodal Rendering, D. Tost, A. Puig, M. Ferré, Research Report LSI-02-38-R, pages 38, 2002
  
• [FeT01]: Integration of Multimodal Data based on Surface Registration, M. Ferré, D. Tost, METMBS'01, page 73-78, 2001
• [FeT00]: Alineación de imágenes cerebrales en 3D con registros multimodales, M. Ferré, D. Tost, X Congreso Español de Informática Gráfica, CEIG 2000, page 341-353, 2000
• [FeT99]: Integration, Modeling and Visualization of Multimodal Data of the Human Brain, M. Ferré, D. Tost, Research Report LSI-99-31-R, DEI-RR-99-004, pages 35, 1999

Contact person

• Dani Tost:  dani at lsi.upc.edu