Print
Univis
Search
 
FAU-Logo
Techn. Fakultät Website deprecated and outdated. Click here for the new site. FAU-Logo

Dipl.-Inf. Jakob Wasza

Alumnus of the Pattern Recognition Lab of the Friedrich-Alexander-Universität Erlangen-Nürnberg

Real-time range imaging

Projects 2012

Real-time Motion Compensated Patient Positioning and Non-rigid Deformation Estimation Using 4-D Shape Priors
J.Wasza, S.Bauer, J.Hornegger
    • Over the last years, range imaging (RI) techniques have been proposed for patient positioning and respiration analysis in motion compensation. Yet, current RI based approaches for patient positioning employ rigid-body transformations, thus neglecting free-form deformations induced by respiratory motion. Furthermore, RI based respiration analysis relies on non-rigid registration techniques with run-times of several seconds. In this paper we propose a real-time framework based on RI to perform respiratory motion compensated positioning and non-rigid surface deformation estimation in a joint manner. The core of our method are pre-procedurally obtained 4-D shape priors that drive the intra-procedural alignment of the patient to the reference state, simultaneously yielding a rigid-body table transformation and a free-form deformation accounting for respiratory motion. We show that our method outperforms conventional alignment strategies by a factor of 3.0 and 2.3 in the rotation and translation accuracy, respectively. Using a GPU based implementation, we achieve run-times of 40 ms.

    Articles in Conference Proceedings
    MICCAI 2012, Part II, LNCS 7511 (15th International Conference on Medical Image Computing and Computer Assisted Intervention), Nizza, 01.10.2012, pp. 576-583, 2012 (BiBTeX, Who cited this?)
Sparse Principal Axes Statistical Surface Deformation Models for Respiration Analysis and Classification
J. Wasza, S. Bauer, S. Haase, J. Hornegger
  • Detection, analysis and compensation of respiratory motion is a key issue for a variety of medical applications, such as tumor tracking in fractionated radiotherapy. One class of approaches aims for predicting the internal target movement by correlating intra-operatively captured body surface deformations to a preoperatively learned deformable model. Here, range imaging (RI) devices assume a prominent role for dense and real-time surface acquisition due to their non-intrusive and markerless nature. 

    In this work we present an RI based statistical model built upon sparse principal axes for body surface deformations induced by respiratory motion. In contrast to commonly employed global models based on principal component analysis, we exploit orthomax rotations in order to enable the differentiation between distinctive and local respiratory motion patterns such as thoracic and abdominal breathing. In a case study, we demonstrate our model's capability to capture dense respiration curves and the usage of our model for simulating realistic distinctive respiratory motion patterns.

    Articles in Conference Proceedings
    Bildverarbeitung für die Medizin 2012 (Bildverarbeitung für die Medizin), Berlin, Germany, 20.03.2012, pp. 316-321, 2012, ISBN 978-3-642-28501-1 (BiBTeX, Who cited this?)
Photometric Estimation of 3-D Surface Motion Fields for Respiration Management
S. Bauer, J. Wasza, J. Hornegger
  • In radiation therapy, the estimation of torso deformations due to respiratory motion is an essential component for real-time tumor tracking solutions. Using range imaging (RI) sensors for continuous monitoring during the treatment, the 3-D surface motion field is reconstructed by a non-rigid registration of the patient’s instantaneous body surface to a reference. Typically, surface registration approaches rely on  the pure topology of the target. However, for RI modalities that additionally capture photometric data, we expect the registration to benefit from incorporating this secondary source of information. Hence, in this paper, we propose a method for the estimation of 3-D surface motion fields using an optical flow framework in the 2-D photometric domain. In experiments on real data from healthy volunteers, our photometric method outperformed a geometry-driven surface registration by 6.5% and 22.5% for normal and deep thoracic breathing, respectively. Both the qualitative and quantitative results indicate that the incorporation of photometric information provides a more realistic deformation estimation regarding the human respiratory system.

    Articles in Conference Proceedings
    Bildverarbeitung für die Medizin 2012 (Bildverarbeitung für die Medizin), Berlin, 19.03.2012, pp. 105-110, 2012, ISBN 978-3-642-28501-1 (BiBTeX, Who cited this?)

Projects 2011

Real-time Preprocessing for Dense 3-D Range Imaging on the GPU: Defect Interpolation, Bilateral Temporal Averaging and Guided Filtering
J. Wasza, S. Bauer, J. Hornegger
  • Recent advances in range imaging (RI) have enabled dense 3-D scene acquisition in real-time. However, due to physical limitations and the underlying range sampling principles, range data are subject to noise and may contain invalid measurements. Hence, data preprocessing is a prerequisite for practical applications but poses a challenge with respect to real-time constraints. 

    In this paper, we propose a generic and modality-independent pipeline for efficient RI data preprocessing on the graphics processing unit (GPU). The contributions of this work are efficient GPU implementations of normalized convolution for the restoration of invalid measurements, bilateral temporal averaging for dynamic scenes, and guided filtering for edge-preserving denoising. Furthermore, we show that the transformation from range measurements to 3-D world coordinates can be computed efficiently on the GPU. The pipeline has been evaluated on real data from a Time-of-Flight sensor and Microsoft's Kinect. In a run-time performance study, we show that for VGA-resolution data, our preprocessing pipeline runs at ~100 fps on an off-the-shelf consumer GPU.

    Articles in Conference Proceedings
    IEEE International Conference on Computer Vision (ICCV) Workshops (IEEE Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV)), Barcelona, Spain, 11.12.2011, pp. 1221-1227, 2011 (BiBTeX, Who cited this?)
Multi-modal Surface Registration for Markerless Initial Patient Setup in Radiation Therapy using Microsoft's Kinect Sensor
S. Bauer, J. Wasza, S. Haase, N. Marosi, J. Hornegger
  • In radiation therapy, prior to each treatment fraction, the patient must be aligned to computed tomography (CT) data. Patient setup verification systems based on range imaging (RI) can accurately verify the patient position and adjust the treatment table at a fine scale, but require an initial manual setup using lasers and skin markers. We propose a novel markerless solution that enables a fully-automatic initial coarse patient setup. The table transformation that brings template and reference data in congruence is estimated from point correspondences based on matching local surface descriptors. Inherently, this point-based registration approach is capable of coping with gross initial misalignments and partial matching. Facing the challenge of multi-modal surface registration (RI/CT), we have adapted state-of-the-art descriptors to achieve invariance to mesh resolution and robustness to variations in topology. In a case study on real data from a low-cost RI device (Microsoft Kinect), the performance of different descriptors is evaluated on anthropomorphic phantoms. Furthermore, we have investigated the system's resilience to deformations for mono-modal RI/RI registration of data from healthy volunteers. Under gross initial misalignments, our method resulted in an average angular error of 1.5° and an average translational error of 13.4 mm in RI/CT registration. This coarse patient setup provides a feasible initialization for subsequent refinement with verification systems.

    Articles in Conference Proceedings
    IEEE International Conference on Computer Vision (ICCV) Workshops (IEEE Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV)), Barcelona, Spain, 12.11.2011, pp. 1175-1181, 2011 (BiBTeX, Who cited this?)
Real-time RGB-D Mapping and 3-D Modeling on the GPU using the Random Ball Cover Data Structure
D. Neumann, F. Lugauer, S. Bauer, J. Wasza, J. Hornegger
  • The modeling of three-dimensional scene geometry from temporal point cloud streams is of particular interest for a variety of computer vision applications. With the advent of RGB-D imaging devices that deliver dense, metric and textured 6-D data in real-time, on-the-fly reconstruction of static environments has come into reach. In this paper, we propose a system for real-time point cloud mapping based on an efficient implementation of the iterative closest point (ICP) algorithm on the graphics processing unit (GPU). In order to achieve robust mappings at real-time performance, our nearest neighbor search evaluates both geometric and photometric information in a direct manner. For acceleration of the search space traversal, we exploit the inherent computing parallelism of GPUs. In this work, we have investigated the fitness of the random ball cover (RBC) data structure and search algorithm, originally proposed for high-dimensional problems, for 6-D data. In particular, we introduce a scheme that enables both fast RBC construction and queries. The proposed system is validated on an indoor scene modeling scenario. For dense data from the Microsoft Kinect sensor (640x480 px), our implementation achieved ICP runtimes of < 20 ms on an off-the-shelf consumer GPU.

    Articles in Conference Proceedings
    IEEE International Conference on Computer Vision (ICCV) Workshops (IEEE Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV)), Barcelona, Spain, 12.11.2011, pp. 1161-1167, 2011 (BiBTeX, Who cited this?)
RITK: The Range Imaging Toolkit - A Framework for 3-D Range Image Stream Processing
J. Wasza, S. Bauer, S. Haase, M. Schmid, S. Reichert, J. Hornegger
  • The recent introduction of low-cost devices for real-time acquisition of dense 3-D range imaging (RI) streams has attracted a great deal of attention. Many research communities will benefit from this evolution. However, to date, there exists no open source framework that is explicitly dedicated to real-time processing of RI streams.

    In this paper, we present the Range Imaging Toolkit (RITK). The goal is to provide a powerful yet intuitive software platform that facilitates the development of range image stream applications. In addition to its usage as a library within existing software, the toolkit supports developers with an easy-to-use development and rapid prototyping infrastructure for the creation of application-specific RITK modules or standalone solutions. Putting a strong focus on modularity, this application-layer allows for distribution and reuse of existing modules. In particular, a pool of dynamically loadable plugins can be assembled into tailored RI processing pipelines during runtime. RITK puts emphasis on real-time processing of range image streams and proposes the use of a dedicated pipeline mechanism. Furthermore, we introduce a powerful and convenient interface for range image processing on the graphics processing unit (GPU). Being designed thoroughly and in a generic manner, the toolkit is able to cope with the broad diversity of data streams provided by available RI devices and can easily be extended by custom range imaging sensors or processing modules. 

    RITK is an open source project and is publicly available underOpens internal link in current window www.cs.fau.de/ritk.

    Articles in Conference Proceedings
    Wasza, Jakob; Bauer, Sebastian; Haase, Sven; Schmid, Moritz; Reichert, Sebastian; Hornegger, Joachim
    VMV 2011: Vision, Modeling & Visualization, Berlin, Germany, 04.10.2011, pp. 57-64, 2011, ISBN 978-3-905673-85-2 (BiBTeX, Who cited this?)
High Performance GPU-based Preprocessing for Time-of-Flight Imaging in Medical Applications
J. Wasza, S. Bauer, J. Hornegger
  • Raw data
    Temporal averaging
    Edge-preserving denoising
    Defect pixel interpolation

    Time-of-Flight (ToF) imaging is a promising technology for real-time metric surface acquisition and has recently been proposed for a variety of medical applications such as intra-operative organ surface registration. However, due to limitations of the sensor, range data from ToF cameras are subject to noise and contain invalid outliers. While systematic errors can be eliminated by calibration, denoising provides the fundamental basis to produce steady and reliable surface data. 

    In this paper, we introduce a generic framework for high-performance ToF preprocessing that satisfies real-time constraints in a medical environment. The contribution of this work is threefold. First, we address the restoration of invalid measurements that typically occur with specular reflections on wet organ surfaces. Second, we compare the conventional bilateral filter with the recently introduced concept of guided image filtering for edge preserving denoising. Third, we have implemented the pipeline on the graphics processing unit (GPU), enabling high-quality preprocessing in real-time. In experiments, the framework achieved a depth accuracy of 0.8 mm (1.4 mm) on synthetic (real) data, at a total runtime of 40 ms.

    Articles in Conference Proceedings
    Bildverarbeitung für die Medizin 2011, Lübeck, 22.03.2011, pp. 324-328, 2011, ISBN 978-3-642-19335-4 (BiBTeX, Who cited this?)
Automatic Multi-modal ToF/CT Organ Surface Registration
K. Müller, S. Bauer, J. Wasza, J. Hornegger
  • In the field of image-guided liver surgery (IGLS), the initial registration of the intraoperative organ surface with preoperative tomographic image data is performed on manually selected anatomical landmarks. In this paper, we introduce a fully automatic scheme that is able to estimate the transformation for initial organ registration in a multi-modal setup aligning intraoperative time-of-flight (ToF) with preoperative computed tomography (CT) data, without manual interaction. The method consists of three stages: First, we extract geometric features that encode the local surface topology in a discriminative manner based on a novel gradient operator. Second, based on these features, point correspondences are established and deployed for estimating a coarse initial transformation. Third, we apply a conventional iterative closest point (ICP) algorithm to refine the alignment. The proposed method was evaluated for an open abdominal hepatic surgery scenario with in-vitro experiments on four porcine livers. The method achieved a mean distance of 4.82 ± 0.79 mm and 1.70 ± 0.36 mm for the coarse and fine registration, respectively.

    Articles in Conference Proceedings
    Bildverarbeitung für die Medizin 2011, Lübeck, 20.-22.03.2011, pp. 154-158, 2011, ISBN 9783642193347 (BiBTeX, Who cited this?)
4-D Photogeometric Face Recognition with Time-of-Flight Sensors
S. Bauer, J. Wasza, K. Müller, J. Hornegger
  • Methods for 2D/3D face recognition typically combine results obtained independently from the 2D and 3D data, respectively. There has not been much emphasis on data fusion at an early stage, even though it is at least potentially more powerful to exploit possible synergies between the two modalities. In this paper, we propose photogeometric features that interpret both the photometric texture and geometric shape information of 2D manifolds in a consistent manner. The 4D features encode the spatial distribution of gradients that are derived generically for any scalar field on arbitrary organized surface meshes. We apply the descriptor for biometric face recognition with a time-of-flight sensor. The method consists of three stages: (i) facial landmark localization with a HOG/SVM sliding window framework, (ii) extraction of photogeometric feature descriptors from time-of-flight data, using the inherent grayscale intensity information of the sensor as the 2D manifold's scalar field, (iii) probe matching against the gallery. Recognition based on the photogeometric features achieved 97.5% rank-1 identification rate on a comprehensive time-of-flight dataset (26 subjects, 364 facial images).

    Articles in Conference Proceedings
    IEEE Workshop on Applications of Computer Vision (WACV), Kona, HI, USA, 05.01.2011, pp. 196-203, 2011 (BiBTeX, Who cited this?)