ORIGINAL ARTICLE Annals of Nuclear Medicine Vol. 15, No. 6, 521-525, 2001 Receiver operating characteristic (ROC) analysis of images reconstructed with iterative expectation maximization algorithms Yasuyuki TAKAHASHI,* Kenya MURASE,* Hiroshi HIGASHINO,** Ichiro SOGABE** and Kana SAKAMOTO** *Department of Medical Engineering, Division of Allied Health Sciences, Osaka Graduate Medical School **Department of Radiology, Ehime Prefectural Imabari Hospital Purpose: The quality of images reconstructed by means of the maximum likelihood-expectation maximization (ML-EM) and ordered subset (OS)-EM algorithms, was examined with parameters such as the number of iterations and subsets, then compared with the quality of images reconstructed by the filtered back projection method. Methods: Phantoms showing signals inside signals, which mimicked single-photon emission computed tomography (SPECT) images of cerebral blood flow and myocardial perfusion, and phantoms showing signals around the signals obtained by SPECT of bone and tumor were used for experiments. To determine signals for recognition, SPECT images in which the signals could be appropriately recognized with a combination of fewer iterations and subsets of different sizes and densities were evaluated by receiver operating characteristic (ROC) analysis. The results of ROC analysis were applied to myocardial phantom experiments and scintigraphy of myocardial perfusion. Results: Taking the image processing time into consideration, good SPECT images were obtained by OS-EM at iteration No. 10 and subset 5. Conclusions: This study will be helpful for selection of parameters such as the number of iterations and subsets when using the ML-EM or OS-EM algorithms. Key words: maximum likelihood estimate-expectation maximization, ordered subsets-expectation maximization, receiver operation characteristics INTRODUCTION RECONSTRUCTION of single-photon emission computed tomography (SPECT) images is mainly performed by the filtered back projection method. With the recent spread of workstations of improved processing ability, reconstruction of images by the iterative image reconstruction method, which was rarely used for practical purposes due to long computation time, has become possible. In this study, receiver operating characteristic (ROC)1,2 analysis was used to analyze experiments with phantoms, and the Received June 13, 2001 , revision accepted October 3, 2001 . For reprint contact: Yasuyuki Takahashi, Department of Radiology, Ehime Prefectural Imabari Hospital, 4-5-5, Ishiicho, Imabari. Ehime 794-0006, JAPAN. E-mail: i-y-taka@epnh.pref.ehime.jp iterative image reconstruction methods such as the maximum likelihood-expectation maximization (ML-EM)3,4. and ordered subset (OS)-EM5-7 algorithms which give good SPECT images with fewer operations were evaluated by varying the combination of iterations and subsets. The SPECT images reconstructed by the iterative image reconstruction methods were compared with those obtained by the filtered back projection (FBP) method. MATERIALS AND METHODS Filtered back projection method This method is a convolutional integration of data obtained from each projection direction with the reconstruction filter and reverse projection of the integrated data in each direction 8 The means of the sums of projection data in the reverse direction are used. Iterative expectation maximization algorithms An arbitrary value is assumed (1 in this study), and projection values are calculated by integrating the density of pixels in each direction. Differences are corrected by comparing them with the projection values in each direction. In the ML-EM method, if projection data contain statistical noise showing the Poisson distribution, the most probable reconstructed images are made, taking the noise into consideration, but because convergence is slow by this method, the OS-EM method is used for faster acquisition. In this OS-EM method, after classification of projection data into several subsets, a reconstructed image is calculated by using the first subset, which is used as the initial value for the next subset. Such calculation is repeatedly performed with all the remaining subsets, which completes one round of successive approximation. Theoretically, OS-EM includes a correction for nonuniform absorption, but this feature was eliminated from this study. Basic study The ability to recognize signals inside signals, which mimicked SPECT images of cerebral blood flow and myocardial perfusion, was evaluated in experiments by using column phantoms. The signal density of 1.00 (accumulation rate of 99mTc-HM-PAO, 37 MBq) was established in a 188 x 138 mmfai elliptical phantom (IB-10), and densities of 2.00, 1.00, 0.50 and 0.25 were established in a 15 mmpai phantom for internal signals (Fig. 1 left). The ability to recognize signals around signals, which mimicked SPECT images of bone and tumor, was evaluated. Signal components of different sizes (20, 15, 10 mmphai) and different densities ( 1.00, 0.50, 0.25, 0.00) were established outside a 200 x 200 mmphai column phantom (AZ-660; signal density, 1.00) (Fig. 1 right). Iteration numbers of ML-EM were 1, 10 to 90 at intervals of 10 (including No. 25), whereas in OS-EM, subsets 3 and 5 with iteration numbers 1 , 10, 25 and 50 and subsets 10 with iteration No. 5 were used (total, 20). Reconstructed images were also prepared by FBP. Myocardial phantom experiments and myocardial perfusion scintigraphy were performed. In experiments with a myocardial phantom (RH-2), images reconstructed by ML-EM with iteration No. 99 of projection data obtained by collecting reference data for 1 hour were evaluated. In about 15 images reconstructed at about 50-70 counts/pixel in every collection direction, the numbers of iterations and subsets of ML-EM and OS-EM that gave similar levels of the image quality and the SPECT value obtained with reference data were evaluated, and the similarity to the results of ROC analysis was also evaluated. The combination of iterations and subsets in OS-EM was the same as in the column phantom. In the myocardial phantom, a 20 mmphai infarction was established on the front wall, and a high accumulation site in the liver at the same density was established 1 cm below the lower wall. A healthy 30-year-old man was clinically examined by scintigraphy of myocardial perfusion at rest 30 min after intravenous injection of 99mTc-tetrofosmin 555 MBq. SPECT apparatus and conditions The apparatus used was Toshiba (Tokyo, Japan), GCA-9300A/UI, with a low-energy high-resolution collimator, and a data processor (Toshiba, GMS-5500A/PI, Tokyo, Japan). Data were collected mainly with a matrix size of 128 x 128, in 60 directions at 6-degree intervals (360 degrees collection) and a pixel size of 3.2 mm. The projections were reconstructed with a ramp convolution filter, and high frequency noise was decreased with post-reconstruction Butterworth filtering (cut off = 0.44 cycle/cm, power factor = 8). Attenuation and scatter correction was not performed. Statistical analysis Images were assessed by the multiple-observer confidence rating method at five levels: the signal is (1) not present, (2) improbably present, (3) unknown, (4) probably present, and (5) present. An area test was performed simultaneously by 30 observers with random sampling and random allocation. The continuous confidence rating method was omitted because the signal might move in this experiment. In ROC analysis, sensitivity values obtained are plotted on the abscissa and specificity values on the ordinate, according to the positiveness and negativeness of some points. In othe words, the Y-axis means the true positive rate and the X-axis the pseudo positive rate. The area under the ROC curve was calculated on the basis of the non-parametric assumption and the larger value was taken as significant. If curves cross each other, a comparison is usually made between the two higher types of true positive rate (TPR) and false positive rate (FPR). In this study, only the area under the curve (AUC) was used. RESULTS Signals inside the signals Figure 2 shows the results at a density of 0.50 with a basal density of 1.00. In ML-EM, better results were obtained with increased iteration (Fig. 2 (a)), reaching a stable level at iteration 30 and larger. In OS-EM, images were compared at subset 5 by changing the number of iterations, and iteration No. 10 was found to be good (Fig. 2 (b)). In the images reconstructed by ML-EM (iteration No. 50) and OS-EM (iteration No. 10, subset 5; iteration No. 5, subset 10) keeping iteration No. x subsets = 50, OS-EM with iteration No. 10 and subset 5 gave good results (Fig. 2 (c)). At iteration No. 10, subset 3 was slightly better than subset 5 (no significant difference between A and Z) (Fig. 2 (d)). Signals around the signals Figure 3 shows the results for 15 mmphai at the same density of the signal component. In ML-EM, the recognizability of signals was much better with increased iteration of successive approximation (Fig. 3 (a)). In OS-EM, evaluation was performed in the same manner as in the signals inside the signals, and the best SPECT images were often obtained at subset 5 and iteration No. 10 (Fig. 3 (b-d)). The recognizability of signals at different densities gave similar results. These results of ROC analysis of both signals inside and around the signals suggested that SPECT images obtained at subset 5 and iteration No. 10 were good (no significant difference between ML-EM and OS-EM). Myocardial phantom Imaging of homogeneity of the myocardial region and the infarction site on the anterior was examined with the Bull's eye map and short axis images (Fig. 4). The images of the minimum (anterior)/maximum (lateral) ratio (Fig. 5 (a)) and the mean of the entire Bull's eye map (Fig. 5 (b)) in the scheme indicated at the center of the cardiac muscles including defective regions were similar to the reference image obtained by ML-EM at about iteration No. 40, which agreed with the results of ROC analysis. The SPECT value obtained by FBP was negative in the cardiac lumen on the profile curve (anterior-posterior) shown in short axis images (Fig. 5 (c)), was not observed with ML-EM or OS-EM (Fig. 5 (d)). Considering iteration No. x subsets = 50 in ROC analysis, the images obtained by ML-EM (iteration No. 50) and OS-EM (iteration No. 10, subset 5; iteration No. 5, subset 10) were compared, but no clear difference was detected. Case report In a healthy subject, the Bull's eye maps and short axis images (Fig. 6) obtained by FBP, ML-EM (iteration No. 50) and OS-EM (iteration No. 10, subset 5; iteration No. 5, subset 10) were compared. The interventricular septum was better imaged by ML-EM and OS-EM than by FBP. In the Bull's eye count map (24 divisions, mean +- SD), the contrast was higher in ML-EM (85.65 +- 11.09) and OS-EM (iteration No. 10, subset 5; 85.39 +- 11.21 : iteration No. 5, subset 10; 85.30 +- 11.32) than by FBP (82.35 +- 12.06). DISCUSSION The OS-EM method used in this study is to access one subset consisting of projection data in the reverse direction at an angle showing a large division between the symmetric and the following subset,9 and it was reported that fast acquisition is possible by this method. As seen in our results, pixel values determined by the filtered back projection method are sometimes negative due to noise, Io but convergence is assured in the ML-EM algorithm and necessarily limited to the non-negative range because approximation starts from images with a positive value, resulting in a reduction in noise in regions of low density. The simulation to which the noise level, etc. was changed has previously been reported as a factor in SPECT images.11,12 In the experiments with phantoms, the combination of iteration and subsets in OS-EM, which can appropriately recognize signals with fewer operations, was examined by ROC analysis. There have been few studies on optimal conditions for the OS-EM method because it greatly depends on hardware processing time for use in routine examinations. In this study, ROC analysis of the ability to recognize signals both inside and around other signals indicated that reconstruction of images was good at iteration No. 10 and subset 5. Hudson et al.13 reported that the number of projection data in subsets was established according to the geometry of detectors, that is, 3 subsets is appropriate for SPECT apparatus with 3 detectors. In the present study, ROC analysis of the experiments performed at a constant iteration of No. 10 and at different subsets 1, 3 and 5 revealed that the ability to recognize signals was best at subset 5, which requires more operations. Clinically, there were no significant differences in recognizability between subsets 1 , 5 and 10 at iteration No. x subsets = 50. These results indicated that it is necessary to establish conditions appropriate to each scintigraphy in each institution, considering the processing time for image reconstruction by successive approximation. We believe that this study will be helpful for in selecting parameters such as the number of iterations and subsets when using the ML-EM or OS-EM algorithms. REFERENCES 1. Metz CE, Herman BA, Shen JH. Maximum likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data. Statist Med 1998; 17: 1033-1053. 2. Farquhar TH, Llacer J, Hoh CK, Czernin J, Gambhir SS, Seltzer MA, et al. ROC and localization ROC analyses of lesion detection in whole-body FDG PET: effects of acquisition mode, attenuation correction and reconstruction algorithm. J Nucl Aied 1999; 40: 2043-2052. 3. Sheep LA, Vardi Y. Maximum likelihood reconstruction for emission tomography. IEEE Trans Med Imag 1982; MI-1: 113-122. 4. Murase K, Tanada S, Inoue T. Sugawara Y, Hamamoto K. Improvement of brain single photon emission tomography (SPET) using transmission data acquisition in a four-head SPET scanner. Eur J Nucl Med 1993; 20: 32-38. 5. Hudson HM, Hutton BF, Larkin R. Accelerated EM reconstruction using ordered subsets (abstract). J Nucl Med 1992; 33: 960. 6. Murase K, Tanada S, Sugawara Y, Tauxe WN, Hamamoto K. An evaluation of the accelerated expectation maximization algorithms for single-photon emission tomography image reconstruction. Eur J Nucl Med 1994; 21 : 597-603. 7. Bai J, Hashimoto J, Suzuki T, Nakahara T, Kubo A, Iwanaga S, et al. Comparison of image reconstruction algorithms in myocardial perfusion scintigraphy. Ann Nucl Med 2001 ; 15: 79-83. 8. Bellini S, Piacentini M. Cafforio C, Rocca F. Compensation of tissue absorption in emission tomography. IEEE Trans Acoustics, Speech and Signal Processing 1979; ASSP-27: 213-218. 9. Li J, Jaszczak RJ, Greer KL, Coleman RE. Implementation of an accelerated iterative algorithm for cone-beam SPECT. Phys Med Biol 1994; 39: 643-653. 10. Murayama H, Tanaka E, Toyama H. Statistical noise with the weighted backprojection method for single photon emission computed tomography. KAKU IGAKU (Jpn J Nucl Med) 1985; 22: 307-319. 11. Ogawa K, Takahashi M. Selection of projection sets and order of calculation in ordered subsets expectation maximization method. IEICE Transactions 1999; J82-D-II: 1093-1226. 12. Shinohara H, Yamamoto T, Niio Y, Kuniyasu Y, Hayashi T, Obuchi M, et al. Application of ML-EM algorithm in nuclear medicine. Jpn J Med Phys 1999; 19: 174-183. 13. Hudson HM, Larkin RS. Accelerated image reconstruction using ordered subsets of projection data. IEEE Trans Med Imag 1994; 13: 601-609.