1952 OPTICS LETTERS / Vol. 18, No. 22 / November 15, 1993

Optoelectronic morphological image processor

Javier GarciaDepartamento Interuniversitari de Optica, Universitat de Valencia, 46100 Bujassot, Spain

Tomasz SzoplikInstytut Geofizyki, Uniwersytet Warszawski, 02-093 Warszawa, Poland

Carlos FerreiraDepartamento Interuniversitari de Optica, Universitat de Valencia, 46100 Bujassot, Spain

Received June 14, 1993

A morphological optoelectronic image processor based on the threshold decomposition concept is described anddemonstrated. Binary slices of a gray-scale input image are optically convolved with a binary structuring elementof arbitrary size and shape in a noncoherent convolver. The slices are displayed on a liquid-crystal spatiallight modulator of 320 x 264 pixels. The kernels are implemented as modifications of the system impulseresponse. The processor's convolution patterns are recorded with a CCD camera and fed into a PC by a framegrabber. Subsequent elementary morphological operations are looped. Examples of processing an input imageof 256 x 256 pixels and 16 gray levels with kernels of arbitrary size are presented.

An intensive amount of image analysis and process-ing by the mathematical morphology method began inthe mid-1980's." 2 It resulted in a set-based theory ofmathematical morphology operations used for analy-sis and modification of the structure and shape of ob-jects in images. Morphological filters (MF's), whichhave been under development for the past 10 years,have nonlinear properties.3 4 Additivity and scalarmultiplication of linear shift-invariant filters in MF'sis replaced by inclusiveness. The inclusion relationmeans that any MF is an increasing transformation,and thus filtration leads to a loss of information. Incontrast, linear systems preserve information withintheir transmission bands. As a consequence, mor-phological filtering is irreversible, whereas linear fil-tering is, in principle, reversible. Another propertyof MF's is called idempotence, i.e., unchangednesswhen multiplied by itself. The same property char-acterizes nonlinear rank-order filters, which, afterseveral passes over an image can result in findingthe image roots.5 Thus MF's have different prop-erties from those of optical linear filters based onconvolution. However, the basic morphological op-erations, i.e., dilation and erosion, can be realizedby means of convolution and threshold. Thereforethe linear optical systems can perform morphologi-cal filtering if they are complemented with electron-ics in which a nonlinear thresholding operation iscarried out. The possibility of optoelectronic imple-mentations of MF's results from the study of Mara-gos and Schafer on relations among morphological,order statistics, and stack filters.6 Those authorsproved the equivalence of rank-order and stack filtersand those MF's that commute with thresholding.This condition is met in the case of both binary andgray-scale images interacting with binary structuringelements.

Let us recall some previous research devoted tooptical implementations of MF's.7-9 In one of the

earliest studies on shape- and structure-changing op-tical image filtering,' a gray-scale image was thresh-old decomposed10 into binary slices. Then the sliceswere convolved in a time sequence with a structuringelement. Proper choice of the threshold level gave ei-ther dilation or erosion. The erosion is defined as thelocus of the center of the structuring element B whenB is included in the binary slice X; i.e., in the extremecase it follows the border tangentially from inside.The dilation is defined as the locus of the center of thestructuring element when B intersects X; i.e., in theextreme case it follows the border tangentially fromoutside. Processing of a gray-scale image slice byslice is based on the threshold decomposition concept,which led to the definition of the stacking property ofBoolean functions (operators).5 Then a parallel pro-grammable mathematical morphology processor wasproposed, whereby an image was optically processedin real time.8 The erosion was performed througha dilation operation of the binary logic inversionof the input binary image. This permitted easierand more accurate thresholding. The third studypresented the principle of building an optical-digitalmathematical morphology processor by use of ferro-electric liquid-crystal light valves but did not includeexperimental results.9

In this Letter we present a morphological opto-electronic processor with feedback, which handlesrealistic size images and structuring elements of ar-bitrary size and shape. Figure 1 shows a block di-agram of the processor. An input gray-scale im-age is threshold decomposed into a stack of binaryslices. Each slice is optically convolved with a binarystructuring element, resulting in a stack of gray-scale convolution patterns. Then each convolutionpattern is thresholded on the maximum (erosion) orthe minimum (dilation) level. The resultant binaryslices are added pointwise to form the output gray-scale image.

0146-9592/93/221952-03$6.00/0 © 1993 Optical Society of America

November 15, 1993 / Vol. 18, No. 22 / OPTICS LETTERS 1953

OpticalConvolution

Fig. 1. Block diagram of the morphological optoelec-tronic image processor. Operations are shown in circles,and data arrays are shown in squares.

Figure 2 shows the geometry of the optical con-volver. Convolution by use of a plane of misfocus isone of a few possible methods of optical convolving."The camera lens images a slice displayed on thespatial light modulator (SLM) onto the CCD cam-era. We control the system point-spread function bymisfocusing and by using a diaphragm. When theoutput image is recorded in a plane that is shiftedfrom the plane of focus (plane of misfocus), the systempoint-spread function becomes wider, and it can beshaped by the diaphragm. The SLM used in theexperiments is an Epson liquid-crystal light valvewith a resolution of 320 X 264 pixels.

The performance of the processor is demonstratedon the input image shown in Fig. 3(a). We beginwith the morphological opening yB(X) = 3 BEB(X),where X is the image slice and eB is the erosionof X by structuring element B followed by dilation8B of the looped eroded slice by the same kernel.We obtain the opening of a gray-scale image yB(Im)by stacking the binary processed slices. Figure 3(b)shows the result of opening by use of a square binarykernel of 5 X 5 pixels with no corner pixels. Eachconvolution operation is performed in the optical con-volver in such a way that the output image at eachstep is a convolution of the input image with thesystem point-spread function, which corresponds tothe structuring element shaped by the diaphragm.The convolution of a binary slice and a binary kernelhas 256 gray levels; thus it should be thresholded atthe 255 and 0 levels for erosion and dilation, respec-tively. However, in an optoelectronic system withdiffused illumination, composed of a liquid-crystalSLM, a CCD camera, and a frame grabber, there areseveral sources of noise that enters the informationchannel. First, analog signals are resampled threetimes with different resolutions on their loop betweenthe frame grabber, the SLM, and the CCD camera.Second, nonlinearities of the CCD camera and theSLM reduce the dynamic range of convolved slices.As a result, the processed data arrays are corruptedwith nonnegligible noise. Therefore in the experi-ment the threshold levels differ from the minimumand the maximum by a few percent of the recorded

range. This introduces an error, which can be esti-mated in the second example by comparison of theresults obtained in our processor with those digitallycalculated.

The second example is the morphological gradientgB(Im) = 8 B(Im) - eB(Im) of the gray-scale image,where a minus denotes a pointwise subtraction ofdilated and eroded images that we obtained by stack-ing the binary processed slices. Figure 3(c) showsthe morphological gradient calculated in the opto-electronic processor by use of a square 3 X 3 pixelkernel. It is quite similar to the morphological gra-dient shown in Fig. 3(d), which is digitally calculatedfor the same input image and a kernel of the samesize and shape.

In the proposed processor, the efficient optoelec-tronic nonlinear processing of binary and gray-scaleimages by function- and set-processing MF's'2 canalso be extended to rank-order filters. All the above-mentioned nonlinear filters require total or partialknowledge of local histograms. Therefore we recallan easy method of histogram calculation that can beused in the described processor. A theorem proposedin Ref. 13 says that the local q-level histogram ofthe central pixel of an arbitrary neighborhood inan input image is equal to the pointwise differenceof the two convolution patterns that result fromconvolving the slices at the levels q + 1 and q witha binary mask, thereby defining the neighborhood.Recently the theorem was extended to the case ofweighted neighborhoods, which correspond to gray-scale structuring elements. This extended theoremsays that the local q-level histogram of the centralpixel of a weighted neighborhood in an input imageis equal to the pointwise difference of the twoconvolution patterns that result from convolving theslices at the levels q + 1 and q with a gray-scaleinteger-valued mask."4 Both theorems result fromthe fact that, for each pixel of the input image, thepixel value in the convolution pattern of the q-levelbinary slice and the kernel is equal to the numberof pixels in the neighborhood that are above theq-level value. This number of pixels, subtractedfrom the number of pixels in the neighborhooddefined by the binary kernel used for convolution,is equal to the local rank-order function at the same

HALOGEN DIFFUSER CAMERA CCDLAMP LENS CAMERA

CROSSEDPOLARIZERS

PCCOMPUTER

Fig. 2. Optoelectronic setup of the morphological proces-sor with feedback. The convolver, which uses a planeof misfocus, convolves binary slices of an input imagedisplayed on a liquid-crystal SLM with binary structuringelements of controlled size and shape. The PC is usedas image slicer, frame grabber, thresholder, pointwiseoperator, and postprocessor.

1954 OPTICS LETTERS / Vol. 18, No. 22 / November 15, 1993

(a)

(b)

(C)

(d)Fig. 3. Experimental results: (a) The input image of 256 x 256 pixels with 16 gray levels. (b) Morphological openingby use of a square binary kernel of 5 x 5 pixels with no corner pixels. (c) Morphological gradient by use of a square binarykernel of 3 x 3 pixels. (d) Digitally calculated morphological gradient by use of a square binary kernel of 3 X 3 pixels.

q level. The local rank-order function is also calledthe local cumulative histogram.

This study was supported by Spanish grant CICYT-TAP93-0667-C03-03. J. Garcia acknowledges agrant from the Ministerio de Educaci6n y Ciencia,Spain. T. Szoplik acknowledges a grant from theGeneralitat Valenciana, Conselleria de Cultura,Educaci6n y Ciencia.

References1. G. Matheron, Random Sets and Integral Geometry

(Wiley, New York, 1975).2. J. Serra, Image Analysis and Mathematical Morphol-

ogy (Academic, London, 1982).3. J. Serra, ed., Image Analysis and Mathematical

Morphology, Theoretical Advances (Academic, London,1988).

4. E. R. Dougherty, ed., Mathematical Morphology inImage Processing (Dekker, New York, 1993).

5. I. Pitas and A. N. Venetsanopoulos, Nonlinear Digi-tal Filters. Principles and Applications (Kluwer, Dor-drecht, The Netherlands, 1990).

6. P. Maragos and R. W. Schafer, IEEE Trans. Acoust.Speech Signal Process. ASSP-35, 1170 (1987).

7. J. M. Hereford and W. T. Rhodes, Opt. Eng. 27, 274(1988).

8. Y. Li, A. Kostrzewski, D. H. Kim, and G. Eichmann,Opt. Lett. 14, 981 (1989).

9. P. Cambon and J. L. De Bougrenet de la Tocknaye,Appl. Opt. 28, 3456 (1989).

10. J. P. Fitch, E. J. Coyle, and N. C. Gallagher, Jr., IEEETrans. Acoust. Speech Signal Process. ASSP-32, 1183(1984).

11. J. Knopp and M. F. Becker, Appl. Opt. 17, 984 (1978).12. P. Maragos, Opt. Eng. 26, 623 (1987).13. V. Kober, T. Cichocki, M. Gedziorowski, and T.

Szoplik, Appl. Opt. 32, 692 (1993).14. V. Kober, J. Garcia, T. Szoplik, and L. P. Yaroslavsky,

Int. J. Opt. Comput. 2, 367 (1993).