Fast Pixelated Detectors: A New Era for STEM

  • Summary

JEOL NEWS Vol.53 No.1 Peter D Nellist and Gerardo T Martinez
Department of Materials, University of Oxford

Conventional detectors for the scanning transmission electron microscope (STEM), such as the annular dark-field or annular bright-field detectors, integrate the intensity in the detector plane of the STEM over a range of scattering angles. In doing so, they neglect the rich information present in the intensity variations in the detector plane. Here we demonstrate that a recently developed fast pixelated detector, the JEOL 4DCanvas™ system, now allows the routine recording of the full four-dimensional STEM imaging data set. The four dimensions consist of the two real-space coordinates corresponding to the illuminating probe position, and the two reciprocal-space coordinates corresponding to position in the detector plane. Because the STEM probe pixel dwell time is now limited by the frame speed of the camera, one of the key developments is high frame rates of greater than 1000 frames per second. A second key development is direct electron detection with single electron sensitivity. We show that the 4D data set can be used to synthesize a range of STEM imaging modes from a single scan. We go on to use ptychography to retrieve the phase shift of the transmitted beam, showing how efficient phase imaging can now be achieved in STEM. It is shown that aberration correction is possible post-acquisition, and we explore the effects of dynamical scattering from heavier samples.


Over the past two decades, the scanning transmission electron microscope (STEM) has become the instrument of choice for atomic resolution imaging and spectroscopy studies of materials, especially where quantitative information is required. There are two main reasons for this: (i) STEM allows for simultaneous imaging and spectroscopy revealing structure, composition and bonding at atomic resolution. (ii) The commonly-used imaging STEM modes are incoherent which leads to easier interpretation of the data [1]. The most commonly-used imaging mode makes use of an annular dark-field (ADF) detector to detect the intensity of the scattering to relatively high angles. The resulting ADF images show both an incoherent nature and compositional sensitivity and are therefore are a very powerful way of imaging materials [2]. In ADF STEM imaging, the total intensity incident upon the entire ADF detector is summed to give a value for the image pixel corresponding to the probe position. Any detail or variation of intensity in the STEM detector plane within the collection area of the detector is therefore lost. In this paper we explore how such intensity variations can be used in STEM, in particular through the use of ptychography to provide phase imaging.
Before the widespread availability of high-resolution STEM instruments, atomic resolution imaging was performed using phase contrast imaging in a conventional TEM (CTEM), a technique which became referred to as high-resolution TEM (HRTEM) [3]. In such images, dynamical scattering of electrons in the sample and changes in the precise imaging parameters can strongly affect the image, including leading to contrast reversals where it is not immediately clear whether the atoms or atomic columns appear as dark or bright contrast (see for example [4]). In contrast, the incoherent nature of ADF STEM always leads to bright peaks for atoms or atomic columns.
Samples, such as graphene, that are thin and contain light elements are much more efficiently imaged in HRTEM compared to ADF STEM because the electron scattering from such samples can be regarded as only producing a small phase shift in the transmitted electron wave. It can be shown that weak phase objects produce very little signal in ADF STEM [5], whereas in the CTEM aberrations can be used to form a virtual phase plate that allow weak phase contrast imaging. The importance of phase contrast imaging in the CTEM has been highlighted in the biological imaging field by the award of the 2017 Nobel Prize in Chemistry for the development of cryo-EM. The imaging used for single particle analysis in biological imaging is phase contrast imaging, and indeed this has been the driver for the development of phase plates to enhance the phase contrast [6].
By the principle of reciprocity [7, 8], the configuration of the STEM detector plays the same role as the illumination configuration in a CTEM. As shown in Fig. 1, a small, axial bright-field (BF) detector in STEM is equivalent to highly parallel, axial illumination in the CTEM. A larger STEM detector is equivalent to a more highly convergent incoherent beam in the CTEM. The ADF detector in STEM is somewhat similar to hollow-cone illumination in CTEM. For HRTEM phase contrast imaging, a highly parallel, high-coherence beam is required, which is equivalent to the small axial STEM detector. It is now clear why the CTEM is more efficient than the STEM for HRTEM work: in the CTEM, a parallel beam illuminates the sample, and the scattering up to some angle corresponding to the numerical aperture of the lens is collected and imaged with the majority of the scattered electrons being detected. In the STEM, a highly convergent beam illuminates the sample but only electrons transmitted to a small axial detector are collected, so only a small minority of the transmitted electrons are detected. For radiation-sensitive materials where the efficiency of the imaging process is crucial, BF STEM imaging is not optimal.
In this paper we consider a fast pixelated detector (FPD) that records a two-dimensional intensity map of the STEM detector plane for each probe position in a two-dimensional scan, resulting in a four-dimensional (4D) data-set that can be regarded as the universal STEM imaging data set. We show that the 4D data set allows quantitative phase imaging, and because all the transmitted electrons are detected is very electron-efficient allowing relatively low-dose imaging.

Fig. 1

A comparison of the imaging configuration for phase contrast imaging in the CTEM and the STEM demonstrating the principle of reciprocity. In the CTEM, a small illuminating aperture is used to provide close to parallel illumination. The convergence angle of the beam is much smaller than the acceptance angle (numerical aperture) of the objective lens. All the unscattered and much of the scattered electrons are therefore detected. By reciprocity, the equivalent for STEM is a small bright-field detector that is much smaller than the unscattered, bright-field disc in the detector plane. Much of the unscattered and scattered electrons are therefore not detected, and is therefore not an efficient use of electrons.

Experimental Details

When recording the 4D STEM data set, the probe pixel dwell time in the scan is limited by the frame-speed of the detector. Typical STEM dwell times are less than 100 μs, and so detectors with extremely high frame rates are required. The detector should also offer single electron sensitivity with a high detector quantum efficiency. The work presented here was all recorded using a JEOL 4DCanvas™ system [9] mounted on a JEOL JEM-ARM200F cold field emission STEM system fitted with a probe optics aberration corrector. The 4DCanvas™ is a highly sensitive multi-channel STEM detector with channels of dimension 264 × 264. It can be read out at a speed of 1,000 frames per second (fps), or commensurately faster with binning (for example 4 by 1 binning gives 4,000 fps). The sensor of this pixelated detector is a direct electron detection charge coupled device. The Oxford system is shown in Fig. 2.

Fig. 2

A photograph of the JEOL JEM-ARM200F instrument at the Department of Materials in Oxford along with a photograph of the JEOL 4DCanvas™ system as fitted to the microscope.


Synthesis of conventional STEM images

We start by considering the imaging of the edge of a sample of Pt prepared by focused ion-beam lift-out and oriented along the <110> direction. A 4D data set was recorded at a beam energy of 200 keV from a 512 by 512 probe scan area with the detector operating without binning at 1,000 fps. Figure 3a shows a single detector image frame. The single electron detection is clear in the image. Summing over all probe positions to give a position-averaged convergent beam electron diffraction pattern (PACBED)(Figures 3b and 3c) shows the usual form of a convergent beam electron diffraction pattern.
From this data set, images from a range of different STEM detectors can be synthesized. This is achieved by integrating the 4D data set over the desired detector geometry in the detector plane of the data, resulting in a 2D image. Figure 4 shows the images from the incoherent bright field (IBF), annular bright-field (ABF), annular dark-field (ADF) and low-angle annular dark-field (LAADF) geometries with their integration regions displayed using the PACBED intensities. In particular it can be seen how the LAADF image shows a "halo" type contrast. This can be explained by considering that the LAADF intensity will maximize when the BF disc is at its maximum deflection, which will occur when the probe is slightly displaced from the centre of an atomic column and the illuminating electrons are experiencing the maximum net electric field, similar to the effect seen for differential phase contrast imaging [10] and similar to the effect seen for first-moment imaging [11].

Fig. 3

Data recorded from the 4DCanvasTM system during a 512 by 512 probe position scan over a sample of Pt <110> with the camera operating in full-frame mode at 1,000 fps. (a) A single frame where the bright points of intensity represent single electrons being detected. (b) The sum of diffraction patterns from the entire scanned area to form a position averaged CBED (PACBED) pattern.(c) The logarithm of the intensity of the PACBED pattern so that the Kikuchi lines are visible. The shadow of the JEOL ADF1 detector is also visible.

Fig. 4

Synthesised STEM images from the data recorded in Fig. 3. (a), (c), (e) and (g) show images for IBF, ABF, ADF and LAADF respectively, with the integration regions over the detector illustrated in (b)(d)(f)(h) respectively.

Phase imaging through ptychography

In addition to allowing a flexible choice of imaging detector geometries that can be selected post-acquisition, the 4D data set creates a range of opportunities for new imaging modes that are only just starting to be explored. One such new mode is phase imaging through electron ptychography. Ptychography was proposed by Hoppe [12] as a method to solve the phase problem in electron diffraction, and was demonstrated experimentally in the early 1990s in the context of focused-probe STEM by Rodenburg and co-workers [13, 14]. At that time, camera and computing technical capabilities severely limited the technique, and images with typically only 32 by 32 pixels were achieved. The development of FPDs has enabled ptychography to become a viable and powerful technique in STEM. The 4DCanvasTM system installed on a JEM-ARM200F STEM at Oxford was the instrumentation on which ptychography was first used to solve the previously unknown structure of a recently synthesized material [15].
As described in [15], ptychography makes use of overlapping discs in a coherent convergent beam electron diffraction pattern. In the STEM configuration, the sample is illuminated by a highly convergent beam that is focused to form the probe. For a crystalline sample, the diffracted beams will form discs in the STEM detector plane, and in the overlap between these discs, coherent interference will occur. The resulting intensity will depend on the phase of the diffracted beams, any aberrations in the probe-forming optics, and the probe position. As the probe is scanned, the intensity in the disc overlap regions will fluctuate. Indeed, it is this fluctuation that is the origin of lattice contrast in any STEM image. Assuming the aberrations are corrected to zero, the phase of this fluctuation with respect to probe position is the phase difference between the interfering diffracted beams. From this information, the phases of all the beams can be determined. Once the phase problem is solved, it no longer makes sense to describe a method as being imaging or diffraction since the data can be readily converted from one to the other through a Fourier transform. Ptychography is thus a combination of diffraction and imaging.
It should be noted that the ptychography method implemented here for focused-probe STEM is not limited to perfect crystals, but is general to any object as long as the transmission by the sample can be modelled as a multiplicative transmission function. The mathematical approach used is described in more detail in [16] and modified for the current work as described in [15], but for completeness we describe it briefly here. The 4D measured data set is denoted |M(Kf, Ro)|2 where the position in the detector plane is given by the reciprocal space vector Kf and the illuminating probe position by Ro. Taking the Fourier transform of the data set with respect to the Ro coordinate, but not the Kf coordinate gives

G(Kf,Qp)=A(Kf)A*(KQp) ⊗Kf ψ(Kf)ψ*(KQp)

where Qp is the image spatial frequency variable conjugate to Ro, A(K) is the aperture function for the illumination with a modulus controlled by the size and position of the objective aperture and phase reflecting any aberrations present, ψ(K) is the Fourier transform of the specimen transmission function, and Kf denotes a convolution with respect to the detector plane position variable. If A(K) is known, then the product to the left of the convolution can be deconvolved, and the specimen transmission function determined from the product on the right. Thus the amplitude and phase of the specimen transmission function are determined, and both can be plotted fully quantitatively. Given the discussion in the introduction, it is important to note that the phase can be determined quantitatively even if there are no aberrations present. Efficient quantitative phase imaging is possible without the need for a phase plate using STEM ptychography. Figure 5 shows a comparison of images from the same sample taken using a JEOL JEM-3000F instrument running as an HRTEM and a ptychography image from a STEM showing that HRTEM-like imaging is now fully available in STEM. 

Fig. 5

Images of a thin film of a C60/C70 mixture: (a) recorded in a JEOL JEM-3000F instrument running in a CTEM configuration at 300 kV accelerating voltage; (b) recorded in a JEOL JEM-ARM200F instrument running at 200 kV using the 4DCanvas™ detector followed by ptychographic reconstruction. Note the similarity in the contrast revealed using the two types of imaging.

Enabling low-dose imaging

Because all the transmitted electrons are detected when using an FPD, we might expect to form images with much lower noise that was possible with non-segmented detectors, and thus to be able to lower the electron dose while still maintaining sufficient signal to noise in the image. Equation (1) also allows us to know exactly where in the detector plane the information is arising for each spatial frequency in the image, and therefore by just using those regions, the noise (which is distributed across the entire detector plane) is somewhat rejected. It is like having a STEM detector that is adapting itself to be optimal for each different spatial frequency in the image. Figure 6 shows a comparison of imaging using ADF and ptychographic STEM recorded simultaneously of a monolayer of hexagonal boron nitride. In the ptychographic image the noise is very low, and the location of a boron vacancy can be readily identified.

Fig. 6

An ADF image and a ptychographic image of hexagonal boron nitride recorded simultaneously at a beam energy of 60 keV. The ptychography image can be seen to be much lower in noise, and a boron vacancy defect can be readily identified.

Aberration correction

Prior to the development of hardware to correct for the aberrations in the electron microscope, it was envisaged that ptychography would offer a solution to the problem of spherical aberration. Once the complex transmission function is known, the effects of aberrations can be deconvolved. This aim for ptychography was overtaken by the successful development of aberration correctors. Nonetheless, it remains the case that often, perhaps because of small aberration drift or imperfect corrector tuning, some residual aberrations remain. The more recently developed iterative methods for ptychography make no initial assumptions about the aperture which is then solved during the iterative process [17]. The direct method used for the results here, does require the aperture function to be known, but it has also been shown that in the case of a weak-phase object the residual aberrations can be directly measured from the function given in Equation (1), and then can be deconvolved [15]. Figure 7 shows that even for a substantially misaligned instrument the aberration correction offered by ptychography is able to recover an image correctly reflecting the structure of the sample.
An additional benefit arising from the ability to correct aberrations is that a reconstruction can be performed assuming a specific defocus. It has been shown that this approach allows for an optical sectioning effect leading to three-dimensional reconstructions of the object [15]. The 3D information is inherently stored in the 4D data set recorded from the microscope even though the data has been recorded from a single scan at a fixed defocus.

Fig. 7

(a) An image of graphene recorded at 80 kV with the microscope misaligned leading to large residual aberrations. (b) From the ptychographic data set, the aberrations have been measured and corrected so that the lattice is now visible. The Fourier transform of the images show that the second ring of spots are all now visible, unlike the Fourier transform of image (a).

Dynamical effects

The theoretical basis for ptychography described above assumes that the interaction of the electron beam with the sample can be described by a multiplicative transmission function. For thicker and heavier samples, dynamical electron scattering conditions apply, and in this case the multiplicative approximation cannot be made. In the multiplicative approximation, it is assumed that the amplitude or phase of a diffraction beam is not dependent on the angle of the incoming beam with respect to the sample. In the case of dynamical scattering, there is a dependence.
Nonetheless, there is nothing to stop the 4D data set being recorded, and we can apply the same ptychographic reconstruction method to the data. Returning to the Pt wedge sample used in the data for Fig. 4, we can now perform a ptychographic reconstruction, as shown in Fig. 8. The peaks in the phase image can still be seen to be localized to the atomic column positions and there are no contrast reversals visible. At some thicknesses, the peaks show a "halo"-like structure. Similar results have been shown by Yang et al. [18]. Although a more detailed study is required, it appears that the ptychographically reconstructed phase images are more robust to dynamical effects and thickness changes than HRTEM images.

Fig. 8

The ADF image (a) and the ptychographic phase image (b) from the Pt wedge sample also used in Fig. 4. As the thickness increases, the phase image starts to form "halo" like contrast, but the peak is still located at the atomic column position and contrast reversals are not seen. Note that there is an inclined stacking fault towards the lower right of the image so additional atomic columns are visible.


The development of FPDs for STEM has allowed for highly flexible imaging in STEM and has created opportunities for new imaging modes. Here we have explored applications of electron ptychography, and shown how focused probe electron ptychography can be performed alongside conventional STEM modes such as ADF. The resulting phase image bears many similarities to HRTEM, but is also seen to have a very high signal-to-noise ratio and is robust to dynamical effects. Ptychography also allows for the correction of residual aberrations which further improves image contrast and allows for optical sectioning for 3D imaging.
Although STEM has become the preeminent instrument for atomic resolution studies, HRTEM has remained popular for light and thin samples, such as graphene and other layered materials, and of course is the main mode for cryo-EM of biological structures. Given that it has now been demonstrated that ptychography in STEM can deliver low-noise phase images, alongside all the other benefits of STEM, it may be that we are on the cusp of a paradigm shift where STEM becomes regarded as a powerful phase imaging instrument. The development of FPDs for STEM now allow fields of view comparable with HRTEM, and Figure 9 shows a 1k by 1k scanned image.
Finally, we note that ptychography is just one new mode possible with an FPD detector. Other authors have explored possibilities associated with measuring the angular dependence of the scattering at higher angles. Methods such as transmission Kikuchi diffraction become available, and using lower convergence angles the strength of all available diffraction spots can be measured as a function of probe position to give multiple diffraction contrast images in parallel, giving much greater information for dislocation burgers vector determination through g.b analysis for example.

Fig. 9

A simultaneously recorded (a) ADF and (b) ptychographic phase image from a Pt <110> wedge sample with 1 k by 1 k probe sampling recorded with a FPD frame speed of 4,000 frames per second demonstrating that large fields of view are possible in focused-probe STEM.


We acknowledge the fruitful collaboration with Y Kondo and R Sagawa, JEOL Tokyo, M Simson, M Huth, H Soltau, PNDetector GmbH, and L Strueder PNSensor GmbH, Germany. We also acknowledge experimental assistance from L Jones. Samples have been provided by S Nam and D Bradley (University of Oxford), Y Sasaki (Japan Fine Ceramics Centre), A Béché and D Batuk (University of Antwerp). Support for this project has been received from the EPSRC (grant number grant EP/M010708/1).


  • P. D. Nellist, Scanning Transmission Electron Microscopy, in: Science of Microscopy P.W. Hawkes, J. C. H. Spence (Eds.), Springer, 2007, pp. 65-132.
  • P. D. Nellist, S. J. Pennycook, The Principles and Interpretation of Annular Dark-Field Z-Contrast Imaging, Advances in Imaging and Electron Physics, 113 (2000) 148-203.
  • D. J. Smith, The realisation of atomic resolution with the electron microscope, Reports of Progress in Physics, 60 (1997) 1513-1580.
  • R. W. Glaisher, A. E. C. Spargo, D. J. Smith, A Systematic Analysis of HREM Imaging of Elemental Semiconductors, Ultramicroscopy, 27 (1989) 35-52.
  • T. J. Pennycook, A. R. Lupini, H. Yang, M. F. Murfitt, L. Jones, P. D. Nellist, Efficient phase contrast imaging in STEM using a pixelated detector. Part 1: Experimental demonstration at atomic resolution, Ultramicroscopy, 151 (2015) 160-167.
  • M. Marko, A. Leith, C. Hsieh, R. Danev, Retrofit implementation of Zernike phase plate imaging for cryo-TEM, Journal of Structural Biology, 174 (2011) 400-412.
  • J. M. Cowley, Image Contrast in a Transmission Scanning Electron Microscope, Applied Physics Letters, 15 (1969) 58-59.
  • E. Zeitler, M. G. R. Thomson, Scanning Transmission Electron Microscopy, Optik, 31 (1970) 258-280 & 359-366.
  • N. Shibata, S. D. Findlay, Y. Kohno, H. Sawada, Y. Kondo, Y. Ikuhara, Differential phase-contrast microscopy at atomic resolution, Nature Physics, 8 (2012) 611-615.
  • K. Müller, F. F. Krause, A. Béché, M. Schowalter, V. Galioit, S. Löffler, J. Verbeeck, J. Zweck, P. Schattschneider, A. Rosenauer, Atomic electric fields revealed by a quantum mechanical approach to electron picodiffraction, Nature Communications, 5 (2014) 5653.
  • W. Hoppe, Beugung im Inhomogenen Primärstrahlwellenfeld. I. Prinzip einer Phasenmessung von Elektronenbeugungsinter ferenzen, Acta Crystalographica A, 25 (1969) 495-501.
  • J. M. Rodenburg, B. C. McCallum, P. D. Nellist, Experimental tests on double-resolution coherent imaging via STEM., Ultramicroscopy, 48 (1993) 303-314.
  • P. D. Nellist, B. C. McCallum, J. M. Rodenburg, Resolution beyond the 'information limit' in transmission electron microscopy, Nature, 374 (1995) 630-632.
  • H. Yang, R. N. Rutte, L. Jones, M. Simson, R. Sagawa, H. Ryll, M. Huth, T. J. Pennycook, M. L. H. Green, H. Soltau, Y. Kondo, B. G. Davis, P. D. Nellist, Simultaneous atomic-resolution electron ptychography and Z-contrast imaging of light and heavy elements in complex nanostructures, Nature Communications, 7 (2016) 12532.
  • J. M. Rodenburg, R. H. T. Bates, The Theory of Super-Resolution Electron Microscopy via Wigner-Distribution Deconvolution, Philosophical Transactions of the Royal Society of London A, 339 (1992) 521-553.
  • A. M. Maiden, J. M. Rodenburg, An improved ptychographical phase retrieval algorithm for diffractive imaging, Ultramicroscopy, 109 (2009) 1256-1262.
  • H. Yang, I. MacLaren, L. Jones, G. T. Martinez, M. Simson, M. Huth, H. Ryll, H. Soltau, R. Sagawa, Y. Kondo, C. Ophus, P. Ercius, L. Jin, A. Kovács and P. D. Nellist. Electron ptychographic phase imaging of light elements in crystalline materials using Wigner distribution deconvolution, Ultramicroscopy 180 (2017) 173-179.

Related Products