Hamamatsu Photonics K.K. · TSE:6965
Hamamatsu Photonics K.K — Technical Presentation (September 2013)
40 slides from Hamamatsu Photonics K.K's technical presentation, Sep 10, 2013
Slide 1: Bridging the Gap: Impact and Correction of Camera Noise
BRIDGING Impact and Correction of THE Camera Noise for Computational Microscopy including Precision GAP Localization Nanoscopy Keith Bennett, Ph.D., Stephanie Fullerton, Ph.D., Eiji Toda, Hiroyuki Kawai, Teruo Takahashi ADAPTED FROM PRESENTATION GIVEN AT METHODS AND APPLICATIONS OF FLUORESCENCE, GEN
Slide 2: Cameras are NOT perfect!
PHOTON 1S OUR BUSINESS Cameras are NOT perfect! Changing the Game Copyright © Hamamatsu Photonics K.K. All Rights Reserved.
Slide 3: Why is a camera manufacturer proclaiming that cameras are not perfect?
Why is a camera manufacturer proclaiming that cameras are not perfect?
Slide 4: Because NO camera is perfect
Because NO camera is perfect & Because understanding why matters to your science
Slide 5: What is the Gap?
WHAT IS THE GAP? The difference between the performance of an actual camera and a theoretically perfect camera Perfect Camera The GAP { Actual Camera
Slide 6: The Amount of the Gap Depends On
THE AMOUNT OF THE GAP DEPENDS ON: 1. Sensor technology { CCD EMCCD sCMOS 2. Camera specifications { Quantum Efficiency Camera Noise { Read noise Excess Noise Photo‐response non‐ uniformity (PRNU) { Ultra low light 3. Input photon Low Light level Intermediate High
Slide 7: Understanding why there is a gap enables better results
UNDERSTANDING WHY THERE IS A GAP ENABLES: • Appropriate camera selection • Optimized camera usage Better • Optimized experimental design Results • More reliable data analysis
Slide 8: The Real Cameras: CCD
THE REAL CAMERAS CCD • Well established technology • All electron to digital conversion done in one chain • Limited speed • Moderate read noise • Very low dark current • High QE • Best pixel response uniformity
Slide 9: The Real Cameras: EMCCD
THE REAL CAMERAS EMCCD • Back‐thinned for increased QE • High voltage gain register on sensor to achieve on‐chip amplification • All electron to digital conversion through one chain(either for EM or no EM) • Read noise is low due to gain • Stochastic EM amplification adds excess noise and long tail
Slide 10: The Real Cameras: CMOS
THE REAL CAMERAS CMOS • Newest technology • Every pixel and column has own amplifier • Very low mean rms read noise • Pixel dependent read noise • Fastest speeds and largest field of view • FPGA processing achieves excellent response uniformity (low PRNU)
Slide 11: Seeing the Predicted Gap
SEEING THE Single Pixel Noise & SNR PREDICTED Fixed Pattern Noise & GAP Image SNR (from specs)
Slide 12: The Perfect Camera
THE PERFECT CAMERA 100% QE { Every photon is converted into one electron read noise { Every electron is digitized exactly as expected every time 0 e‐ 0% fixed { Every pixel and amplifier perform identically and predictably pattern noise Perfect Camera Signal to Noise Ratio Signal to Noise Ratio (SNR
Slide 13: Effect of QE on the Gap
EFFECT OF QE ON THE GAP Signal to Noise Ratio (SNR) Signal to Noise Ratio 100 Perfect QE 70% A reduction in QE 50% 10 QE reduces SNR at all light 0.1 1 1 10 100 1000 10000 levels 0 Signal (Photons)
Slide 14: Relative SNR (rSNR) plots clearly show the gap
RELATIVE SNR (rSNR) PLOTS CLEARLY SHOW THE GAP 1 0.9 0.8 rSNR is the SNR for a Relative SNR (rSNR) 0.7 camera plotted relative to 0.6 Perfect the perfect camera 0.5 QE 70% 0.4 QE 50% rSNR shows differences 0.3 among cameras over full 0.2 range of signal level 0.1 0 0.1 1 10 100 1000 10000 Signal (Ph
Slide 15: The Simple Signal to Noise Ratio (SNR)
THE SIMPLE SIGNAL TO NOISE RATIO (SNR) QE: Quantum Efficiency S: Input Signal Photon Number (photon/pixel) F: Noise Factor (= 1 for CCD/sCMOS and √2 for EM‐CCD) Nr: Readout Noise M: EM Gain (=1 for CCD / CMOS) Ib: Background “Changing the Game”
Slide 16: EMCCDs: Excess Noise is the Reason for the Gap
EMCCDS: EXCESS NOISE IS THE REASON FOR THE GAP SNR for CCD / CMOS SNR for EM‐CCD QE P M QE P QE P SNR SNR QE P Fn M QE P Fn 2 QEeff P QE P QE QE QEeff 2 QE: Quantum Efficiency, Fn 2 P: Input Signal Photon Number, M: EM Gain Fn: Noise Factor (assumes dark current and
Slide 17: The Real Cameras
THE REAL CAMERAS CCD EMCCD CMOS Sensor Type Charged Coupled Device Electron Multiplying CCD Complimentary Metal Interline Back‐thinned Oxide Sensor Camera Name ORCA‐R2 ImagEM x2 ORCA Flash4.0 V2 Pixel Number 1024 x 1344 512 x 512 2048 x 2048 Pixel Size 6.45 µm x 6.45 µm 16 µm x 16 µm 6.5 µm x 6.5 µm
Slide 18: Mind the Gap: Predicted Pixel rSNR Performance
MIND THE GAP: PREDICTED PIXEL rSNR PERFORMANCE FOR THE MOST COMMON CAMERAS 1.{ 2.{ A camera with the highest The SNR of an EMCCD above 1 SNR at the lowest light level electron/pixel is comparable to a may not be the best at camera with QEeff =QE/2 due to higher light levels excess noise from EM gain
Slide 19: Beyond the Sweet Spot: Gap Expands if PRNU Not Corrected
BEYOND THE SWEET SPOT: THE GAP EXPANDS AT HIGH LIGHT IF PRNU IS NOT CORRECTED Single Frame rSNR 1.0 0.9 • PRNU reduces SNR at high light 0.8 Relative SNR 0.7 0.6 • Cannot be subtracted from image 0.5 0.4 • “Raw” PRNU varies by sensor 0.3 0.2 0.1 • Can be corrected in camera to 0.0 varying degrees 0.
Slide 20: Measuring the Real Gap
MEASURING An in‐depth look at noise in THE CCD, EMCCD and CMOS REAL GAP cameras
Slide 21: ORCA-R2 Interline CCD: Predictable and Robust
ORCA‐R2 INTERLINE CCD: PREDICTABLE AND ROBUST 1. { PRNU is insignificant Bright Image: shot noise limited { Read noise histogram 2. has single Gaussian Mean intensity: 17,300 e- : 130.5e- PRNU: not measurable distribution Read Noise Shot Noise (Nr/QE) & QE 10000 1.0 0.9 58% QE Limit 1000 0.8 0.7 Co
Slide 22: EMCCD: Some Surprising Results
EMCCD: SOME SURPRISING RESULTS 550 nm 850 nm { Thickness variations from back- 1. thinning process causes spectrally- dependent PRNU • Cannot be removed during manufacturing Mean: 30157 Mean: 30508 s: 369.5 (1.2%) s: 432 (1.4%) • Must be calibrated by users for their specific spectrum. Calculated Si
Slide 23: Complex Behavior: EMCCD SNR with High and Low Gain
COMPLEX BEHAVIOR: A CLOSER LOOK AT EMCCD SNR WITH HIGH AND LOW GAIN Complex Behavior Saturation - Excess noise (eQE) 1 90% QE Limit - PRNU - Saturation 0.9 0.8 Excess Noise - High read noise (34 e- @ M=5, 70 fps) Relative SNR 0.7 - Gain hard to measure 0.6 0.5 0.4 0.3 0.2 Series1 Gain = 5 0.1 Series
Slide 24: EMCCD Gain Causes Uneven Probability Distributions
EMCCD GAIN CAUSES UNEVEN PROBABILITY DISTRIBUTIONS In simulated probability distribution functions for EMCCD, the output at high gain is not Poisson due to the electron multiplication process! 2 Photon Average Input 10 Photon Average Input Gain = 200 Gain = 200 0.0008 0.00025 0.0007 Probability [a.u
Slide 25: ORCA-Flash4.0 V2 (sCMOS): A Very Comfortable Sweet Spot
ORCA‐FLASH4.0 V2 (SCMOS): A VERY COMFORTABLE SWEET SPOT The “Sweet Spot” Shot Noise & QE 1.0 0.9 70% QE Limit 0.8 0.7 0.6 rSNR 0.5 0.4 0.3 0.2 0.1 0.0 1 10 100 1,000 10,000 Signal (Photons @650 nm)
Slide 26: Not All Cameras Are Created Equal: Flash4.0 Sweet Spot
NOT ALL CAMERAS ARE CREATED EQUAL: FLASH4.0 SWEET SPOT BROUGHT TO YOU BY HAMAMATSU CAMERA ENGINEERS Raw Data Corrected Data { { Bright Image Bright Image PRNU: ~2% PRNU: ~0.5% Signal amplified and digitized in column‐parallel ADC. FPGA provides offset and gain correction to the raw digitized signal.
Slide 27: sCMOS: Pixel-Dependent Read Noise
SCMOS: PIXEL‐DEPENDENT READ NOISE 1,000,000 0.85 e‐Median Single Frame Read 100,000 Noise (measured) Number of pixels Rms read noise matches 1.51 e‐ rms 10,000 single frame rSNR. 1,000 100 (30 s/ row) 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Temporal read noise (e‐, rms) Single Frame Dark Histog
Slide 28: Bridging the Gap
BRIDGING Using knowledge of camera noise to get highest camera THE GAP performance provides improved Precision Localization Results
Slide 29: What is most important?
sample contrast frame rate WHAT IS resolution MOST accuracy IMPORTANT? background
Slide 30: Two Phases of Precision Localization Microscopy
TWO PHASES OF PRECISION LOCALIZATION MICROSCOPY Prepare Sample 1. Collect Minimize Background Image Data Optimize Optical System Consider Camera Induced Noise 2. Reconstruct Calibrate Camera Implement Noise Corrections Superresolution Apply Statistical Algorithms Image
Slide 31: Standard Practice is Not the Best Practice
STANDARD PRACTICE IS NOT THE BEST PRACTICE: USING EMCCD WITH GAIN YIELDS LEAST ACCURATE RESULTS CCD QE: 100%, read noise = 1.8 ph, no background; No fixed pattern noise. Adapted from: J. Chao et al (Ober Lab), Nat. Meth10, 2013) doi:10.1038/nmeth.2396 http://www.wardoberlab.com/
Slide 32: Compensating Read Noise Variation
COMPENSATING READ NOISE VARIATION Courtesy Prof. Joerg Bewersdorf, Yale University Incorporating pixel‐specific read noise into the Maximum Likelihood Probability Model eliminates and narrows the asymmetric distribution of localized molecules caused by higher read noise pixels. Courtesy F. Huang, Be
Slide 33: Selecting and Using Cameras: Case Studies
SELECTING AND USING CAMERAS: CASE STUDIES { { {
Slide 34: Case Study: Two-Color Distance Measurement
{ Accurate measurement of the distance between two fluorophores of different colors. distance ~0.77 nm using a Results dichroic beamsplitter to direct each color of light to separate halves of the CCD camera. Camera Correction{ Measured PRNU maps for each color. Improved localization relative accur
Slide 35: Case Study: Minimal Bleaching Localization Microscopy
Cholera toxin B subunit scale bar: 1 m { Localization Microscopy with Minimal Bleaching. Plasma Results membrane dynamics for > 60 s1 m(594 frames). 40%1 m better localization precision than “conventional” EMCCD localization Camera Correction{ Implemented detailed statistical EM noise model into
Slide 36: Case Study: Fast sCMOS Reconstruction
{ 32 reconstructed frames / sec (6.6 x 6.6 m2) field of view; Results fixed and living cells showed cellular dynamics not visible in reconstructions using longer data collection times. { Camera Implemented pixel‐specific read noise into probability model for Correction MLEM. Details { Speed: 0.03 s
Slide 37: Localization Precision: Conventional EMCCD vs. sCMOS
Localization Precision “conventional” EMCCD vs. sCMOS Courtesy of F. Huang. Bewersdorf Lab, Yale Adapted from F. Huang et al., Nature Methods 10(7): 653‐658 (2013)
Slide 38: Minimizing the Gap: Matching the Camera to Your Needs
MINIMIZING THE GAP: MATCHING THE CAMERA TO YOUR NEEDS Higher Accuracy Better Resolution Lower Sample Contrast Scientific CMOS (BT)‐CCD “Conventional” Light Required Localization EMCCD (Gain‐On) EMCCD “Conventional” (Gain ON) Localization UAIM Speed / Field of View Faster (or more pixels)
Slide 39: Have You Done a Gap Analysis?
HAVE YOU DONE A GAP ANALYSIS? The relative performance of CCD, 1. How much light do I have? { back‐thinned CCD, EMCCD and sCMOS cameras is light‐level dependent. { 2. Do I know my No camera is perfect; proper use camera’s strengths is required for the best results and weaknesses? and to avoid errors
Slide 40: Acknowledgements
Acknowledgements Prof. Zhen‐li Huang, Huanzhong University of Science and Technology F. Long et al, OPTICS EXPRESS 17741 (2012) Prof. Joerg Bewersdorf, Yale University F. Huang et al., Nature Methods 10(7): 653‐658 (2013) See a movie of 32 fps dynamics in the supplementary materials of Prof. Raimund
Hamamatsu Photonics K.K. — Technical Presentation (September 2013) on 6ix