.. _data_processing: ########################### MIMA Quick Data proccessing ########################### Introduction ############ This document walks through MIMA basic data processing features and how to setup the software to run the processing. First, let see some feautures of the software. FITS Data Format ================ The data produced by MIMA is stored in FITS (Flexible Image Transport System) file standard as a 2D image, and according to FITS_ standard (v4.0, 2018). For Python implementation of FITS file saving and reading, MCS uses the package `Astropy FITS`_. The FITS file is compound of a single HDU. An HDU (Header Data Unit) is the highest level component of the FITS file structure, consisting of a header and (typically) a data array or table. In our case, the HDU consists of a header and a 2D data array. The header is compound of the next list pairs (keyword, value):: SIMPLE = T / conforms to FITS standard BITPIX = 16 / array data type NAXIS = 2 / number of array dimensions NAXIS1 = 1024 NAXIS2 = 1024 EXTEND = T DATE = '2020-06-26T13:52:16.437023' / date of file creation AUTHOR = 'MCS ' / author of the data FILENAME= '20200626-135216.fits' / name of the file DATE-OBS= '2020-06-26T13:48:39.276186' / [d] UT-date exposure started EXPTIME = 200.0 / exposure time in seconds IMAGETYP= 'light ' / Image type READMODE= 'ReadoutSpeed._1MHz_LN' / Readout speed mode SHUTTER = 'ShutterStatus.LEAVE_AS_IS' / Requested shutter state INSTRUME= 'Fake Cameras Inc.' / Name of instrument CCD = 'Fake CCDs Inc.' / CCD Model CCD_DIMX= 1024 / CCD X Dimension Size CCD_DIMY= 1024 / CCD Y Dimension Size CCDPXSZX= 24 / CCD X Pixel Size [micrometer] CCDPXSZY= 24 / CCD Y Pixel Size [micrometer] CCD-TEMP= -20.08724568677194 / CCD Temperature at Exposure Start [deg. C] CCDSUM = '1 1 ' / CCD on-chip summing SITE = 'OSN ' / Site name (in config) OBSGEO-B= 37.06416 / [deg] Site latitude (=North+) OBSGEO-L= -3.38472 / [deg] Site longitude (=East+) OBSGEO-H= 2900.0 / [m] Site altitude SUN-ELE = 65.27387295747029 / [degree] Sun obs. elevation [-90,90] MOON-ELE= 41.21638152022859 / [degree] Moon obs. elevation [-90,90] MOON-PHA= 0.3040238530830346 / Moon illumination [0-1] OBSERVAT= 'OSN ' / Sierra Nevada Observatory (IAA-CSIC) ORIGIN = 'IAA-CSIC' / Institute of Astrophysics of Andalusia DATAMEAN= 5590.843698501587 / Mean data value in the image DATASTD = 82.71160318426344 / STD data value of the image DATAMIN = 4566 / Min data value in the image DATAMAX = 12688 / Max data value in the image FLI-SDK = '1.104 ' / FLI SDK software version FLI-KRNL= '1.3.2 ' / FLI Linux kernel version MCS-VER = '2.0.3.dev0+g6311fc0.d20200625165239' / MIMA Control Software version OBS-CAMP= 'Camp3 ' / Observation campaign OBS-MODE= 'ObsMode1' / Observation mode OBSERVER= 'CampaignRunner' / Observer SIMULAT = T / MCS is in simulation mode FWHEEL = 'Fake Filter Instrumentation' / FilterWheel Model FILTER = 'Na ' / Filter used for this observation TEMPCTLR= 'IAA-TempCtrl' / TempCtrler Model FLT-SETT= '30.0 ' / Set temperature FLT-TEMP= '30.0 ' / Current temperature PWCTRL = 'Fake-PowerCtrl' / PowerCtrler Model PWSTATUS= 'filter_wheel=ON camera=ON temp_ctlr=ON usb_externder=ON' / PowerCtrle DOMECTLR= 'TH2E ' / Dome T/H sensor DOME-TEM= 'None ' / Current Dome Temperature (C) DOME-HUM= 'None ' / Current Dome Humidity (%s) SUN-ELM = 70.0 / [degree] Sun max. elevation MOON-ELM= 70.0 / [degree] Moon max. elevation MOON-FIL= '/home/jmiguel/mc.py' / Moon phase correction file COMMENT Image generated by CampaignRunner END MIMA images are saved with BITPIX=16, what means 16-bit two’s complement binary integer. Thus, a not binned image (1024x1024) will require 2 MB of storage. Functionalities implemented =========================== #. Conversion of old SATI FITS files to be compliant with FITS standard (v4.0, 2018), and also compliant with MIMA data processing pipeline. It is done for a whole given directory. #. Pre-processing (for a given directory): * master darks creation, grouping by EXPTIME * master flat creation, grouping by EXPTIME and type (dusk, dawn), and subtracting previosly the nearest exposure master dark. * combine of science raw files, grouped by FILTER, and doing previously the dark subtraction. * science raw file calibration doing dark subtraction and flat-fielding. #. Spectroscopic processing: * determination of the center of the FPI image using an iterative method based on fringe peaks symmetry (as described in paper XXX), fitting a Gaussian to the line peaks. * determination of the center of the FPI image using Canny filter and circular Hough transform. * full circle averaged spectrum extraction and plotting given the center of the image, and line emission detection * FPI 2D image slicing in sectors, and extraction (and plotting) of a averaged 1D spectrum of each sector. Data Proccesing steps ===================== MIMA output images on the CCD shows concentric fringes, each of which corresponds to an emission line, as shown in figure below. By taking the intensity ratio of these fringes, one can measure the rotational temperature of the emitting molecules. .. figure:: ../_static/mima_sample.png :align: center :width: 60% MIMA sample image As described above, concentric fringes are obtained for each filter, corresponding to individual airglow lines. From these fringe images, we took the following four data processing steps to calculate rotational temperatures and airglow emission rates #. Cosmic ray removal #. Dark count subtraction #. Fringe center determination #. Filter temperature drift After these four data processing steps, we fit the synthetic spectra to the observed spectra using to obtain the rotational temperatures and airglow emission rates. .. _MIMA: http://gapt.iaa.es/content/mima .. _OSN: http://www.osn.iaa.es .. _iaa_web: http://www.iaa.es .. _source code: http://github.com/ppmim/MIMA .. _FITS: https://fits.gsfc.nasa.gov/fits_standard.html .. _Astropy FITS : https://docs.astropy.org/en/stable/io/fits/