This tutorial demonstrates how to create a Docker container to distribute a complete Jupyter notebook environment.
This intends to be a first introduction to TensorFlow 2.0 not as much from a machine learning point of view but from an optimization and inverse problem perspective.
Description MRS3D is based on two packages, IDL and Healpix. MRS3D can be used only if these two packages have been installed. MRS3D contains IDL and C++ routines for 3D spherical wavelet transform on the sphere .
Source code and more information are available here. For more information, please contact Francois Lanusse (firstname.lastname@example.org).
Publications Paper related to the software:
- F. Lanusse,A. Rassat,and J.-L. Starck, "Spherical 3D Isotropic Wavelets", Astronomy and Astrophysics , 540, A92, 2012.
Glimpse is a weak lensing mass-mapping tool relying a robust sparsity-based regularisation scheme to recover high resolution convergence from either gravitational shear alone or from a combination of shear and flexion. Including flexion allows us to supplement the shear on small scales in order to increase the sensitivity to substructures and the overall resolution of the convergence map.
In order to preserve all available small-scale information, Glimpse avoids any binning of the irregularly sampled input shear and flexion fields and treats the mass-mapping problem as a general ill-posed inverse problem, regularised using a multi-scale wavelet sparsity prior. The resulting algorithm incorporates redshift, reduced shear, and reduced flexion measurements for individual galaxies and is made highly efficient by the use of fast Fourier estimators.
The source code for Glimpse is made publicly available and is hosted on Github at https://github.com/CosmoStat/Glimpse
Test on realistic simulated dark matter distributions
Glimpse was tested on a set of realistic weak lensing simulations corresponding to typical HST/ACS cluster observations and demonstrate our ability to recover substructures with the inclusion of flexion which are lost if only shear information is used. In particular, we can detect substructures at the 15′′scale well outside of the critical region of the clusters. In addition, flexion also helps to constrain the shape of the central regions of the main dark matter halos. These simulations, along with the reconstructions produced by Glimpse can be found in this archive : flexion_benchmark.tar.lzma.
Applications to real data
A520 Cluster Merger
We have used Glimpse to reconstruct the mass distribution of Abell 520, a merging galaxy cluster system also known as the 'cosmic train wreck'. We obtained high-resolution mass maps using two separate galaxy catalogs derived from HST observations and compared the results to previous weak-lensing studies of the system.
The galaxy catalogs in FITS format and configuration files for Glimpse can be downloaded here. Example outputs are included for each data set.
To generate the convergence map of the C12 data with a regularization parameter of 3.0, for example, edit the config_A520_c12.ini file and set the 'lambda' option equal to 3.0 under [parameters]. Then run
$ glimpse config_A520_c12.ini A520_cat_c12.fits kappa.fits
to obtain an output convergence map called 'kappa.fits'.
DES SV data
Glimpse has been used to map the matter density field of the Dark Energy Survey (DES) Science Verification (SV) data. The Glimpse reconstruction was compared to two other mass-mapping methods: standard Kaiser-Squires inversion and the Wiener filter.
- F. Lanusse, J.-L. Starck, A. Leonard, S. Pires, High Resolution Weak Lensing Mass-Mapping Combining Shear and Flexion, 2016, arXiv:1603.01599
- A. Peel, F. Lanusse, J.-L. Starck, Sparse Reconstruction of the Merging A520 Cluster System, 2017, arXiv:1708.00269
- N. Jeffrey, F. B. Abdalla, O. Lahav, F. Lanusse, J.-L. Starck, Improving Weak Lensing Mass Map Reconstructions using Gaussian and Sparsity Priors: Application to DES SV, 2018, arXiv:1801.08945
Detection of supernovae and, more generally, of transient events in large surveys can provide numerous false detections.In the case of a deferred processing of survey images, this implies reconstructing complete light curves for all detections, requiring sizable processing time and resources.Optimizing the detection of transient events is thus an important issue for both present and future surveys.We present here the optimization done in the SuperNova Legacy Survey (SNLS) for the 5-year data deferred photometric analysis. In this analysis, detections are derived from stacks of subtracted images with one stack per lunation.The 3-year analysis provided 300,000 detections dominated by signals of bright objects that were not perfectly subtracted.Allowing these artifacts to be detected leads not only to a waste of resources but also to possible signal coordinate contamination.We developed a subtracted image stack treatment to reduce the number of non SN-like events using morphological component analysis.This technique exploits the morphological diversity of objects to be detected to extract the signal of interest.At the level of our subtraction stacks, SN-like events are rather circular objects while most spurious detections exhibit different shapes.A two-step procedure was necessary to have a proper evaluation of the noise in the subtracted image stacks and thus a reliable signal extraction.We also set up a new detection strategy to obtain coordinates with good resolution for the extracted signal.SNIa MC generated images were used to study detection efficiency and coordinate resolution.When tested on SNLS 3 data this procedure decreases the number of detections by a factor of two, while losing only 10% of SN-like events, almost all faint.MC results show that SNIa detection efficiency is equivalent to that of the original method for bright events, while the coordinate resolution is improved.
SNLS deferred photometric detection pipeline
Supernovae are not only extremely luminous objects but they are also transient events, which makes their detection possible by looking for brightness variations on the sky.
The strategy adopted in the SNLS deferred photometric pipeline is to build a high quality reference image of the sky which is then substracted to subsequent exposures. Fig 1. illustrates the substraction of the reference frame for a galaxy hosting a supernova. The image of the galaxy, which remains constant, is substracted out and only the transient supernovae remains after substraction.
The example in Fig. 1 is of course an illustration of a very clean, high SNR, supernova event. In practice however, faint supernovae can not be detected from individual exposures as their SNR is too low. Therefore, in order to increase the signal to noise, the approach implemented in this SNLS pipeline is to build image stacks of a large number of exposures (after reference frame substraction) and to detect supernovae candidates from these stacks.
However, image stacks produced by this SNLS pipeline are plagued by a large number of defects, accumulated at the different levels of the production of stack images (removal of bright stars, resampling defects, imperfect subtractions,...). Some of these defects are illustrated on Figure 2.
Because of these defects, the detection of supernovae candidates on these images leads to an extremely large number of spurious detections. Figure 3 shows individual detections on these image stacks for one field of SNLS3. For this field, the detection pipeline produced nearly 100,000 detections, 99.6 % of which are purely spurious detections. As can be seen on this figure, spurious detections concentrate around bright stars and detector borders and defects.
Image cleaning using MCA
In order to reduce the large number of false detections due to defects in the stack images, the approach proposed in Moller et al. (2015) was to clean these images using Morphological Component Analysis prior to performing the detection. to separate supernovae-like signal from defects. Indeed, most defects have morphologies very distinct from that of typical supernovae.
Supernovae signal is very well represented using isotropic wavelets (Starlets) while most of the defects in the image (lines, stars, incorrect substractions) can be extracted using a combination of ridgelets, curvelets and bi-orthogonal wavelets.
In the proposed detection procedure, image stacks are first processed using MCA to extract 4 different morphological components. Then the starlet component, corresponding to supernovae signal, is processed in a similar fashion as in the original pipeline to detect candidates, without being contaminated by the defects removed by the MCA. Figure 5 shows and example of an image stack and the corresponding detection map before cleaning. A clear excess of detections can be found at the vicinity of the bright star in this field. Figure 6 shows the result of the MCA cleaning on the same image, keeping only noise and starlet components. The defects are completely removed and the corresponding detection map is much cleaner.
The new detection procedure has been extensively tested using Monte-Carlo simulations. The overall false detection rate is reduced by more than a factor 2 while conserving very similar detection efficiency (see Figure 7, the efficiency is only reduced for very faint SNae which are not suitable for cosmology in any case). A similar improvement could not have been obtained with the original detection pipeline which had already been extensively optimised.
Additionnally, the new detection procedure was found to reduce the magnitude bias of the detected SNe Ia which will have an impact on the cosmological analysis done with this sample.
- Möller, A., Ruhlmann-Kleider, V., Lanusse, F., Neveu, J., Palanque-Delabrouille, N., Starck, J.-L., 2015, SNIa detection in the SNLS photometric analysis using Morphological Component Analysis. Accepted in JCAP, arxiv:1501.02110
Professor Vassilis Charmandaris, Director of the Athens Observatory, is visiting CosmoStat lab this week.
A European team, involving researchers from the l’Ecole Polytechnique Fédérale de Lausanne (EPFL) and the Astrophysical Department Sap-AIM of CEA-Irfu, has found that some of the defects in the Cosmic Microwave Background of the universe present in the images obtained by the WMAP and Planck satellites may only be due to poor image reconstruction and incomplete subtraction of the contributions of our own galaxy. These results are published in the Journal of Cosmology and Astroparticle Physics August 2014.
The LGMCA method has been used to reconstruct the Cosmic Microwave Background (CMB) image from WMAP 9 year and Planck-PR1 data. Based on the sparse modeling of signals – a framework recently developed in applied mathematics – the proposed component separation method is well-suited for the extraction of foreground emissions.
A joint WMAP9 year and Planck PR1 CMB has been reconstructed for the first time and produce a very high quality CMB map, especially on the galactic center where it is the most difficult due to the strong foreground emissions of our Galaxy. This webpage provides some comparisons between the PR1 and WPR1 maps and codes to recompute the map in the spirit of reproducible research.
We present GLIMPSE - Gravitational Lensing Inversion and MaPping with Sparse Estimators - a new algorithm to generate density reconstructions in three dimensions from photometric weak lensing measurements. This is an extension of earlier work in one dimension aimed at applying compressive sensing theory to the inversion of gravitational lensing measurements to recover 3D density maps. Using the assumption that the density can be represented sparsely in our chosen basis - 2D transverse wavelets and 1D line of sight dirac functions - we show that clusters of galaxies can be identified and accurately localised and characterised using this method. Throughout, we use simulated data consistent with the quality currently attainable in large surveys. We present a thorough statistical analysis of the errors and biases in both the redshifts of detected structures and their amplitudes. The GLIMPSE method is able to produce reconstructions at significantly higher resolution than the input data; in this paper we show reconstructions with 6x finer redshift resolution than the shear data. Considering cluster simulations with 0.05 <= z <= 0.75 and 3e13/h Msun <= Mvir <= 1e15/h Msun, we show that the redshift extent of detected peaks is typically 1-2 pixels, or Dz <~ 0.07, and that we are able to recover an unbiased estimator of the redshift of a detected cluster by considering many realisations of the noise. We also recover an accurate estimator of the mass, that is largely unbiased when the redshift is known, and whose bias is constrained to <~ 5% in the majority of our simulations when the estimated redshift is taken to be the true redshift. This shows a substantial improvement over earlier 3D inversion methods, which showed redshift smearing with a typical standard deviation of 0.2-0.3, a significant damping of the amplitude of the peaks detected, and a bias in the detected redshift.