A Convolutional Neural Network (CNN) architecture for classifying standard and modified gravity (MG) cosmological models based on the weak-lensing convergence maps they produce.
This repository contains the code and data used to produce the results in A. Peel et al. (2018), arXiv:1810.11030.
The Convolutional Neural Network (CNN) is implemented in Keras using TensorFlow as backend. Since the DUSTGRAIN-pathfinder simulations are not yet public, we are not able to include the original convergence maps obtained from the various cosmological runs. We do provide, however, the wavelet PDF datacubes derived for the four models as described in the paper: one standard LCDM and three modified gravity f(R) models.
Keras with Tensorflow as backend
$ python3 train_mgcnn.py -n0
The three options for the noise flag "-n" are (0, 1, 2), which correspond to noise standard deviations of sigma = (0, 0.35, 0.70) added to the original convergence maps. Additional options are "-i" and "-e" for the number of training iterations and epochs, respectively.
Confusion matrices and evaluation metrics (loss function and validation accuracy) are saved as numpy arrays in the generated output/ directory after each iteration.
Numerical routines to calculate cosmology and weak-lensing quantities.
nicaea [ni'kaia]: NumerIcal Cosmology And lEnsing cAlculations
Martin Kilbinger, CEA Saclay, Service d'Astrophysique (SAp), France
nicaea is a C-code providing numerical routines to calculate cosmology and weak-lensing quantities and functions from theoretical models of the large-scale structure. nicaea is the base of the cosmology module of the CosmoPMC package.
Get the latest stable version by cloning the most recent version from github . A readme file (types .rst, .html, .pdf and other) is included in the package. Check also readthedocs for documentation. New features in version 2.7 (Feb 2017):
New lensing projection types: extended Limber, spherical-sky prefactor, second-order Limber, full projection (Kilbinger et al. 2017, arXiv:1702.05301)
Photometric redshift errors (so far supported Gaussian with second Gaussian for outliers)
Modification of halomodel: mass function now normalized to physical volume (new division by a^3)
Added CMB normalization A_s
Added options to lensingdemo
For older versions of nicaea please contact me (martin.kilbinger at cea.fr). Note that v2.6 was skipped, the previous released version is 2.5
There is no dedicated paper that describes nicaea. To reference nicaea, please use the following publication: arXiv:0810.5129, in which something that resembles the first version of nicaea has been used.
CONTACT INFORMATION Please feel free to send questions, feedback and bug reports to firstname.lastname@example.org. If you want to be added to the nicaea mailing list, to get updates about new versions and bug-fixes, send me a mail to email@example.com.
A tree code for calculating second-order correlation functions.
athena: Tree code for second-order correlation functions
Martin Kilbinger, CEA Saclay, Service d'Astrophysique (SAp), France
METHOD athena is a 2d-tree code written in C, which estimates second-order correlation functions from input galaxy catalogues. These include shear-shear correlations (cosmic shear), position-shear (galaxy-galaxy lensing) and position-position (spatial angular correlation).
Added FITS file support. Input catalogues and output correlation function files can be both in ascii or fits format. (Note: If reading a FITS file causes a segmentation fault, remove the compiler option "-std=c99", either from CMakeLists.txt or src/Makefile.athena".)
Format of resample files changed, only relevant columns are output.
Compilation of code automated using cmake. Alternatively, the traditional Makefile is still usable.
Directory structure changed.
To compile and run the code, you need a C-compiler. To calculate the angular correlation function, including reading mask files and creating random catalogues, gsl and perl and required. The library cfitsio is optional (for FITS file support).
Further scripts are part of the athena package:
The python script pallas.py calculates (band-)power spectrum by integrating over the correlation function using an estimator from this paper. Further, the aperture-mass dispersion is compuated, also via integrating the correlation function.
The perl script woftheta_xcorr.pl is the master script for angular correlation function calculations. It creates random catalogues and calls athena for all necessary combinations of data and random catalogues, including redshift bins, and outputs the Landy & Szalay (1993) and Hamilton (1993) estimators of the correlation function.
Two perl scripts (cat2gal.pl and center_gal.pl) calculate projections of an input catalogue in spherical coordinates, and transform an arbitrary (ascii) input catalogue into an athena-readable format.
The python script test_suite_athena.py runs a series of tests for easy comparison with expected results.
Various scripts to transform and plot resampled data (e.g. Jackknife)
For older versions of athena please contact me (martin.kilbinger at cea.fr).
With helpful suggestions from Henry McCracken, Lance Miller, and Barnaby Rowe. Ami Choi, Jonathan Benjamin, Matthieu Béthermin, and Shahab Joudaki are thanked for testing the code and bug-hunting.
CONTACT Please feel free to send questions, feedback and bug reports to firstname.lastname@example.org. If you want to be added to the athena mailing list, to get updates about new versions and bug-fixes, send me a mail to email@example.com.
Weak Lensing provides a unique method to directly map the distribution of dark matter in the universe. Ongoing efforts are made to improve the detection of cosmic shear on existing telescopes and future instruments dedicated to survey are planned. Several methods are used to derive the lensing shear from the shapes of background galaxies. But the shear map obtained is always noisy, and when it is converted into a map of the projected mass map, the result is dominated by the noise.
MRLens offers a new algorithm for the reconstruction of Weak Lensing mass maps.
MRLens (Multi-Resolution tools for gravitational Lensing) is a software written in C++ with an IDL interface. This method uses the Multiscale Entropy concept (which is based on wavelets) and the False Discovery Rate (FDR) which allows us to derive robust detection levels in wavelet space. MRLens has been used to process the COSMOS map (see Figure above)..
More than a software dedicated to a new reconstruction method, MRLens software includes many other tools useful to process, analyze and visualize lensing data. The user manual introduces Weak Lensing field and describes the MRLENS tools. Some results are presented and an accurate description of IDL routines are available.
System Requirements : 1- Make sure you have approximately 400 MB of disk space available. After installation MRLENS package occupies approximately 56 MB or 205MB (version with data) of disk space. 2- The binaries C++ called by IDL routines are not available under all the systems therefore you cannot use the package on all platforms. The supported platforms are : SUN-Solaris, PC-Linux, Mac OS X. Next release will include PC Windows.
Software Requirements : The IDL MRLENS software requires that IDL (version 6.0 or later) to be installed on your computer. Starting IDL using the script program mrl.pro allows the user to add the MRLENS software to the IDL environment. Thus, all routines described in the user manual can be called. An online help is available by using the mrh.pro program.oftwares are required:
This package is a compilation of some algorithms and methods which were developed and/or used successfully in the applications reported in the 2 following publications:
Weak Lensing Mass Reconstruction using Wavelets, J.-L. Starck, S. Pires and A. Réfrégier, Astronomy and Astrophysics, March 2006
Map of the universe's Dark Matter scaffolding, R. Massey, J. Rhodes, R. Ellis, N. Scoville, A. Leathaud, A. Finoguenov, P. Capak, D. Bacon, H. Aussel, J.-P. Kneib, A. Koekemoer, H. McCracken, B. Mobasher, S. Pires, A. Réfrégier, S. Sasaki, ,J.-L. Starck, Y. Taniguchi and J. Taylor, Nature, January 2007
Sunyaev-Zeldovich cluster reconstruction in multiband bolometer camera surveys, S. Pires, J.-B. Juin, D. Yvon, Y. Moudden, S. Anthoine and E. Pierpaoli, Astronomy and Astrophysics, April 2006 More than a software dedicated to a new reconstruction method, this package includes many other tools useful to process, analyze and visualize lensing data.
Please acknowledge use of the code in any resulting work, citing Starck, et al, 2006. We would be interested to collaborate with anyone requiring more advanced applications, and are always interested to hear about new applications. For questions and feedback or to be informed of the forthcoming versions, send an email to Sandrine Pires.
The analysis of weak lensing data requires to account for missing data such as masking out of bright stars. To date, the majority of lensing analyses uses the two point-statistics of the cosmic shear field. These can either be studied directly using the two-point correlation function, or in Fourier space, using the power spectrum. The two-point correlation function is unbiased by missing data but its direct calculation will soon become a burden with the exponential growth of astronomical data sets. The power spectrum is fast to estimate but a mask correction should be estimated. Others statistics can be used but these are strongly sensitive to missing data.
The solution that is proposed by FASTLens is to properly fill-in the gaps with only NlogN operations, leading to a complete weak lensing mass map from which we can compute straight forwardly and with a very good accuracy any kind of statistics like power spectrum or bispectrum. The inpainting method relies strongly on the notion of sparsity and on the construction of sparse representations in large redundant dictionaries.
Simulated mass map with the mask pattern of CFHTLS data on D1 field (on the left), inpainted maps map (on the right).
FASTLens (Fast STatistics for weak Lensing) is a package written in C++ that includes:
- An inpainting code to derive complete weak lensing mass maps from incomplete shear maps
- A power spectrum estimator
- A bispectrum estimator (for equilateral and isoscele configurations)
We propose also a new method to compute fastly and accurately the power spectrum and the bispectrum with a polar FFT algorithm.
The user manual introduces the missing data problem in statistic estimation and presents the available routines. An accurate description of IDL routines is given.
The IDL FASTlens software requires IDL (version 6.0 or later) to be installed on your computer. The binaries C++ called by IDL routines are not available under all the systems therefore you cannot use the package on all platforms. The supported platforms are : PC-Linux and Mac OS X.
Please acknowledge use of the code in any resulting work, citing Pires, et al, 2009. We would be interested to collaborate with anyone requiring more advanced applications, and are always interested to hear about new applications. For questions and feedback or to be informed of the forthcoming versions, send an email to Sandrine Pires.
A weak lensing mass-mapping tool that implements sparsity-based regularisation.
Glimpse is a weak lensing mass-mapping tool relying a robust sparsity-based regularisation scheme to recover high resolution convergence from either gravitational shear alone or from a combination of shear and flexion. Including flexion allows us to supplement the shear on small scales in order to increase the sensitivity to substructures and the overall resolution of the convergence map.
In order to preserve all available small-scale information, Glimpse avoids any binning of the irregularly sampled input shear and flexion fields and treats the mass-mapping problem as a general ill-posed inverse problem, regularised using a multi-scale wavelet sparsity prior. The resulting algorithm incorporates redshift, reduced shear, and reduced flexion measurements for individual galaxies and is made highly efficient by the use of fast Fourier estimators.
Test on realistic simulated dark matter distributions
Glimpse was tested on a set of realistic weak lensing simulations corresponding to typical HST/ACS cluster observations and demonstrate our ability to recover substructures with the inclusion of flexion which are lost if only shear information is used. In particular, we can detect substructures at the 15′′scale well outside of the critical region of the clusters. In addition, flexion also helps to constrain the shape of the central regions of the main dark matter halos. These simulations, along with the reconstructions produced by Glimpse can be found in this archive : flexion_benchmark.tar.lzma.
Applications to real data
A520 Cluster Merger
We have used Glimpse to reconstruct the mass distribution of Abell 520, a merging galaxy cluster system also known as the 'cosmic train wreck'. We obtained high-resolution mass maps using two separate galaxy catalogs derived from HST observations and compared the results to previous weak-lensing studies of the system.
The galaxy catalogs in FITS format and configuration files for Glimpse can be downloaded here. Example outputs are included for each data set.
To generate the convergence map of the C12 data with a regularization parameter of 3.0, for example, edit the config_A520_c12.ini file and set the 'lambda'option equal to 3.0 under [parameters]. Then run $ glimpse config_A520_c12.ini A520_cat_c12.fits kappa.fits
to obtain an output convergence map called 'kappa.fits'.
DES SV data
Glimpse has been used to map the matter density field of the Dark Energy Survey (DES) Science Verification (SV) data. The Glimpse reconstruction was compared to two other mass-mapping methods: standard Kaiser-Squires inversion and the Wiener filter.
F. Lanusse, J.-L. Starck, A. Leonard, S. Pires, High Resolution Weak Lensing Mass-Mapping Combining Shear and Flexion, 2016, arXiv:1603.01599
A. Peel, F. Lanusse, J.-L. Starck, Sparse Reconstruction of the Merging A520 Cluster System, 2017, arXiv:1708.00269
N. Jeffrey, F. B. Abdalla, O. Lahav, F. Lanusse, J.-L. Starck, Improving Weak Lensing Mass Map Reconstructions using Gaussian and Sparsity Priors: Application to DES SV, 2018, arXiv:1801.08945