Euclid: Reconstruction of weak-lensing mass maps for non-Gaussianity studies
|Authors:||S. Pires, V. Vandenbussche, V. Kansal, R. Bender, L. Blot, D. Bonino, A. Boucaud, J. Brinchmann, V. Capobianco, J. Carretero, M. Castellano, S. Cavuoti, R. Clédassou, G. Congedo, L. Conversi, L. Corcione, F. Dubath, P. Fosalba, M. Frailis, E. Franceschi, M. Fumana, F. Grupp, F. Hormuth, S. Kermiche, M. Knabenhans, R. Kohley, B. Kubik, M. Kunz, S. Ligori, P.B. Lilje, I. Lloro, E. Maiorano, O. Marggraf, R. Massey, G. Meylan, C. Padilla, S. Paltani, F. Pasian, M. Poncet, D. Potter, F. Raison, J. Rhodes, M. Roncarelli, R. Saglia, P. Schneider, A. Secroun, S. Serrano, J. Stadel, P. Tallada Crespí, I. Tereno, R. Toledo-Moreo, Y. Wang|
|Journal:||Astronomy and Astrophysics|
Weak lensing, namely the deflection of light by matter along the line of sight, has proven to be an efficient method to constrain models of structure formation and reveal the nature of dark energy. So far, most weak lensing studies have focused on the shear field that can be measured directly from the ellipticity of background galaxies. However, within the context of forthcoming full-sky weak lensing surveys such as Euclid, convergence maps (mass maps) offer an important advantage over shear fields in terms of cosmological exploitation. While carrying the same information, the lensing signal is more compressed in the convergence maps than in the shear field, simplifying otherwise computationally expensive analyses, for instance non-Gaussianity studies. However, the inversion of the non-local shear field requires accurate control of systematic effects due to holes in the data field, field borders, noise and the fact that the shear is not a direct observable (reduced shear). In this paper, we present the two mass inversion methods that are being included in the official Euclid data processing pipeline: the standard Kaiser & Squires method (KS) and a new mass inversion method (KS+) that aims to reduce the information loss during the mass inversion. This new method is based on the KS methodology and includes corrections for mass mapping systematic effects. The results of the KS+ method are compared to the original implementation of the KS method in its simplest form, using the Euclid Flagship mock galaxy catalogue. In particular, we estimate the quality of the reconstruction by comparing the two-point correlation functions, third- and fourth-order moments obtained from shear and convergence maps, and we analyse each systematic effect independently and simultaneously. We show that the KS+ method reduces substantially the errors on the two-point correlation function and moments compared to the KS method. In particular, we show that the errors introduced by the mass inversion on the two-point correlation of the convergence maps are reduced by a factor of about 5 while the errors on the third- and fourth-order moments are reduced by a factor of about 2 and 10 respectively.
This repository contains the code and data used to produce the results in A. Peel et al. (2018), arXiv:1810.11030.
The Convolutional Neural Network (CNN) is implemented in Keras using TensorFlow as backend. Since the DUSTGRAIN-pathfinder simulations are not yet public, we are not able to include the original convergence maps obtained from the various cosmological runs. We do provide, however, the wavelet PDF datacubes derived for the four models as described in the paper: one standard LCDM and three modified gravity f(R) models.
- Python 3
- Keras with Tensorflow as backend
$ python3 train_mgcnn.py -n0
The three options for the noise flag "-n" are (0, 1, 2), which correspond to noise standard deviations of sigma = (0, 0.35, 0.70) added to the original convergence maps. Additional options are "-i" and "-e" for the number of training iterations and epochs, respectively.
Confusion matrices and evaluation metrics (loss function and validation accuracy) are saved as numpy arrays in the generated output/ directory after each iteration.
Martin Kilbinger, CEA Saclay, Service d'Astrophysique (SAp), France
nicaea is a C-code providing numerical routines to calculate cosmology and weak-lensing quantities and functions from theoretical models of the large-scale structure. nicaea is the base of the cosmology module of the CosmoPMC package.
Get the latest stable version by cloning the most recent version from github . A readme file (types .rst, .html, .pdf and other) is included in the package. Check also readthedocs for documentation. New features in version 2.7 (Feb 2017):
- New lensing projection types: extended Limber, spherical-sky prefactor, second-order Limber, full projection (Kilbinger et al. 2017, arXiv:1702.05301)
- Photometric redshift errors (so far supported Gaussian with second Gaussian for outliers)
- Modification of halomodel: mass function now normalized to physical volume (new division by a^3)
- Added CMB normalization A_s
- Added options to lensingdemo
For older versions of nicaea please contact me (martin.kilbinger at cea.fr). Note that v2.6 was skipped, the previous released version is 2.5
There is no dedicated paper that describes nicaea. To reference nicaea, please use the following publication: arXiv:0810.5129, in which something that resembles the first version of nicaea has been used.
Karim Benabed (error propagation, code design)
Jean Coupon (HOD, halomodel)
Henry J. McCracken (HOD)
Liping Fu (decomp_eb)
François Lanusse (many enhancements and interface additions)
Please feel free to send questions, feedback and bug reports to email@example.com. If you want to be added to the nicaea mailing list, to get updates about new versions and bug-fixes, send me a mail to firstname.lastname@example.org.
CosmoPMC (cosmology sampling with Population Monte Carlo [PMC])
pmclib (Population Monte Carlo library)
camelus (Model for weak-lensing peak counts)
athena (tree code for second-order correlations)
Last updated April 2018.
athena: Tree code for second-order correlation functions
METHOD athena is a 2d-tree code written in C, which estimates second-order correlation functions from input galaxy catalogues. These include shear-shear correlations (cosmic shear), position-shear (galaxy-galaxy lensing) and position-position (spatial angular correlation).
- Added FITS file support. Input catalogues and output correlation function files can be both in ascii or fits format.
(Note: If reading a FITS file causes a segmentation fault, remove the compiler option "-std=c99", either from CMakeLists.txt or src/Makefile.athena".)
- Format of resample files changed, only relevant columns are output.
- Compilation of code automated using cmake. Alternatively, the traditional Makefile is still usable.
- Directory structure changed.
- Added FITS file support. Input catalogues and output correlation function files can be both in ascii or fits format.
To compile and run the code, you need a C-compiler. To calculate the angular correlation function, including reading mask files and creating random catalogues, gsl and perl and required. The library cfitsio is optional (for FITS file support).
Further scripts are part of the athena package:
- The python script pallas.py calculates (band-)power spectrum by integrating over the correlation function using an estimator from this paper. Further, the aperture-mass dispersion is compuated, also via integrating the correlation function.
- The perl script woftheta_xcorr.pl is the master script for angular correlation function calculations. It creates random catalogues and calls athena for all necessary combinations of data and random catalogues, including redshift bins, and outputs the Landy & Szalay (1993) and Hamilton (1993) estimators of the correlation function.
- Two perl scripts (cat2gal.pl and center_gal.pl) calculate projections of an input catalogue in spherical coordinates, and transform an arbitrary (ascii) input catalogue into an athena-readable format.
- The python script test_suite_athena.py runs a series of tests for easy comparison with expected results.
- Various scripts to transform and plot resampled data (e.g. Jackknife)
For older versions of athena please contact me (martin.kilbinger at cea.fr).
- athena on the Astrophysics Source Code Library: ascl link, ads link.
- pallas: Schneider, van Waerbeke, Kilbinger & Mellier, 2002, A&A, 396, 1
With helpful suggestions from Henry McCracken, Lance Miller, and Barnaby Rowe. Ami Choi, Jonathan Benjamin, Matthieu Béthermin, and Shahab Joudaki are thanked for testing the code and bug-hunting.
Please feel free to send questions, feedback and bug reports to email@example.com. If you want to be added to the athena mailing list, to get updates about new versions and bug-fixes, send me a mail to firstname.lastname@example.org.
Last updated February 2017.
Counts of Amplified Mass Elevations from Lensing with Ultrafast Simulations
Chieh-An Lin (University of Edinburgh)
Camelus is a fast weak lensing peak count modeling in C. It provides a prediction on peak counts from input cosmological parameters.
Here is the summary of the algorithm:
- Sample halos from a mass function
- Assign density profiles, randomize their positions
- Compute the projected mass, add noise
- Make maps and create peak catalogues
For a more detailed description, please take a look at Lin & Kilbinger (2015a).
Please check the GitHub page of Camelus.
The following softwares are required:
Current release: Camelus v1.31
New features in v1.31 - Mar 22, 2016:
- Made installation more friendly by removing the dependency on cfitsio and mpi
- Added the routine for computing 1-halo & 2-halo terms of the convergence profile
- Flexible parameter space for PMC ABC
- Remove files: FITSFunctions.c/.h
New features in v1.3 - Dec 09, 2015:
- New files: constraint.c/.h
- Allowed multiscale peaks in one data vector
- Allowed a data matrix from several realizations
- Used the local galaxy density as the noise level in the S/N
- Increased the parameter dimension for PMC ABC
- Changed the summary statistic options for PMC ABC
New features in v1.2 - Apr 06, 2015:
- Improved the computation speed by a factor of 6~7
- Converted the halo array structure into a binned structure, called "halo_map"
- Converted the galaxy tree structure into a binned structure, called "gal_map"
- Added the population Monte Carlo approximate Monte Carlo (PMC ABC) algorithm
New features in v1.1 - Jan 19, 2015:
- Fixed the bug from calculating halo radii
New features in v1.0 - Oct 24, 2014:
- Fast weak lensing peak count modeling
- Bartelmann & Schneider (2001). Phys. Rep., 340, 291.
- Fan et al. (2010). ApJ, 719, 1408.
- Lin & Kilbinger (2017), submitted to A&A.
- Lin, Kilbinger & Pires (2016), A&A, 593, A88.
- Lin & Kilbinger (2015a). A&A, 576, A24.
- Lin & Kilbinger (2015b), 583, A70.
- Lin & Kilbinger (2015c), ascl:1502.015 (software page)
- Peel, Lin et al. (2017), 599, A79.
- Marin et al. (2011).
- Takada & Jain (2003a). MNRAS, 340, 580.
- Weyant et al. (2013). ApJ, 764, 116.
Please feel free to send questions, feedback and bug reports to calin AT roe DOT ac DOT uk.
Last updated Jun 26, 2015.
Weak Lensing provides a unique method to directly map the distribution of dark matter in the universe. Ongoing efforts are made to improve the detection of cosmic shear on existing telescopes and future instruments dedicated to survey are planned. Several methods are used to derive the lensing shear from the shapes of background galaxies. But the shear map obtained is always noisy, and when it is converted into a map of the projected mass map, the result is dominated by the noise.
MRLens offers a new algorithm for the reconstruction of Weak Lensing mass maps.
MRLens (Multi-Resolution tools for gravitational Lensing) is a software written in C++ with an IDL interface. This method uses the Multiscale Entropy concept (which is based on wavelets) and the False Discovery Rate (FDR) which allows us to derive robust detection levels in wavelet space. MRLens has been used to process the COSMOS map (see Figure above)..
More than a software dedicated to a new reconstruction method, MRLens software includes many other tools useful to process, analyze and visualize lensing data. The user manual introduces Weak Lensing field and describes the MRLENS tools. Some results are presented and an accurate description of IDL routines are available.
Fast download : (Only binaries)
Standard Download : (Binaries and data)
System Requirements : 1- Make sure you have approximately 400 MB of disk space available. After installation MRLENS package occupies approximately 56 MB or 205MB (version with data) of disk space.
2- The binaries C++ called by IDL routines are not available under all the systems therefore you cannot use the package on all platforms. The supported platforms are : SUN-Solaris, PC-Linux, Mac OS X. Next release will include PC Windows.
Software Requirements : The IDL MRLENS software requires that IDL (version 6.0 or later) to be installed on your computer. Starting IDL using the script program mrl.pro allows the user to add the MRLENS software to the IDL environment.
Thus, all routines described in the user manual can be called.
An online help is available by using the mrh.pro program.oftwares are required:
This package is a compilation of some algorithms and methods which were developed and/or used successfully in the applications reported in the 2 following publications:
Weak Lensing Mass Reconstruction using Wavelets, J.-L. Starck, S. Pires and A. Réfrégier, Astronomy and Astrophysics, March 2006
Map of the universe's Dark Matter scaffolding, R. Massey, J. Rhodes, R. Ellis, N. Scoville, A. Leathaud, A. Finoguenov, P. Capak, D. Bacon, H. Aussel, J.-P. Kneib, A. Koekemoer, H. McCracken, B. Mobasher, S. Pires, A. Réfrégier, S. Sasaki, ,J.-L. Starck, Y. Taniguchi and J. Taylor, Nature, January 2007
Sunyaev-Zeldovich cluster reconstruction in multiband bolometer camera surveys, S. Pires, J.-B. Juin, D. Yvon, Y. Moudden, S. Anthoine and E. Pierpaoli, Astronomy and Astrophysics, April 2006
More than a software dedicated to a new reconstruction method, this package includes many other tools useful to process, analyze and visualize lensing data.
Please acknowledge use of the code in any resulting work, citing Starck, et al, 2006. We would be interested to collaborate with anyone requiring more advanced applications, and are always interested to hear about new applications. For questions and feedback or to be informed of the forthcoming versions, send an email to Sandrine Pires.
The analysis of weak lensing data requires to account for missing data such as masking out of bright stars. To date, the majority of lensing analyses uses the two point-statistics of the cosmic shear field. These can either be studied directly using the two-point correlation function, or in Fourier space, using the power spectrum. The two-point correlation function is unbiased by missing data but its direct calculation will soon become a burden with the exponential growth of astronomical data sets. The power spectrum is fast to estimate but a mask correction should be estimated. Others statistics can be used but these are strongly sensitive to missing data.
The solution that is proposed by FASTLens is to properly fill-in the gaps with only NlogN operations, leading to a complete weak lensing mass map from which we can compute straight forwardly and with a very good accuracy any kind of statistics like power spectrum or bispectrum. The inpainting method relies strongly on the notion of sparsity and on the construction of sparse representations in large redundant dictionaries.
FASTLens (Fast STatistics for weak Lensing) is a package written in C++ that includes:
- An inpainting code to derive complete weak lensing mass maps from incomplete shear maps
- A power spectrum estimator
- A bispectrum estimator (for equilateral and isoscele configurations)
We propose also a new method to compute fastly and accurately the power spectrum and the bispectrum with a polar FFT algorithm.
The user manual introduces the missing data problem in statistic estimation and presents the available routines. An accurate description of IDL routines is given.
The IDL FASTlens software requires IDL (version 6.0 or later) to be installed on your computer.
The binaries C++ called by IDL routines are not available under all the systems therefore you cannot use the package on all platforms. The supported platforms are : PC-Linux and Mac OS X.
Inpainting routines (inpainting for weak lensing)
Statistic routines (power spectrum and bispectrum estimators)
FASTLens (FAst STatistics for weak Lensing) : Fast method for weak lensing statistics and map making, S. Pires, J.-L. Starck, A. Amara, A. Refregier and J. Fadili, MNRAS, 395, 1265-1279, 2009
Please acknowledge use of the code in any resulting work, citing Pires, et al, 2009. We would be interested to collaborate with anyone requiring more advanced applications, and are always interested to hear about new applications. For questions and feedback or to be informed of the forthcoming versions, send an email to Sandrine Pires.
Glimpse is a weak lensing mass-mapping tool relying a robust sparsity-based regularisation scheme to recover high resolution convergence from either gravitational shear alone or from a combination of shear and flexion. Including flexion allows us to supplement the shear on small scales in order to increase the sensitivity to substructures and the overall resolution of the convergence map.
In order to preserve all available small-scale information, Glimpse avoids any binning of the irregularly sampled input shear and flexion fields and treats the mass-mapping problem as a general ill-posed inverse problem, regularised using a multi-scale wavelet sparsity prior. The resulting algorithm incorporates redshift, reduced shear, and reduced flexion measurements for individual galaxies and is made highly efficient by the use of fast Fourier estimators.
The source code for Glimpse is made publicly available and is hosted on Github at https://github.com/CosmoStat/Glimpse
Test on realistic simulated dark matter distributions
Glimpse was tested on a set of realistic weak lensing simulations corresponding to typical HST/ACS cluster observations and demonstrate our ability to recover substructures with the inclusion of flexion which are lost if only shear information is used. In particular, we can detect substructures at the 15′′scale well outside of the critical region of the clusters. In addition, flexion also helps to constrain the shape of the central regions of the main dark matter halos. These simulations, along with the reconstructions produced by Glimpse can be found in this archive : flexion_benchmark.tar.lzma.
Applications to real data
A520 Cluster Merger
We have used Glimpse to reconstruct the mass distribution of Abell 520, a merging galaxy cluster system also known as the 'cosmic train wreck'. We obtained high-resolution mass maps using two separate galaxy catalogs derived from HST observations and compared the results to previous weak-lensing studies of the system.
The galaxy catalogs in FITS format and configuration files for Glimpse can be downloaded here. Example outputs are included for each data set.
To generate the convergence map of the C12 data with a regularization parameter of 3.0, for example, edit the config_A520_c12.ini file and set the 'lambda' option equal to 3.0 under [parameters]. Then run
$ glimpse config_A520_c12.ini A520_cat_c12.fits kappa.fits
to obtain an output convergence map called 'kappa.fits'.
DES SV data
Glimpse has been used to map the matter density field of the Dark Energy Survey (DES) Science Verification (SV) data. The Glimpse reconstruction was compared to two other mass-mapping methods: standard Kaiser-Squires inversion and the Wiener filter.
- F. Lanusse, J.-L. Starck, A. Leonard, S. Pires, High Resolution Weak Lensing Mass-Mapping Combining Shear and Flexion, 2016, arXiv:1603.01599
- A. Peel, F. Lanusse, J.-L. Starck, Sparse Reconstruction of the Merging A520 Cluster System, 2017, arXiv:1708.00269
- N. Jeffrey, F. B. Abdalla, O. Lahav, F. Lanusse, J.-L. Starck, Improving Weak Lensing Mass Map Reconstructions using Gaussian and Sparsity Priors: Application to DES SV, 2018, arXiv:1801.08945