Category: Weak Lensing Software

Software related to WL research.

shear bias

shear bias

Posted on by Martin Kilbinger

 

Authors:  M. Kilbinger, A. Pujol
Language: Python
Download: GitHub
Description: shear_bias is a package that contains tools and scripts for shear bias estimation for weak gravitational lensing analysis.

Installation

Download the code from the github repository.

git clone https://github.com/CosmoStat/shear_bias

A directory shear_bias is created. There, call the setup script to install the package.

cd shear_bias
python setup.py install
MGCNN

MGCNN

Posted on by Samuel Farrens

 

Authors: F. Lalande, A. Peel
Language: Python 3
Download: mgcnn.tar.gz
Description: A Convolutional Neural Network (CNN) architecture for classifying standard and modified gravity (MG) cosmological models based on the weak-lensing convergence maps they produce.

Introduction

This repository contains the code and data used to produce the results in A. Peel et al. (2018), arXiv:1810.11030.

The Convolutional Neural Network (CNN) is implemented in Keras using TensorFlow as backend. Since the DUSTGRAIN-pathfinder simulations are not yet public, we are not able to include the original convergence maps obtained from the various cosmological runs. We do provide, however, the wavelet PDF datacubes derived for the four models as described in the paper: one standard LCDM and three modified gravity f(R) models.

Requirements

  • Python 3
  • numpy
  • Keras with Tensorflow as backend
  • scikit-learn

Usage

$ python3 train_mgcnn.py -n0

The three options for the noise flag "-n" are (0, 1, 2), which correspond to noise standard deviations of sigma = (0, 0.35, 0.70) added to the original convergence maps. Additional options are "-i" and "-e" for the number of training iterations and epochs, respectively.

Confusion matrices and evaluation metrics (loss function and validation accuracy) are saved as numpy arrays in the generated output/ directory after each iteration.

Nicaea

Nicaea

Posted on by Samuel Farrens

 

Authors: M. Kilbinger
Language: C
Download: github/nicaea
Description: Numerical routines to calculate cosmology and weak-lensing quantities.
Notes:  

nicaea [ni'kaia]: NumerIcal Cosmology And lEnsing cAlculationsNice coat of arms

Martin Kilbinger, CEA Saclay, Service d'Astrophysique (SAp), France

METHOD

nicaea is a C-code providing numerical routines to calculate cosmology and weak-lensing quantities and functions from theoretical models of the large-scale structure. nicaea is the base of the cosmology module of the CosmoPMC package.

DOWNLOAD

Get the latest stable version by cloning the most recent version from github . A readme file (types .rst, .html, .pdf and other) is included in the package. Check also readthedocs for documentation. New features in version 2.7 (Feb 2017):

  • New lensing projection types: extended Limber, spherical-sky prefactor, second-order Limber, full projection (Kilbinger et al. 2017, arXiv:1702.05301)
  • Photometric redshift errors (so far supported Gaussian with second Gaussian for outliers)
  • Modification of halomodel: mass function now normalized to physical volume (new division by a^3)
  • Added CMB normalization A_s
  • Added options to lensingdemo

For older versions of nicaea please contact me (martin.kilbinger at cea.fr). Note that v2.6 was skipped, the previous released version is 2.5

REFERENCES

There is no dedicated paper that describes nicaea. To reference nicaea, please use the following publication: arXiv:0810.5129, in which something that resembles the first version of nicaea has been used.

AUTHORS

Martin Kilbinger
Karim Benabed (error propagation, code design)
Jean Coupon (HOD, halomodel)
Henry J. McCracken (HOD)
Liping Fu (decomp_eb)
François Lanusse (many enhancements and interface additions)

CONTACT INFORMATION
Please feel free to send questions, feedback and bug reports to martin.kilbinger@cea.fr. If you want to be added to the nicaea mailing list, to get updates about new versions and bug-fixes, send me a mail to martin.kilbinger@cea.fr.

Links
CosmoPMC (cosmology sampling with Population Monte Carlo [PMC])
pmclib (Population Monte Carlo library)
camelus (Model for weak-lensing peak counts)
athena (tree code for second-order correlations)
reduced-shear corrections
home

Last updated April 2018.

Athena

Athena

Posted on by Martin Kilbinger

 

Authors: M. Kilbinger
Language: C
Download: athena_1.7.tgz
Description: A tree code for calculating second-order correlation functions.
Notes:  

athena: Tree code for second-order correlation functions

athena olive tree Martin Kilbinger, CEA Saclay, Service d'Astrophysique (SAp), France

METHOD athena is a 2d-tree code written in C, which estimates second-order correlation functions from input galaxy catalogues. These include shear-shear correlations (cosmic shear), position-shear (galaxy-galaxy lensing) and position-position (spatial angular correlation).

  DOWNLOAD Get the latest version athena_1.7.tgz. A readme file is available. Run the code on the test data set. New features and bug fixes in version 1.7 (Mar 2014):

  • General
    • Added FITS file support. Input catalogues and output correlation function files can be both in ascii or fits format.
      (Note: If reading a FITS file causes a segmentation fault, remove the compiler option "-std=c99", either from CMakeLists.txt or src/Makefile.athena".)
    • Format of resample files changed, only relevant columns are output.
    • Compilation of code automated using cmake. Alternatively, the traditional Makefile is still usable.
    • Directory structure changed.

To compile and run the code, you need a C-compiler. To calculate the angular correlation function, including reading mask files and creating random catalogues, gsl and perl and required. The library cfitsio is optional (for FITS file support).

Further scripts are part of the athena package:

  • The python script pallas.py calculates (band-)power spectrum by integrating over the correlation function using an estimator from this paper. Further, the aperture-mass dispersion is compuated, also via integrating the correlation function.
  • The perl script woftheta_xcorr.pl is the master script for angular correlation function calculations. It creates random catalogues and calls athena for all necessary combinations of data and random catalogues, including redshift bins, and outputs the Landy & Szalay (1993) and Hamilton (1993) estimators of the correlation function.
  • Two perl scripts (cat2gal.pl and center_gal.pl) calculate projections of an input catalogue in spherical coordinates, and transform an arbitrary (ascii) input catalogue into an athena-readable format.
  • The python script test_suite_athena.py runs a series of tests for easy comparison with expected results.
  • Various scripts to transform and plot resampled data (e.g. Jackknife)

  • For older versions of athena please contact me (martin.kilbinger at cea.fr).

REFERENCES

AUTHORS
Martin Kilbinger
Christopher Bonnett (gal-gal lensing)
Jean Coupon (venice)

With helpful suggestions from Henry McCracken, Lance Miller, and Barnaby Rowe. Ami Choi, Jonathan Benjamin, Matthieu Béthermin, and Shahab Joudaki are thanked for testing the code and bug-hunting.

CONTACT
Please feel free to send questions, feedback and bug reports to martin.kilbinger@cea.fr. If you want to be added to the athena mailing list, to get updates about new versions and bug-fixes, send me a mail to martin.kilbinger@cea.fr. 

LINKS
CosmoPMC (cosmology sampling with Population Monte Carlo [PMC])
nicaea (cosmology and lensing package)
reduced-shear corrections home

Last updated February 2017.

Camelus

Camelus

Posted on by Martin Kilbinger

 

Authors: Chieh-An Lin
Language: C
Download: GitHub
Description: A code for fast weak lensing peak count modelling.
Notes:  

Counts of Amplified Mass Elevations from Lensing with Ultrafast Simulations
Chieh-An Lin (University of Edinburgh)

Camelus

Description

Camelus is a fast weak lensing peak count modeling in C. It provides a prediction on peak counts from input cosmological parameters.

Here is the summary of the algorithm:

  • Sample halos from a mass function
  • Assign density profiles, randomize their positions
  • Compute the projected mass, add noise
  • Make maps and create peak catalogues

For a more detailed description, please take a look at Lin & Kilbinger (2015a).

Downloads

Please check the GitHub page of Camelus.

Requirements

The following softwares are required:

Updates

Current release: Camelus v1.31

New features in v1.31 - Mar 22, 2016:

  • Made installation more friendly by removing the dependency on cfitsio and mpi
  • Added the routine for computing 1-halo & 2-halo terms of the convergence profile
  • Flexible parameter space for PMC ABC
  • Remove files: FITSFunctions.c/.h

New features in v1.3 - Dec 09, 2015:

  • New files: constraint.c/.h
  • Allowed multiscale peaks in one data vector
  • Allowed a data matrix from several realizations
  • Used the local galaxy density as the noise level in the S/N
  • Increased the parameter dimension for PMC ABC
  • Changed the summary statistic options for PMC ABC

New features in v1.2 - Apr 06, 2015:

  • Improved the computation speed by a factor of 6~7
  • Converted the halo array structure into a binned structure, called "halo_map"
  • Converted the galaxy tree structure into a binned structure, called "gal_map"
  • Added the population Monte Carlo approximate Monte Carlo (PMC ABC) algorithm

New features in v1.1 - Jan 19, 2015:

  • Fixed the bug from calculating halo radii

New features in v1.0 - Oct 24, 2014:

  • Fast weak lensing peak count modeling

References

Contact information

Authors:

Please feel free to send questions, feedback and bug reports to calin AT roe DOT ac DOT uk.

Last updated Jun 26, 2015.

MRLens

MRLens

Posted on by Jean-Luc Starck

 

Authors: S. Pires, J-L. Starck, A. Réfrégier
Language: C++ (IDL wrapper)
Download: mrle.tar.gz
Description: A weak lensing mass-mapping tool.
Notes: Documentation: ManualMRLens.pdf

Weak Lensing provides a unique method to directly map the distribution of dark matter in the universe. Ongoing efforts are made to improve the detection of cosmic shear on existing telescopes and future instruments dedicated to survey are planned. Several methods are used to derive the lensing shear from the shapes of background galaxies. But the shear map obtained is always noisy, and when it is converted into a map of the projected mass map, the result is dominated by the noise.

MRLens offers a new algorithm for the reconstruction of Weak Lensing mass maps.

Convergence Map in the COSMOS field reconstructed with MRLens

Description 

MRLens (Multi-Resolution tools for gravitational Lensing) is a software written in C++ with an IDL interface. This method uses the Multiscale Entropy concept (which is based on wavelets) and the False Discovery Rate (FDR) which allows us to derive robust detection levels in wavelet space. MRLens has been used to process the COSMOS map (see Figure above)..

User Manual

More than a software dedicated to a new reconstruction method, MRLens software includes many other tools useful to process, analyze and visualize lensing data. The user manual introduces Weak Lensing field and describes the MRLENS tools. Some results are presented and an accurate description of IDL routines are available.

Downloads

Fast download :  (Only binaries)

Standard Download :  (Binaries and data)

System Requirements : 1- Make sure you have approximately 400 MB of disk space available. After installation MRLENS package occupies approximately 56 MB or 205MB (version with data) of disk space.
2- The binaries C++ called by IDL routines are not available under all the systems therefore you cannot use the package on all platforms. The supported platforms are : SUN-Solaris, PC-Linux, Mac OS X. Next release will include PC Windows.

Software Requirements : The IDL MRLENS software requires that IDL (version 6.0 or later) to be installed on your computer. Starting IDL using the script program mrl.pro allows the user to add the MRLENS software to the IDL environment.
Thus, all routines described in the user manual can be called.
An online help is available by using the mrh.pro program.oftwares are required:

References

This package is a compilation of some algorithms and methods which were developed and/or used successfully in the applications reported in the 2 following publications:

Weak Lensing Mass Reconstruction using Wavelets, J.-L. Starck, S. Pires and A. Réfrégier, Astronomy and Astrophysics, March 2006

Map of the universe's Dark Matter scaffolding, R. Massey, J. Rhodes, R. Ellis, N. Scoville, A. Leathaud, A. Finoguenov, P. Capak, D. Bacon, H. Aussel, J.-P. Kneib, A. Koekemoer, H. McCracken, B. Mobasher, S. Pires, A. Réfrégier, S. Sasaki, ,J.-L. Starck, Y. Taniguchi and J. Taylor, Nature, January 2007

Sunyaev-Zeldovich cluster reconstruction in multiband bolometer camera surveys, S. Pires, J.-B. Juin, D. Yvon, Y. Moudden, S. Anthoine and E. Pierpaoli, Astronomy and Astrophysics, April 2006
More than a software dedicated to a new reconstruction method, this package includes many other tools useful to process, analyze and visualize lensing data.

Acknowledging MRLens

Please acknowledge use of the code in any resulting work, citing Starck, et al, 2006. We would be interested to collaborate with anyone requiring more advanced applications, and are always interested to hear about new applications. For questions and feedback or to be informed of the forthcoming versions, send an email to Sandrine Pires.

Contact information

Authors:

Last modified on January 6th, 2015 by Sandrine Pires
For questions and feedback or to be informed of the forthcoming versions, send an email to Sandrine Pires

FASTLens

FASTLens

Posted on by Jean-Luc Starck

 

Authors: S. Pires, J-L. Starck, A. Amara, A. Réfrégier, J. Fadili
Language: C++ (IDL wrapper)
Download: CEA_Inpainting.tar.gz, CEA_PolarSpectrum.tar.gz
Description: A weak lensing mass-mapping tool.
Notes: Documentation: doc_fastlens.pdf

The analysis of weak lensing data requires to account for missing data such as masking out of bright stars. To date, the majority of lensing analyses uses the two point-statistics of the cosmic shear field. These can either be studied directly using the two-point correlation function, or in Fourier space, using the power spectrum. The two-point correlation function is unbiased by missing data but its direct calculation will soon become a burden with the exponential growth of astronomical data sets. The power spectrum is fast to estimate but a mask correction should be estimated. Others statistics can be used but these are strongly sensitive to missing data. 

The solution that is proposed by FASTLens is to properly fill-in the gaps with only NlogN operations, leading to a complete weak lensing mass map from which we can compute straight forwardly and with a very good accuracy any kind of statistics like power spectrum or bispectrum. The inpainting method relies strongly on the notion of sparsity and on the construction of sparse representations in large redundant dictionaries.

Simulated mass map with the mask pattern of CFHTLS data on D1 field (on the left), inpainted maps map (on the right).
 

Description

FASTLens (Fast STatistics for weak Lensing) is a package written in C++ that includes:

- An inpainting code to derive complete weak lensing mass maps from incomplete shear maps

- A power spectrum estimator 

- A bispectrum estimator (for equilateral and isoscele configurations)

We propose also a new method to compute fastly and accurately the power spectrum and the bispectrum with a polar FFT algorithm.

User Manual

The user manual introduces the missing data problem in statistic estimation and presents the available routines. An accurate description of IDL routines is given.

Downloads

The IDL FASTlens software requires IDL (version 6.0 or later) to be installed on your computer. 
The binaries C++ called by IDL routines are not available under all the systems therefore you cannot use the package on all platforms. The supported platforms are : PC-Linux and Mac OS X.

Inpainting routines (inpainting for weak lensing) 

Statistic routines (power spectrum and bispectrum estimators)

References

FASTLens (FAst STatistics for weak Lensing) : Fast method for weak lensing statistics and map making, S. Pires, J.-L. Starck, A. Amara, A. Refregier and J. Fadili, MNRAS,  395, 1265-1279, 2009

Acknowledging FASTLens

Please acknowledge use of the code in any resulting work, citing Pires, et al, 2009. We would be interested to collaborate with anyone requiring more advanced applications, and are always interested to hear about new applications. For questions and feedback or to be informed of the forthcoming versions, send an email to Sandrine Pires.

Contact information

Authors:

Last modified on January 6th, 2015 by Sandrine Pires 
For questions and feedback or to be informed of the forthcoming versions, send an email to Sandrine Pires

 

Glimpse

Glimpse

Posted on by François Lanusse

 

Authors: F. Lanusse
Language: C++
Download: GitHub
Description: A weak lensing mass-mapping tool that implements sparsity-based regularisation.
Notes:  

Glimpse is a weak lensing mass-mapping tool relying a robust sparsity-based regularisation scheme to recover high resolution convergence from either gravitational shear alone or from a combination of shear and flexion. Including flexion allows us to supplement the shear on small scales in order to increase the sensitivity to substructures and the overall resolution of the convergence map.

In order to preserve all available small-scale information, Glimpse avoids any binning of the irregularly sampled input shear and flexion fields and treats the mass-mapping problem as a general ill-posed inverse problem, regularised using a multi-scale wavelet sparsity prior. The resulting algorithm incorporates redshift, reduced shear, and reduced flexion measurements for individual galaxies and is made highly efficient by the use of fast Fourier estimators. 

The source code for Glimpse is made publicly available and is hosted on Github at https://github.com/CosmoStat/Glimpse 

 

Test on realistic simulated dark matter distributions

Glimpse was tested on a set of realistic weak lensing simulations corresponding to typical HST/ACS cluster observations and demonstrate our ability to recover substructures with the inclusion of flexion which are lost if only shear information is used. In particular, we can detect substructures at the 15′′scale well outside of the critical region of the clusters. In addition, flexion also helps to constrain the shape of the central regions of the main dark matter halos. These simulations, along with the reconstructions produced by Glimpse can be found in this archive : flexion_benchmark.tar.lzma.

nbodySimu
Input simulated convergence fields
GlimpseMeanRec
Reconstructed convergence maps using Glimpse. The top row only uses shear information while the bottom row combines shear and flexion to improve the resolution of the recovered maps. Mean of 100 independent noise realizations.

 

Applications to real data

A520 Cluster Merger

We have used Glimpse to reconstruct the mass distribution of Abell 520, a merging galaxy cluster system also known as the 'cosmic train wreck'.  We obtained high-resolution mass maps using two separate galaxy catalogs derived from HST observations and compared the results to previous weak-lensing studies of the system.

Glimpse mass maps reconstructed using the galaxy catalogues of Clowe et al. (2012, left) and Jee et al. (2014, right).

 

The galaxy catalogs in FITS format and configuration files for Glimpse can be downloaded here. Example outputs are included for each data set.

To generate the convergence map of the C12 data with a regularization parameter of 3.0, for example, edit the config_A520_c12.ini file and set the 'lambda' option equal to 3.0 under [parameters]. Then run
$ glimpse config_A520_c12.ini A520_cat_c12.fits kappa.fits
to obtain an output convergence map called 'kappa.fits'.

DES SV data

Glimpse has been used to map the matter density field of the Dark Energy Survey (DES) Science Verification (SV) data. The Glimpse reconstruction was compared to two other mass-mapping methods: standard Kaiser-Squires inversion and the Wiener filter.

Mass map reconstructed from DES SV data using Glimpse.

Publications

  • F. Lanusse, J.-L. Starck, A. Leonard, S. Pires, High Resolution Weak Lensing Mass-Mapping Combining Shear and Flexion, 2016, arXiv:1603.01599
  • A. Peel, F. Lanusse, J.-L. Starck, Sparse Reconstruction of the Merging A520 Cluster System, 2017, arXiv:1708.00269
  • N. Jeffrey, F. B. Abdalla, O. Lahav, F. Lanusse, J.-L. Starck, Improving Weak Lensing Mass Map Reconstructions using Gaussian and Sparsity Priors: Application to DES SV, 2018, arXiv:1801.08945