PySAP

 

Authors:  S. Farrens, A. Grigis, L. El Gueddari, Z. Ramzi, Chaithya G. R., S. Starck, B. Sarthou, H. Cherkaoui, P.Ciuciu, J-L. Starck
Language: Python
Download: GitHub
Description: PySAP (Python Sparse data Analysis Package) is a Python module for sparse data analysis.
Notes:

PySAP paper


Installation

The installation of PySAP has been extensively tested on Ubuntu and macOS, however we cannot guarantee it will work on every operating system (e.g. Windows).

If you encounter any installation issues be sure to go through the following steps before opening a new issue:

  1. Check that that all of the installed all the dependencies listed above have been installed.
  2. Read through all of the documentation provided, including the troubleshooting suggestions.
  3. Check if you problem has already been addressed in a previous issue.

Further instructions are available here.

From PyPi

To install PySAP simply run:

$ pip install python-pysap

Depending on your Python setup you may need to provide the --user option.

$ pip install --user python-pysap

Locally

To build PySAP locally, clone the repository:

$ git clone https://github.com/CEA-COSMIC/pysap.git

and run:

$ python setup.py install

or:

$ python setup.py develop

As before, use the --user option if needed.

macOS

Help with installation on macOS is available here.

Linux

Please refer to the PyQtGraph homepage for issues regarding the installation of pyqtgraph.

Contributing

If you want to contribute to pySAP, be sure to review the contribution guidelines and follow to the code of conduct.

F-CUR3D

 

Authors: A. Woiselle, J.L. Starck and M.J. Fadili
Language: Matlab
Download: Mac OSX | Windows
Description: A code for fast 3D curvelet transform and reconstruction.
Notes: Documentation: jmiv2010.pdf


Fast Curvelet Transform Version 1.0

Description F-CUR3D is a software, based on the MATLAB package, which contains routines for the Fast 3D Curvelet transform and reconstruction. The F-CUR3D documentation is available in PDF format.

See also the "3D curvelet algorithm description", and examples .

F-CUR3D is available for Windows and MAC.   Publications Papers related to the software:

MRS3D

 

Authors: F. Lanusse, A. Rassat,and J.-L. Starck
Language: C++ (with IDL wrapper)
Download: MRS3D_v1.0b.tar.gz
Description: A code for performing 3D spherical wavelet transforms on the sphere.
Notes: Requires Healpix installation.


Description MRS3D is based on two packages, IDL and Healpix. MRS3D can be used only if these two packages have been installed. MRS3D contains IDL and C++ routines for 3D spherical wavelet transform on the sphere .

Source code and more information are available here. For more information, please contact Francois Lanusse (francois.lanusse@cea.fr).

Publications Paper related to the software:

MRLens

 

Authors: S. Pires, J-L. Starck, A. Réfrégier
Language: C++ (IDL wrapper)
Download: mrle.tar.gz
Description: A weak lensing mass-mapping tool.
Notes: Documentation: ManualMRLens.pdf


Weak Lensing provides a unique method to directly map the distribution of dark matter in the universe. Ongoing efforts are made to improve the detection of cosmic shear on existing telescopes and future instruments dedicated to survey are planned. Several methods are used to derive the lensing shear from the shapes of background galaxies. But the shear map obtained is always noisy, and when it is converted into a map of the projected mass map, the result is dominated by the noise.

MRLens offers a new algorithm for the reconstruction of Weak Lensing mass maps.

Convergence Map in the COSMOS field reconstructed with MRLens

Description 

MRLens (Multi-Resolution tools for gravitational Lensing) is a software written in C++ with an IDL interface. This method uses the Multiscale Entropy concept (which is based on wavelets) and the False Discovery Rate (FDR) which allows us to derive robust detection levels in wavelet space. MRLens has been used to process the COSMOS map (see Figure above)..

User Manual

More than a software dedicated to a new reconstruction method, MRLens software includes many other tools useful to process, analyze and visualize lensing data. The user manual introduces Weak Lensing field and describes the MRLENS tools. Some results are presented and an accurate description of IDL routines are available.

Downloads

Fast download :  (Only binaries)

Standard Download :  (Binaries and data)

System Requirements : 1- Make sure you have approximately 400 MB of disk space available. After installation MRLENS package occupies approximately 56 MB or 205MB (version with data) of disk space.
2- The binaries C++ called by IDL routines are not available under all the systems therefore you cannot use the package on all platforms. The supported platforms are : SUN-Solaris, PC-Linux, Mac OS X. Next release will include PC Windows.

Software Requirements : The IDL MRLENS software requires that IDL (version 6.0 or later) to be installed on your computer. Starting IDL using the script program mrl.pro allows the user to add the MRLENS software to the IDL environment.
Thus, all routines described in the user manual can be called.
An online help is available by using the mrh.pro program.oftwares are required:

References

This package is a compilation of some algorithms and methods which were developed and/or used successfully in the applications reported in the 2 following publications:

Weak Lensing Mass Reconstruction using Wavelets, J.-L. Starck, S. Pires and A. Réfrégier, Astronomy and Astrophysics, March 2006

Map of the universe's Dark Matter scaffolding, R. Massey, J. Rhodes, R. Ellis, N. Scoville, A. Leathaud, A. Finoguenov, P. Capak, D. Bacon, H. Aussel, J.-P. Kneib, A. Koekemoer, H. McCracken, B. Mobasher, S. Pires, A. Réfrégier, S. Sasaki, ,J.-L. Starck, Y. Taniguchi and J. Taylor, Nature, January 2007

Sunyaev-Zeldovich cluster reconstruction in multiband bolometer camera surveys, S. Pires, J.-B. Juin, D. Yvon, Y. Moudden, S. Anthoine and E. Pierpaoli, Astronomy and Astrophysics, April 2006
More than a software dedicated to a new reconstruction method, this package includes many other tools useful to process, analyze and visualize lensing data.

Acknowledging MRLens

Please acknowledge use of the code in any resulting work, citing Starck, et al, 2006. We would be interested to collaborate with anyone requiring more advanced applications, and are always interested to hear about new applications. For questions and feedback or to be informed of the forthcoming versions, send an email to Sandrine Pires.

Contact information

Authors:

Last modified on January 6th, 2015 by Sandrine Pires
For questions and feedback or to be informed of the forthcoming versions, send an email to Sandrine Pires

FASTLens

 

Authors: S. Pires, J-L. Starck, A. Amara, A. Réfrégier, J. Fadili
Language: C++ (IDL wrapper)
Download: CEA_Inpainting.tar.gz, CEA_PolarSpectrum.tar.gz
Description: A weak lensing mass-mapping tool.
Notes: Documentation: doc_fastlens.pdf


The analysis of weak lensing data requires to account for missing data such as masking out of bright stars. To date, the majority of lensing analyses uses the two point-statistics of the cosmic shear field. These can either be studied directly using the two-point correlation function, or in Fourier space, using the power spectrum. The two-point correlation function is unbiased by missing data but its direct calculation will soon become a burden with the exponential growth of astronomical data sets. The power spectrum is fast to estimate but a mask correction should be estimated. Others statistics can be used but these are strongly sensitive to missing data. 

The solution that is proposed by FASTLens is to properly fill-in the gaps with only NlogN operations, leading to a complete weak lensing mass map from which we can compute straight forwardly and with a very good accuracy any kind of statistics like power spectrum or bispectrum. The inpainting method relies strongly on the notion of sparsity and on the construction of sparse representations in large redundant dictionaries.

Simulated mass map with the mask pattern of CFHTLS data on D1 field (on the left), inpainted maps map (on the right).
 

Description

FASTLens (Fast STatistics for weak Lensing) is a package written in C++ that includes:

- An inpainting code to derive complete weak lensing mass maps from incomplete shear maps

- A power spectrum estimator 

- A bispectrum estimator (for equilateral and isoscele configurations)

We propose also a new method to compute fastly and accurately the power spectrum and the bispectrum with a polar FFT algorithm.

User Manual

The user manual introduces the missing data problem in statistic estimation and presents the available routines. An accurate description of IDL routines is given.

Downloads

The IDL FASTlens software requires IDL (version 6.0 or later) to be installed on your computer. 
The binaries C++ called by IDL routines are not available under all the systems therefore you cannot use the package on all platforms. The supported platforms are : PC-Linux and Mac OS X.

Inpainting routines (inpainting for weak lensing) 

Statistic routines (power spectrum and bispectrum estimators)

References

FASTLens (FAst STatistics for weak Lensing) : Fast method for weak lensing statistics and map making, S. Pires, J.-L. Starck, A. Amara, A. Refregier and J. Fadili, MNRAS,  395, 1265-1279, 2009

Acknowledging FASTLens

Please acknowledge use of the code in any resulting work, citing Pires, et al, 2009. We would be interested to collaborate with anyone requiring more advanced applications, and are always interested to hear about new applications. For questions and feedback or to be informed of the forthcoming versions, send an email to Sandrine Pires.

Contact information

Authors:

Last modified on January 6th, 2015 by Sandrine Pires 
For questions and feedback or to be informed of the forthcoming versions, send an email to Sandrine Pires

 

iSAP

 

Authors: F. SureauF. Lanusse and J.-L. Starck
Language: C++
Download: ISAP_V3.1
Description: Interactive Sparse Astronomical Data Analysis Packages.
Notes: Documentation: doc_iSAP_V3.0


Interactive Sparse Astronomical Data Analysis Packages

iSAP is a collection of packages, in IDL and C++, related to sparsity and its application in astronomical data analysis (the IDL software (http://www.idl-envi.com) is analogous to Matlab and is very widely used in astrophysics and in medical imaging). The C++ routines can be used independently of IDL.

The iSAP V3.1 software, including all these packages, can be downloaded here: ISAP_V3.1.
The documentation of iSAP is available in PDF format: doc_iSAP_V3.0

It contains the following packages:

  • Sparse2D V1.0Sparsity for 1D and 2D data sets
    IDL and C++ code, allowing sparse decomposition, denoising and deconvolution.
     
  • MSVST V1.0: Multi-Scale Variance Stabilizing Transform for 1D and 2D data sets
    IDL and C++ code for Poisson noise removal. Matlab code can also be found here.
     
  • MRS V3.2: MultiResolution on the Sphere
    IDL and C++ code for sparse representation on the sphere. More details can be found here.
     
  • SparsePOL V1.1: Polarized Spherical Wavelets and Curvelets
    IDL code for sparse representation of polarized data on the sphere. More details can be found here.
     
  • MRS-MSVSTS V1.1: Multi-Scale Variance Stabilizing Transform on the Sphere
    IDL code for Poisson noise removal and deconvolution on mono-channel and multichannel spherical data. More details can be found here.
     
  • SparseGal V1.0: Sparsity for galaxies survey analysis
    IDL code, with two subpackages:
    • ISW V1.0: Integrated Sachs-Wolfe effect detection.
    • DarthFader V1.0: Spectroscopic Redshift Estimation using sparsity.
  • SparseCMB V1.0: Sparsity for CMB data analysis
    IDL code, with three subpackages:
    • TOUSI V1.0: True Cosmic Microwave Background Power Spectrum Estimator.
    • Anomalies V1.0: CMB large scale anomalies detection.
    • PRISM V1.0: Sparse Recovery of the Primordial Power Spectrum.

SASIR

 

Authors: J. N. Girard, M. Jiang, J-L. Starck, S. Corbel, H. Garsden and A. Woiselle
Language: C++
Download: GitHub
Description: A deconvolution algorithm (written in C++ and Python) dedicated to radio interferometric imaging.
Notes:  


SASIR is a deconvolution algorithm (written in C++ and Python) dedicated to radio interferometric imaging and based on the convex optimization using sparse representations (refered to the framework of Compressed Sensing).

As an alternative to CLEAN, it allows a robust reconstruction of the sky brightness composed of a mix of extended emission and point sources, with improved image resolution and fidelity.

It has been developped and tested in the context of the giant radio interferometer LOFAR. It is being adapted on recent imagers use for LOFAR and other SKA pathfinders/precursors.

Super-resolved image of the radiosource Cygnus A (real data), reconstructed by the new Sparse imager (SASIR).
Super-resolved image of the radiosource Cygnus A (real data), reconstructed by the new Sparse imager (SASIR).

The correct reconstruction of radio images from visibility data is an intense field of research since the coming of new generation radio interferometers such as LOFAR (LOw Frequency Array) and SKA (Square Kilometre Array). These instruments require a correct approach taking into account Direction-(in)Dependent Effects  (such as variation of the beam, polarization, ...).  The mathematical framework for calibration is provided by the RIME, the Radio Interferometer Measurement Equation, (refer to Hamaker, Sault, Bregman series of papers and Smirnov 2011 series) enables a proper handling for data modelling and calibration.

In spite of the high angular/time/frequency resolutions and the large variety of baselines, these interferometers measure a finite number of visibilities over the course of an observation, giving an incomplete frequency sampling of the sky Fourier Transform. This incomplete knowledge of the sky FT creates distorted images when the visibility data are gridded and projected back to the image plane. These images are distorted by the instrumental Point Spread Function (PSF) which encode this lack of information. A robust imaging of radio interferometric data resides in the robustness of the deconvolution algorithm used to remove the effect of this PSF. CLEAN and its derivatives (see family of CLEAN algorithms and associated papers) have been performing this task for decades on point sources will relatively good performance and robustness.

However, Sparse data and the existence of multi-scale radio emission (mix of point source and extended emissions) are obstacles to the deconvolution. The framework of Compressed Sensing offer us an opportunity to redefine the deconvolution problem as an optimization problem (inpainting problem) targeting solutions close to the real sky brightness with the lowest reconstruction bias.

Our approach was to construct an alternative to CLEAN with the implementation of another deconvolution method based on the FISTA algorithm (Beck et Teboulle, 2009).

SASIR (Sparse Aperture Synthesis Image Reconstruction) is an in-painting imager which combines sparse representation with $$l_1$$-minimization.

SASIR 2D-1D is a spin-off of SASIR (based on Condat-Vu algorithm) applied to the reconstruction of transient sources in the image plane.

Project contributorsDr. Julien N. Girard, Ming Jiang, Jean-Luc Starck, Stéphane Corbel and formerly Dr. H. Garsden and Dr. A. Woiselle.

Sparse Aperture Synthesis Image Reconstruction - 2D

SASIR 2D comes in two versions:

  • A C++ stand-alone in the ISAP package (not connected to an imager but using fits files as input/output).
  • A C++ "LOFAR" version integrated in LWimager (check out the github repository) and using Docker containers to facilitate the compilation.
  • A Python/C++ implementation pySASIR is currently being implemented on a separate github repository (to be released soon). Current efforts are focused on the implementation of this deconvolution algorithm as a minor cycle in WSCLEAN (Offringa et al., 2014)  and DDFacet (Tasse et al., to be submitted).

Sparse Aperture Synthesis Image Reconstruction for Radio Transients reconstruction - 2D-1D

SASIR 2D1D will come in one version:

  • A Python/C++ implementation pySASIR-2D1D is currently being implemented on a separate github repository (to be released soon). Its implementation as a minor cycle

Questions on compilation, the usage, access to the paper toy models and examples are available upon request to julien.girard [at] cea.fr.

 

Published

  • H. Garsden, J. N. Girard, J. L. Starck, S. Corbel, C. Tasse, A. Woiselle, J. P. McKean and 74 coauthors.
    LOFAR sparse image reconstruction. Astronomy & Astrophysics, 575:A90, March 2015 ADS
  • J. N. Girard, H. Garsden, J. L. Starck, S. Corbel, A. Woiselle, C. Tasse, J. P. McKean, and J. Bobin.
    Sparse representations and convex optimization as tools for LOFAR radio interferometric imaging. Journal
    of Instrumentation, 10:C8013, August 2015 ADS

Conferences Proceedings

  • M. Jiang, J. N. Girard, J. L. Starck, S. Corbel, and C. Tasse. Compressed sensing and radio interferometry.
    In Signal Processing Conference (EUSIPCO), 2015 23rd European, pages 1646-1650, August 2015 IEEE
  • M. Jiang, J. N. Girard, J.-L. Starck, S. Corbel, and C. Tasse. Interferometric Radio Transient Reconstruction
    in Compressed Sensing Framework. In F. Martins, S. Boissier, V. Buat, L. Cambresy, and P. Petit, editors,
    SF2A-2015: Proceedings of the Annual meeting of the French Society of Astronomy and Astrophysics. Eds.:
    F. Martins, S. Boissier, V. Buat, L. Cambresy, P. Petit, pp.231-236, pages 231-236, December 2015 (ADS)

To be published

  • J. N. Girard, M. Jiang, J. L. Starck, S. Corbel, Sparse spatio-temporal imaging of radio transients.

Acknowledgements: We acknowledge the financial support from the UnivEarthS Labex program of Sorbonne Paris Cité (ANR-10-LABX-0023 and ANR-11-IDEX-0005-02) and from the European Research Council grant SparseAstro (ERC-228261)

The following results are extracted from Garsden et al. 2015.

  • Photometry

Simulated LOFAR dataset containing a grid of 10 x 10 point sources over a field of 8°x8°.

bla
Fig 1 Output vs. input flux density of 100 point sources with Cotton-Schwab CLEAN (red) and SASIR (blue)

  • Effective angular resolution

- Two points sources (one at the phase center and one with a varying distance $$deltatheta$$ from the phase center).

$$deltatheta=[30'' - 3']$$ by steps of $$30''$$.

test
Fig. 2 Numerical test to show the super-resolution capability of SASIR (left) SASIR output image (center) Cotton-Schwab CLEAN components (right) CLEAN beam convolved image. The dash line marks the x coordinate of the phase center.

  • Extended emission (W50)

test
Fig. 3 Reconstruction comparison using CLEAN, MS-CLEAN and SASIR

  • Reconstruction of Cygnus A

test
Fig. N  Comparison of Cygnus A reconstruction with CLEAN, MS-CLEAN and SASIR

test
Fig. N+1 (color map) LOFAR data at 150 MHz (contours) VLA at 325 MHz (Click the see the animated figure)

  • Other (to come)

Baolab

 

Authors: A. Labatie, J.L. Starck, M. Lachieze-Rey
Language: IDL
Download: BAOlab.zip
Description: An IDL code for studying BAO.
Notes: Contains additional C++ routines.


BAOlab is related to the study of Baryon Acoustic Oscillations (BAO) using the 2-point correlation function. It enables to perform different tasks, namely BAO detection and BAO parameter constraints. The main novelty of this approach is that it enables to obtain a model-dependent covariance matrix which can change the results both for BAO detection and for parameter constraints.

Software: BAOlab Version 1.0

  • BAOlab contains IDL and C++ routines.
  • Source code and more information are available here.

Publications

Papers related to the software:

Darth Fader

 

Authors: D. Machado, A. Leonard, J-L. Starck, and F. Abdalla
Language: IDL
Download: DFV1.0.tgz
Description: An IDL code designed for estimating galaxy redshifts from spectroscopic data using wavelets.
Notes: Requires iSAP installation


The Darth Fader method is a catalog cleaning method for redshift estimation which is described in:

Code

The code will be soon integrated in the next version of the iSAP software. Meanwhile, it can be used following these instructions:

The documentation is available here.

Results

Benchmark data are available here, and the following routine shows how to use it on these benchmark data:

Running this routine, we obtain the following results:

  • % of catastrophic failures before cleaning = 22.09
  • % of galaxies retained after cleaning = 75.80
  • % of catastrophic failures after cleaning = 4.29

Contact information

For any information related to the code, please contact Adrienne Leonard (adrienne.leonard AT cea DOT fr).