New Features and Important Changes in ctools 1.6.0 18 October 2018 Introduction ------------ This document briefly summarizes the changes that have been made in this ctools release with respect to the precedent release. Configuration ------------- Keep existing library path in installation script on Mac OS X (#2420). Python interface ---------------- ctools and cscripts can now be pickled from Python (#1938). Support model ------------- None Base classes ------------ The ctool base class now loads the application parameters from the file provided in the "syspfiles" folder and updates parameter values using the file found in the user's "pfiles" folder, so that any change to the parameter file structure is handled transparently and no longer leads to an exception (#2513). The ctool::time_reference constant was removed since it is not required anymore. The time reference is now transparently handled by the tools, and if there is a choice, a "mjdref" parameter was added to the tools (#2054). The ctool::create_cta_obs() method now queries the optional "mjdref" parameter so that a MJD reference time can be specified by the tool. This functionality is now used by ctobssim (#2054). The ctools::get_gti() method now takes GTimeReference as argument to enable tools to retrieve Mission Elapsed Time (MET) in the time reference of the observation (#2054). The ctools::is_valid_filename() method was removed since the validity check of the filename is now done by the GApplicationPar class. ctbin - CTA event binning ------------------------- ctbin now accouts for varying energy thresholds when computing the weight array (#2685). Add hidden "nthreads" parameter to control parallel computations (#2421). Copy the time reference from the first input observation (#2054). ctbkgcube - CTA background cube generation ------------------------------------------ Add ctbkgcube::cntcube() method so that a counts cube be specified as instance before running the tool. This avoids storing the counts cube on disk if the counts cube definition should be used to generate a background cube. In addition, the background cube values are now devided by the weight values of the counts cube since later, during model evaluation, the model is multiplied by the weight values. This makes sure that the normalisation is correct (#2685). ctbutterfly - Generate butterfly diagrams ----------------------------------------- Add hidden "like_accuracy" parameter to allow control of the accuracy of the maximum log-likelikelihood value (#2662). ctcubemask - CTA cube filtering ------------------------------- None ctedispcube - CTA energy dispersion cube generation --------------------------------------------------- None cterror - Likelihood profile error computation ---------------------------------------------- Add hidden "like_accuracy" parameter to allow control of the accuracy of the maximum log-likelikelihood value (#2662). Add hidden "nthreads" parameter to control parallel computations (#2421). ctexpcube - CTA exposure cube generation ---------------------------------------- None ctlike - Maximum likelihood fitting ----------------------------------- Add hidden "like_accuracy" parameter to allow control of the accuracy of the maximum log-likelikelihood value (#2662). Add hidden "nthreads" parameter to control parallel computations (#2421). ctmapcube - CTA map cube generation ----------------------------------- None ctmodel - CTA counts cube modelling ----------------------------------- The tools now computes DETX and DETY for every event bin if they are not available so that model that rely on the existence of these parameters can be handled (#2694). Add hidden "nthreads" parameter to control parallel computations (#2421). ctobssim - CTA observation simulation ------------------------------------- Add hidden "nthreads" parameter to control parallel computations (#2421). Add "mjdref" parameter to allow specification of the MJD reference time for the simulation (#2054). ctphase - Event phase computation --------------------------------- None ctprob - Event probability computation -------------------------------------- None ctpsfcube - CTA point spread function cube generation ----------------------------------------------------- None ctselect - CTA event selection ------------------------------ Do no longer set missing energy and RoI boundaries in ctool::get_observations and ctool::setup_observations since this led to a problem in ctselect. For H.E.S.S. data, where the boundaries are not set, the boundaries were preset with the User parameters, which made event selection with the usepnt=no option impossible. It is not clear why the setting of missing boundaries was introduced but it looks like a dangerous thing to do. The setting has therefore been removed (#2686). Avoid an error in case that the RoI radius is identical to the existing RoI radius (#2682). Interpret MET specified by "tmin" and "tmax" parameters in the time reference of the input observations (#2054). ctskymap - CTA sky mapping -------------------------- Add the "inmap", "iterations" and "threshold" parameters. The hidden "inmap" parameter allows specifying a sky map file containing a "COUNTS" and "ACCEPTANCE" extension which avoids looping over event lists in an observation container. The "iterations" parameter allows the iterative determination of an exclusion map from the resulting significance map for the RING background method. The exclusion map will flag all pixels with significances above the "threshold" value so that they will be excluded in the subsequent RING background determination. Since this will impact the pixel significances, the procedure needs to be repeated a couple of times to stabilise (#2311). Accelerate the IRF and RING background methods. For the RING background method the computation is either performed using a FFT or using a direct computation. The FFT computation is faster, but assumed Euclidian distances for the sky map grid, the direct computation is slower, but accurately computes the distances between sky map pixels (#2309). cttsmap - Generation of Test Statistic map ------------------------------------------ Add hidden "like_accuracy" parameter to allow control of the accuracy of the maximum log-likelikelihood value (#2662). ctulimit - Compute upper limit ------------------------------ Add hidden "like_accuracy" parameter to allow control of the accuracy of the maximum log-likelikelihood value (#2662). Add "parname" parameter to allow upper limit computations also for a node function spectral model. In addition, it is now checked that the parameter values actually bracket the upper limit (#2233). support - Support classes ------------------------- None csbkgmodel - Generate background model for 3D analysis ------------------------------------------------------ Introduced the script that generates a background for a 3D analysis. This script is particularly useful to generate a background model for the analysis of the H.E.S.S. public data release (#2694). cscaldb - Inspect calibration database -------------------------------------- None csebins - Generate energy boundaries ------------------------------------ None csfindobs - Find IACT observations ---------------------------------- None csiactcopy - Copy IACT observations ----------------------------------- None csiactdata - Inspect IACT data store ------------------------------------ None csiactobs - Inspect IACT observation definition XML file -------------------------------------------------------- None cslightcrv - Generate light curve --------------------------------- Add hidden "use_irf_bkg" parameter for forwarding to csphagen. Add hidden "nthreads" parameter and implement parallelisation of light curve computation (#2421). Add hidden "mjdref" parameter for the specification of the MET reference (#2054). csmodelinfo - Return model information -------------------------------------- None csmodelmerge - Merge models --------------------------- None csmodelselect - Select models ----------------------------- None csmodelsois - Puts selected sources in diffuse model cube --------------------------------------------------------- None csobs2caldb - Generate calibration database entry for an IACT observation ------------------------------------------------------------------------- Change background template table name from "BGD" to "BKG" (#2343). csobsdef - Generate observation definition XML file --------------------------------------------------- Add hidden "mjdref" parameter for the specification of the MET reference (#2054). csobsinfo - Return observation definition information ----------------------------------------------------- Add ras() and decs() methods (#2313) csobsselect - Select observations --------------------------------- Use time reference of input observations if time selection is given in MET (#2054). csphagen - Generate On/Off observations --------------------------------------- Add hidden "use_irf_bkg" parameter. By default the parameter is set to "yes", which means that the background acceptance provided in the IRF is used for the computation of the alpha parameter and the background rate, stored in the "BACKSCAL" column of the On spectrum and the "BACKRESP" column of the Off spectrum, respectively. If the parameter is set to "no" it is assumed that the background rate in the On and Off regions are the same, and the IRF background template is not used. In that case, the "BACKSCAL" column gives the ratio of the solid angles between On and Off regions, and the "BACKRESP" column is set to zero. The resulting data can only be fitted using the "wstat" statistic (#2659). Add hidden "nthreads" parameter and implement parallelisation of computations (#2421). The true number of energy bins is now computed using int(bins_per_decade * num_of_decades + 0.5) making sure in addition that the result is at least 1. This provides better control over the number of energy bins (#2490). The script now produces data that are compliant with Xspec. An Xspec howto tutorial was added to the user documentation (#2404). csphasecrv - Generate phase curve --------------------------------- Add hidden "nthreads" parameter and implement parallelisation of computations (#2421). cspull - Generate pull distributions ------------------------------------ Add hidden "nthreads" parameter and implement parallelisation of computations (#2421). Add hidden "mjdref" parameter for the specification of the MET reference (#2054). csresmap - Generate residual map -------------------------------- None csresspec - Generate residual spectrum -------------------------------------- An exception occured when using a pre-computed model cube via the modcube parameter due to a bug in the code. This bug was fixed (#2448). csroot2caldb - Generates calibration database from ROOT file ------------------------------------------------------------ Change energy dispersion energy axis name from "ETRUE" to "ENERG" (#2670). Change background template table name from "BGD" to "BKG" (#2343). The ROOT Python module is now imported in the run method and not globally so that no conflit arises with the "--help" option (#2652). cssens - Computes sensitivity curve ----------------------------------- Add hidden "nthreads" parameter and implement parallelisation of computations (#2421). csspec - Generates Spectral Energy Distribution ----------------------------------------------- The script no longer skips energy bins with zero counts (#2671). The full true energy range of the RMF is now used in an On/Off analysis to prevent cutting off the tails of the energy dispersion, which happened before when the script was used for the analysis of H.E.S.S. data (#2656). Add hidden "nthreads" parameter and implement parallelisation of computations (#2421). csspec now properly computes the Test Statistic for an On/Off analysis using the "wstat" statistic in the case that the input model definition XML file did not contain a background model (#2312). csspec now computes upper limits also for the NODES method by fixing all node intensities and varying each node until the log-likelihood increases by a certain amount (#2233). cssrcdetect - Detects sources in sky map ---------------------------------------- The script was made more robust by optionally convolving the input sky map with a smoothing kernel with a specified correlation radius (options are a "DISK" or a "GAUSSIAN" kernel, default is now "DISK"). In addition, the mean and standard deviation of the pixel values are now computed locally over a given radius, controlled through the "avgrad" parameter. This accomodates for counts variations in the field of view (#2498). cstsdist - Computes Test Statistic distribution ----------------------------------------------- Add hidden "nthreads" parameter and implement parallelisation of computations (#2421). cstsmapmerge - Merge Test Statistic maps ---------------------------------------- None cstsmapsplit - Prepare split Test Statistic computation ------------------------------------------------------- Add hidden "like_accuracy" parameter to allow control of the accuracy of the maximum log-likelikelihood value (#2662). csviscube - Generate visibility cube ------------------------------------ Add hidden "mjdref" parameter for the specification of the MET reference (#2054). csworkflow - Execute workflow ----------------------------- None calutils - Calibration utilities -------------------------------- None ioutils - Input/Output utilities -------------------------------- None modutils - Model utilities -------------------------- None mputils - Multiprocessing utilities ----------------------------------- Introduce module (#2421). obsutils - Observation utilities -------------------------------- Add "mjdref" parameter to set_obs() function (#2054). Examples -------- Handle zero model values for SUBVID method in show_response.py script (#2456). Added show_rmf.py script. caldb ----- None models ------ None