Commit 6cea5977ddec301fa9d2110e3ccd1af00fb5a43a

Authored by Jürgen Knödlseder
1 parent da257d99

Integrate code

@@ -13,3 +13,5 @@ Michael Mayer (michael.mayer@physik.hu-berlin.de) @@ -13,3 +13,5 @@ Michael Mayer (michael.mayer@physik.hu-berlin.de)
13 Stefan Ohm (stefan.ohm81@gmail.com) 13 Stefan Ohm (stefan.ohm81@gmail.com)
14 David Sanchez (david.sanchez@lapp.in2p3.fr) 14 David Sanchez (david.sanchez@lapp.in2p3.fr)
15 Hubert Siejkowski (h.siejkowski@cyfronet.pl) 15 Hubert Siejkowski (h.siejkowski@cyfronet.pl)
  16 +Alexander Ziegler (alexander.ziegler@fau.de)
  17 +
ChangeLog
1 -2016-09-26 1 +2016-10-17
2 2
3 * Version 1.2.0 released 3 * Version 1.2.0 released
4 ======================== 4 ========================
5 5
  6 + Add outcovmat parameter to ctlike (#1863)
6 Fix ctbin segfault bug for varying energy thresholds (#1850) 7 Fix ctbin segfault bug for varying energy thresholds (#1850)
7 Make csresmap enumbins parameter hidden (#1844) 8 Make csresmap enumbins parameter hidden (#1844)
8 Rename ctskymap::map() to ctskymap::skymap() (#1828) 9 Rename ctskymap::map() to ctskymap::skymap() (#1828)
1 New Features and Important Changes in ctools 1.2.0 1 New Features and Important Changes in ctools 1.2.0
2 2
3 -26 September 2016 3 +17 October 2016
4 4
5 5
6 Introduction 6 Introduction
@@ -93,6 +93,10 @@ of the tool (#1835). @@ -93,6 +93,10 @@ of the tool (#1835).
93 93
94 The class now derives from ctlikelihood (#1828). 94 The class now derives from ctlikelihood (#1828).
95 95
  96 +The hidden parameter "outcovmat" has been added that by default is set to "NONE".
  97 +If instead a FITS filename is given, ctlike will write the coraviance matrix of
  98 +the fit into a FITS file (#1863).
  99 +
96 100
97 cttsmap - Generation of Test Statistic map 101 cttsmap - Generation of Test Statistic map
98 ------------------------------------------ 102 ------------------------------------------
doc/source/users/reference_manual/ctlike.rst
@@ -49,7 +49,7 @@ ctlike generates an output model XML file that contains the values of the @@ -49,7 +49,7 @@ ctlike generates an output model XML file that contains the values of the
49 best fitting model parameters. For all free parameters, an ``error`` attribute 49 best fitting model parameters. For all free parameters, an ``error`` attribute
50 is added that provides the statistical uncertainty in the parameter estimate. 50 is added that provides the statistical uncertainty in the parameter estimate.
51 In addition, the entire output of the covariance matrix to a separate FITS file 51 In addition, the entire output of the covariance matrix to a separate FITS file
52 -can be queried (parameter ``covmat``). The parameter order given in the file 52 +can be queried (parameter ``outcovmat``). The parameter order given in the file
53 corresponds to the arrangement of the covariance matrix entries. If computation 53 corresponds to the arrangement of the covariance matrix entries. If computation
54 of the Test Statistics has been requested for a model component, a ``ts`` 54 of the Test Statistics has been requested for a model component, a ``ts``
55 attribute providing the Test Statistics value is added. The output model can be 55 attribute providing the Test Statistics value is added. The output model can be
@@ -88,6 +88,9 @@ General parameters @@ -88,6 +88,9 @@ General parameters
88 Output model XML file with values and uncertainties updated by 88 Output model XML file with values and uncertainties updated by
89 the maximum likelihood fit. 89 the maximum likelihood fit.
90 90
  91 +``(outcovmat = NONE) [string]``
  92 + Output FITS file to store covariance matrix.
  93 +
91 ``(stat = POISSON) [string]`` 94 ``(stat = POISSON) [string]``
92 Fitting statistics (POISSON or GAUSSIAN; only affects binned analysis). 95 Fitting statistics (POISSON or GAUSSIAN; only affects binned analysis).
93 96
@@ -97,11 +100,7 @@ General parameters @@ -97,11 +100,7 @@ General parameters
97 ``(fix_spat_for_ts = no) [boolean]`` 100 ``(fix_spat_for_ts = no) [boolean]``
98 Fix spatial parameters for TS computation. 101 Fix spatial parameters for TS computation.
99 102
100 -``(covmat = NONE) [string]``  
101 - Output FITS file to store covariance matrix.  
102 -  
103 103
104 -  
105 Standard parameters 104 Standard parameters
106 ------------------- 105 -------------------
107 106
src/ctlike/ctlike.cpp
@@ -228,19 +228,13 @@ void ctlike::run(void) @@ -228,19 +228,13 @@ void ctlike::run(void)
228 // Optimize model parameters using LM optimizer 228 // Optimize model parameters using LM optimizer
229 optimize_lm(); 229 optimize_lm();
230 230
231 - // Save covariance matrix if valid filename is passed  
232 - if (is_valid_filename(m_covmat)) {  
233 -  
234 - // Write header  
235 - log_header1(TERSE, "Saving covariance matrix");  
236 -  
237 - // Log filename  
238 - log_value(NORMAL, "Covariance matrix file", m_covmat.url());  
239 -  
240 - // Save covarianve matrix  
241 - m_obs.save_covmat(m_covmat);  
242 - } 231 + // Get pointer to curvature matrix
  232 + GMatrixSparse* curvature_ptr =
  233 + const_cast<GObservations::likelihood&>(m_obs.function()).curvature();
243 234
  235 + // Store copy of curvature matrix
  236 + GMatrixSparse curvature = *curvature_ptr;
  237 +
244 // Store Npred 238 // Store Npred
245 double npred = m_obs.npred(); 239 double npred = m_obs.npred();
246 240
@@ -318,6 +312,9 @@ void ctlike::run(void) @@ -318,6 +312,9 @@ void ctlike::run(void)
318 // Restore energy dispersion flags of all CTA observations 312 // Restore energy dispersion flags of all CTA observations
319 restore_edisp(m_obs, save_edisp); 313 restore_edisp(m_obs, save_edisp);
320 314
  315 + // Restore curvature matrix
  316 + *curvature_ptr = curvature;
  317 +
321 // Return 318 // Return
322 return; 319 return;
323 } 320 }
@@ -326,15 +323,17 @@ void ctlike::run(void) @@ -326,15 +323,17 @@ void ctlike::run(void)
326 /***********************************************************************//** 323 /***********************************************************************//**
327 * @brief Save results 324 * @brief Save results
328 * 325 *
329 - * This method saves the fit results in a XML file. 326 + * This method saves the fit results into a XML file and a FITS file. If
  327 + * the filename parameters are "NONE", the files are not saved.
330 ***************************************************************************/ 328 ***************************************************************************/
331 void ctlike::save(void) 329 void ctlike::save(void)
332 { 330 {
333 // Write header 331 // Write header
334 log_header1(TERSE, "Save results"); 332 log_header1(TERSE, "Save results");
335 333
336 - // Get output filename  
337 - m_outmodel = (*this)["outmodel"].filename(); 334 + // Get output filenames
  335 + m_outmodel = (*this)["outmodel"].filename();
  336 + m_outcovmat = (*this)["outcovmat"].filename();
338 337
339 // Save only if filename is valid 338 // Save only if filename is valid
340 if (is_valid_filename(m_outmodel)) { 339 if (is_valid_filename(m_outmodel)) {
@@ -344,6 +343,7 @@ void ctlike::save(void) @@ -344,6 +343,7 @@ void ctlike::save(void)
344 343
345 // Write results out as XML model 344 // Write results out as XML model
346 m_obs.models().save(m_outmodel); 345 m_obs.models().save(m_outmodel);
  346 +
347 } 347 }
348 348
349 // ... otherwise signal that file was not saved 349 // ... otherwise signal that file was not saved
@@ -351,6 +351,22 @@ void ctlike::save(void) @@ -351,6 +351,22 @@ void ctlike::save(void)
351 log_value(NORMAL, "Model definition file", "NONE"); 351 log_value(NORMAL, "Model definition file", "NONE");
352 } 352 }
353 353
  354 + // Save covariance matrix if filename is valid
  355 + if (is_valid_filename(m_outcovmat)) {
  356 +
  357 + // Log filename
  358 + log_value(NORMAL, "Covariance matrix file", m_outcovmat.url());
  359 +
  360 + // Save covarianve matrix
  361 + m_obs.function().save(m_outcovmat);
  362 +
  363 + }
  364 +
  365 + // ... otherwise signal that no covariance matrix was not saved
  366 + else {
  367 + log_value(NORMAL, "Covariance matrix file", "NONE");
  368 + }
  369 +
354 // Return 370 // Return
355 return; 371 return;
356 } 372 }
@@ -405,12 +421,12 @@ void ctlike::get_parameters(void) @@ -405,12 +421,12 @@ void ctlike::get_parameters(void)
405 m_apply_edisp = (*this)["edisp"].boolean(); 421 m_apply_edisp = (*this)["edisp"].boolean();
406 m_fix_spat_for_ts = (*this)["fix_spat_for_ts"].boolean(); 422 m_fix_spat_for_ts = (*this)["fix_spat_for_ts"].boolean();
407 m_chatter = static_cast<GChatter>((*this)["chatter"].integer()); 423 m_chatter = static_cast<GChatter>((*this)["chatter"].integer());
408 - m_covmat = (*this)["covmat"].filename();  
409 424
410 // Optionally read ahead parameters so that they get correctly 425 // Optionally read ahead parameters so that they get correctly
411 // dumped into the log file 426 // dumped into the log file
412 if (read_ahead()) { 427 if (read_ahead()) {
413 - m_outmodel = (*this)["outmodel"].filename(); 428 + m_outmodel = (*this)["outmodel"].filename();
  429 + m_outcovmat = (*this)["outcovmat"].filename();
414 } 430 }
415 431
416 // Set optimizer logger 432 // Set optimizer logger
@@ -543,7 +559,7 @@ void ctlike::init_members(void) @@ -543,7 +559,7 @@ void ctlike::init_members(void)
543 { 559 {
544 // Initialise members 560 // Initialise members
545 m_outmodel.clear(); 561 m_outmodel.clear();
546 - m_covmat.clear(); 562 + m_outcovmat.clear();
547 m_refit = false; 563 m_refit = false;
548 m_max_iter = 100; // Set maximum number of iterations 564 m_max_iter = 100; // Set maximum number of iterations
549 m_max_stall = 10; // Set maximum number of stalls 565 m_max_stall = 10; // Set maximum number of stalls
@@ -574,7 +590,7 @@ void ctlike::copy_members(const ctlike&amp; app) @@ -574,7 +590,7 @@ void ctlike::copy_members(const ctlike&amp; app)
574 // Copy attributes 590 // Copy attributes
575 m_refit = app.m_refit; 591 m_refit = app.m_refit;
576 m_outmodel = app.m_outmodel; 592 m_outmodel = app.m_outmodel;
577 - m_covmat = app.m_covmat; 593 + m_outcovmat = app.m_outcovmat;
578 m_max_iter = app.m_max_iter; 594 m_max_iter = app.m_max_iter;
579 m_max_stall = app.m_max_stall; 595 m_max_stall = app.m_max_stall;
580 m_logL = app.m_logL; 596 m_logL = app.m_logL;
src/ctlike/ctlike.hpp
@@ -70,10 +70,10 @@ protected: @@ -70,10 +70,10 @@ protected:
70 // User parameters 70 // User parameters
71 bool m_refit; //!< Refitting 71 bool m_refit; //!< Refitting
72 GFilename m_outmodel; //!< Source model output XML file name 72 GFilename m_outmodel; //!< Source model output XML file name
  73 + GFilename m_outcovmat; //!< Covariance matrix output FITS file name
73 bool m_apply_edisp; //!< Apply energy dispersion? 74 bool m_apply_edisp; //!< Apply energy dispersion?
74 bool m_fix_spat_for_ts; //!< Fix spatial parameters for TS computation? 75 bool m_fix_spat_for_ts; //!< Fix spatial parameters for TS computation?
75 GChatter m_chatter; //!< Chattiness 76 GChatter m_chatter; //!< Chattiness
76 - GFilename m_covmat; //!< Covariance matrix output FITS file name  
77 77
78 // Members 78 // Members
79 int m_max_iter; //!< Maximum number of iterations 79 int m_max_iter; //!< Maximum number of iterations
src/ctlike/ctlike.par
@@ -5,7 +5,7 @@ @@ -5,7 +5,7 @@
5 # ------------------------------------------------------------------------- # 5 # ------------------------------------------------------------------------- #
6 # # 6 # #
7 # File: ctlike.par # 7 # File: ctlike.par #
8 -# Version: 1.1.0 # 8 +# Version: 1.2.0 #
9 # # 9 # #
10 # Author: Jurgen Knodlseder # 10 # Author: Jurgen Knodlseder #
11 # Jurgen.Knodlseder@irap.omp.eu # 11 # Jurgen.Knodlseder@irap.omp.eu #
@@ -17,17 +17,17 @@ @@ -17,17 +17,17 @@
17 # 17 #
18 # General parameters 18 # General parameters
19 #=================== 19 #===================
20 -inobs, f, a, "events.fits",,, "Input event list, counts cube or observation definition XML file"  
21 -inmodel, f, a, "$CTOOLS/share/models/crab.xml",,, "Input model XML file"  
22 -expcube, f, a, "NONE",,, "Input exposure cube file (only needed for stacked analysis)"  
23 -psfcube, f, a, "NONE",,, "Input PSF cube file (only needed for stacked analysis)"  
24 -edispcube, f, a, "NONE",,, "Input energy dispersion cube file (only needed for stacked analysis)"  
25 -bkgcube, f, a, "NONE",,, "Input background cube file (only needed for stacked analysis)"  
26 -caldb, s, a, "prod2",,, "Calibration database"  
27 -irf, s, a, "South_0.5h",,, "Instrument response function"  
28 -edisp, b, h, no,,, "Apply energy dispersion?"  
29 -outmodel, f, a, "crab_results.xml",,, "Output model XML file"  
30 -covmat, f, h, "NONE",,, "Output covariance matrix FITS file" 20 +inobs, f, a, "events.fits",,, "Input event list, counts cube or observation definition XML file"
  21 +inmodel, f, a, "$CTOOLS/share/models/crab.xml",,, "Input model XML file"
  22 +expcube, f, a, "NONE",,, "Input exposure cube file (only needed for stacked analysis)"
  23 +psfcube, f, a, "NONE",,, "Input PSF cube file (only needed for stacked analysis)"
  24 +edispcube, f, a, "NONE",,, "Input energy dispersion cube file (only needed for stacked analysis)"
  25 +bkgcube, f, a, "NONE",,, "Input background cube file (only needed for stacked analysis)"
  26 +caldb, s, a, "prod2",,, "Calibration database"
  27 +irf, s, a, "South_0.5h",,, "Instrument response function"
  28 +edisp, b, h, no,,, "Apply energy dispersion?"
  29 +outmodel, f, a, "crab_results.xml",,, "Output model XML file"
  30 +outcovmat, f, h, "NONE",,, "Output covariance matrix FITS file"
31 31
32 # 32 #
33 # Maximisation parameters 33 # Maximisation parameters