Commit a0bc9bdf4397173fbe301dacec03224575318355

Authored by Jürgen Knödlseder
1 parent 70701752

Correct analysis scripts

ChangeLog
  1 +2014-11-24 Juergen Knoedlseder <jurgen.knodlseder@irap.omp.eu>
  2 +
  3 + * Version 1.0.0 released
  4 + ========================
  5 +
  6 + Review and adapt example scripts
  7 +
  8 +
1 2014-11-08 Juergen Knoedlseder <jurgen.knodlseder@irap.omp.eu> 9 2014-11-08 Juergen Knoedlseder <jurgen.knodlseder@irap.omp.eu>
2 10
3 * Version 0.8.0 released 11 * Version 0.8.0 released
1 -New Features and Important Changes in ctools 00-08-00 1 +New Features and Important Changes in ctools 1.0.0
2 2
3 Author(s) : Juergen Knoedlseder 3 Author(s) : Juergen Knoedlseder
4 4
5 -8 November 2014 5 +24 November 2014
6 6
7 7
8 1. Introduction 8 1. Introduction
@@ -10,16 +10,6 @@ Author(s) : Juergen Knoedlseder @@ -10,16 +10,6 @@ Author(s) : Juergen Knoedlseder
10 This document briefly summarizes executable by executable the most 10 This document briefly summarizes executable by executable the most
11 important changes made since the last release of ctools. 11 important changes made since the last release of ctools.
12 12
13 -For all ctools, the public method interface was cleaned up so that  
14 -only methods that are really useful for an outside client are  
15 -exposed. Low-level methods that are used by the application are now  
16 -protected, as improper usage of these methods may lead to  
17 -unpredictable results.  
18 -  
19 -The ctool base class has been introduced and all tools now derive  
20 -from ctool instead of GApplication. This allows to avoid code  
21 -duplications and makes the code generally slimmer and more readable.  
22 -  
23 13
24 2. Configuration 14 2. Configuration
25 ----------------- 15 -----------------
@@ -33,108 +23,32 @@ None @@ -33,108 +23,32 @@ None
33 23
34 4. Support model 24 4. Support model
35 ---------------- 25 ----------------
36 -A support module has been introduced that will contain common classes  
37 -and functions. For the moment the only class in this module is the  
38 -ctool base class. 26 +None
39 27
40 28
41 5. ctobssim - CTA observation simulation 29 5. ctobssim - CTA observation simulation
42 ---------------------------------------- 30 ----------------------------------------
43 -ctobssim was not fully thread save as the models were shared among the  
44 -threads which led to concurrent memory access when evaluating the Monte  
45 -Carlo cache. The models are now copied within the parallel section,  
46 -avoiding concurrent memory access (#1258).  
47 -  
48 -ctobssim now also supports the handling of energy dispersion. Energy  
49 -dispersion can be switched on or off using the hidden edisp parameter.  
50 -  
51 -The memory needs for ctobssim have been reduced by first working on a  
52 -local copy of the CTA observation (this de-allocates properly and memory  
53 -cache associated to the response functions), and by saving and disposing  
54 -the event lists immediately if called using the execute() method (this  
55 -is the method called when the tool is run from the command line). 31 +None
56 32
57 33
58 6. ctselect - CTA event selection 34 6. ctselect - CTA event selection
59 --------------------------------- 35 ---------------------------------
60 -The temporary file is now properly close to avoid file opening errors  
61 -when iterating over ctselect (#1149).  
62 -  
63 -Non-CTA time references are now correctly handled by ctselect. While the  
64 -tmin and tmax parameters are given in CTA reference time, they are  
65 -internally converted to the reference time of the GTI before event  
66 -selection (#1329).  
67 -  
68 -ctselect now supports save and user energy thresholds (#1039). There  
69 -are three different possibilities for specifying a threshold. They  
70 -are toggled using the hidden usethres parameter:  
71 -  
72 -NONE: Use the emin and emax parameters to select an energy range.  
73 - This is the default mode which is identical to the previous  
74 - workings. The only difference is now that if the emin and/or  
75 - emax parameter is set to 0.0, the threshold will be ignored.  
76 - This allows to select all events above or below a given  
77 - threshold.  
78 -  
79 -DEFAULT: Use the save threshold that is found in the effective  
80 - area FITS header keywords LO_THRES and HI_THRES. If the  
81 - emin and emax parameters are non-zero, they will provide  
82 - an additional energy constraint.  
83 -  
84 -USER: Use the user specified threshold that are found in the  
85 - XML observation definition file. If the emin and emax  
86 - parameters are non-zero, they will provide an additional  
87 - energy constraint.  
88 -  
89 -Note that any existing energy boundaries will be used as envelope,  
90 -i.e. a wider energy threshold specification will have no effect. 36 +None
91 37
92 38
93 7. ctbin - CTA event binning 39 7. ctbin - CTA event binning
94 ---------------------------- 40 ----------------------------
95 -WARNING: The philosophy behind the ctbin tool changed fundamentally!  
96 -  
97 -ctbin now always produces a single counts cube on output. If a single  
98 -event list is specified on input, the functionnality is basically  
99 -unchanged. If multiple event lists are specified, however, events are  
100 -now merged into a single counts cube file. No observation definition  
101 -XML file is written anymore!  
102 -  
103 -So far, ctbin still writes pointing information in the counts cube,  
104 -although this is not necessarily meaningful if merging event lists  
105 -with different pointings. In the long run, pointing information will  
106 -be summarized in a pointings definition file, and will be removed  
107 -from the counts cube header. The logic is that the pointing definition  
108 -is set to the map centre. This is only identical to the pointing in  
109 -case of binning a single event list and using the usepnt=yes option  
110 -(or specifying the pointing manually as the xref and yref parameters).  
111 -  
112 -Memory usage has been reduced by disposing the event lists after their  
113 -usage. 41 +None
114 42
115 43
116 8. ctlike - Maximum likelihood fitting 44 8. ctlike - Maximum likelihood fitting
117 -------------------------------------- 45 --------------------------------------
118 -ctlike now has full access to the calibration database, allowing  
119 -the use for 2D response functions in FITS files.  
120 -  
121 -ctlike now also supports the handling of energy dispersion. Energy  
122 -dispersion can be switched on or off using the hidden edisp parameter. 46 +None
123 47
124 48
125 9. cttsmap - Generation of Test Statistics map 49 9. cttsmap - Generation of Test Statistics map
126 ---------------------------------------------- 50 ----------------------------------------------
127 -The cttsmap tool has been added that provides a test statistics map  
128 -for a given set of observations. This tool is not restricted to CTA  
129 -data and will work for all types of instruments.  
130 -  
131 -cttsmap will move a test sources over a specified grid of positions  
132 -and compute the likelihood difference between having the source at  
133 -a given position and having not the source. Twice the likelihood  
134 -difference is called the Test Statistic. The computation is done for  
135 -a grid of positions, and the results is saved into a FITS file as  
136 -TS map. Also the value of the fitted parameters are stored as  
137 -extension in the FITS file. 51 +None
138 52
139 53
140 10. ctskymap - CTA sky mapping 54 10. ctskymap - CTA sky mapping
@@ -144,105 +58,45 @@ None @@ -144,105 +58,45 @@ None
144 58
145 11. ctmodel - CTA counts cube modelling 59 11. ctmodel - CTA counts cube modelling
146 --------------------------------------- 60 ---------------------------------------
147 -WARNING: The philosophy behind the ctmodel tool changed fundamentally!  
148 -  
149 -ctmodel now always produces a single model cube on output. There are  
150 -two important input parameters: "infile" and "obsfile". "infile"  
151 -specifies a counts or model cube that is used to define the binning  
152 -of the model cube. The functionnality for this parameter is as before.  
153 -If "infile=NONE", the cube will be defined according to task parameters  
154 -that specify the spatial and the energy binning. The "obsfile"  
155 -parameter is new and allows specifying the observation definition  
156 -for which the model cube should be computed. If an event list or a  
157 -counts cube is presented, the pointing and Good Time Intervals will  
158 -be extracted from the FITS file header. If an observation definition  
159 -XML file is presented, ctmodel will now loop over all CTA observations  
160 -in the file and add the model values to the model cube. If no  
161 -response information is associated to the "obsfile" (which is typically  
162 -the case for event lists or counts cubes), the "caldb" and "irf"  
163 -parameters will be used to identify the response.  
164 -  
165 -ctmodel now has full access to the calibration database, allowing  
166 -the use for 2D response functions in FITS files.  
167 -  
168 -ctmodel now also supports the handling of energy dispersion. Energy  
169 -dispersion can be switched on or off using the hidden edisp parameter.  
170 -  
171 -Memory usage has been reduced by disposing the event lists after their  
172 -usage. 61 +None
173 62
174 63
175 12. ctexpcube - CTA exposure cube generation 64 12. ctexpcube - CTA exposure cube generation
176 -------------------------------------------- 65 --------------------------------------------
177 -The first version of the ctexpcube tool is included in the release.  
178 -ctexpcube computes the exposure cube for an observation definition  
179 -file. The cube definition may be taken from a counts cube or may be  
180 -specified by user parameters. 66 +None
181 67
182 68
183 13. ctpsfcube - CTA point spread function cube generation 69 13. ctpsfcube - CTA point spread function cube generation
184 --------------------------------------------------------- 70 ---------------------------------------------------------
185 -The first version of the ctpsfcube tool is included in the release.  
186 -ctpsfcube computes the point spread function cube for an observation  
187 -definition file. The cube definition may be taken from a counts cube  
188 -or may be specified by user parameters. 71 +None
189 72
190 73
191 14. ctbkgcube - CTA background cube generation 74 14. ctbkgcube - CTA background cube generation
192 ---------------------------------------------- 75 ----------------------------------------------
193 -The first version of the ctbkgcube tool is included in the release.  
194 -ctbkgcube computes the background cube for an observation definition  
195 -file. The cube definition may be taken from a counts cube or may be  
196 -specified by user parameters. The tools also updates the input model  
197 -by replacing all IACT background model components by a single  
198 -background cube background model with a power law as spectral  
199 -parameter. The model container is available to Python using the  
200 -models() method or can be written into an XML file on disk. 76 +None
201 77
202 78
203 15. ctcubemask - CTA cube filtering 79 15. ctcubemask - CTA cube filtering
204 ----------------------------------- 80 -----------------------------------
205 -The first version of the ctcubemask tool is included in the release.  
206 -ctcubemask allows filtering of bins in a counts cube by setting the  
207 -bin content to -1. Filtering can be done by energy, for a circular  
208 -Region of Interest (ROI) or using ds9 exclusion regions. 81 +None
209 82
210 83
211 16. ctbutterfly - Generate butterfly diagrams 84 16. ctbutterfly - Generate butterfly diagrams
212 ============================================= 85 =============================================
213 -The first version of the ctbutterfly tool is included in the release.  
214 -This tools derives a butterfly diagram from the fitted model parameters  
215 -for one specified source. The result will be written into an ASCII  
216 -file. 86 +None
217 87
218 88
219 17. Examples 89 17. Examples
220 ------------ 90 ------------
221 -Added the following example scripts:  
222 -* pipeline_stacked.py (simulation and stacked analysis) 91 +The following scripts have been adapted to fit the actual
  92 +interface definition
  93 +* make_binned_analysis.py
  94 +* make_unbinned_analysis.py
223 95
224 96
225 18. Scripts 97 18. Scripts
226 ----------- 98 -----------
227 -Add logL and event statistics to cspull output file.  
228 -  
229 -Convert all tabs into spaces.  
230 -  
231 -The obsutils script has now been moved into the ctools module namespace.  
232 -  
233 -The following functions have been added to obsutils:  
234 -* set_obs() replaces the now obsolete set() function  
235 -* set_obs_list() returns an observation container filled with a list of  
236 - observations  
237 -* set_obs_patterns() defines standard patterns of CTA observations.  
238 - Current options are:  
239 - - "single": single pointing  
240 - - "four": four pointings offset by specific angle around centre  
241 -* spectrum() generates spectral points by fitting a source model in  
242 - specified energy bins.  
243 -  
244 -Add the csresmap script that uses ctbin and ctmodel to generate a  
245 -residual counts cube for a given model. 99 +None
246 100
247 101
248 19. caldb 102 19. caldb
examples/make_binned_analysis.py
@@ -4,7 +4,7 @@ @@ -4,7 +4,7 @@
4 # ctools. If matplotlib is installed, a counts spectrum and an offset 4 # ctools. If matplotlib is installed, a counts spectrum and an offset
5 # histogram will be displayed. 5 # histogram will be displayed.
6 # 6 #
7 -# Copyright (C) 2011-2012 Jurgen Knodlseder 7 +# Copyright (C) 2011-2014 Jurgen Knodlseder
8 # 8 #
9 # This program is free software: you can redistribute it and/or modify 9 # This program is free software: you can redistribute it and/or modify
10 # it under the terms of the GNU General Public License as published by 10 # it under the terms of the GNU General Public License as published by
@@ -20,404 +20,404 @@ @@ -20,404 +20,404 @@
20 # along with this program. If not, see <http://www.gnu.org/licenses/>. 20 # along with this program. If not, see <http://www.gnu.org/licenses/>.
21 # 21 #
22 # ========================================================================== 22 # ==========================================================================
23 -from ctools import *  
24 -from gammalib import *  
25 -from math import * 23 +import gammalib
  24 +import ctools
26 import os 25 import os
27 import glob 26 import glob
28 import sys 27 import sys
  28 +import math
29 try: 29 try:
30 - import matplotlib.pyplot as plt  
31 - has_matplotlib = True 30 + import matplotlib.pyplot as plt
  31 + has_matplotlib = True
32 except: 32 except:
33 - has_matplotlib = False 33 + has_matplotlib = False
34 34
35 35
36 # ============================= # 36 # ============================= #
37 # Analysis pipeline - version 1 # 37 # Analysis pipeline - version 1 #
38 # ============================= # 38 # ============================= #
39 def pipeline_v1(): 39 def pipeline_v1():
40 - """  
41 - Binned analysis pipeline - save intermediate results. 40 + """
  41 + Binned analysis pipeline - save intermediate results.
42 42
43 This function implements an analysis pipeline that successively calls 43 This function implements an analysis pipeline that successively calls
44 ctobssim, ctbin and ctlike by saving the intermediate results as FITS 44 ctobssim, ctbin and ctlike by saving the intermediate results as FITS
45 files on disk. This replicates the way how the analysis would be done 45 files on disk. This replicates the way how the analysis would be done
46 in a ftools-like approach. 46 in a ftools-like approach.
47 - """  
48 - # Set script parameters  
49 - model_name = "${CTOOLS}/share/models/crab.xml"  
50 - events_name = "events.fits"  
51 - cntmap_name = "cntmap.fits"  
52 - result_name = "results.xml"  
53 - caldb = "${CALDB}/data/cta/dummy/bcf"  
54 - irf = "cta_dummy_irf"  
55 - ra = 83.63  
56 - dec = 22.01  
57 - rad_sim = 10.0  
58 - tstart = 0.0  
59 - tstop = 1800.0  
60 - emin = 0.1  
61 - emax = 100.0  
62 - enumbins = 20  
63 - nxpix = 200  
64 - nypix = 200  
65 - binsz = 0.02  
66 - coordsys = "CEL"  
67 - proj = "CAR"  
68 -  
69 - # Initialise timing  
70 - wall_seconds = 0.0  
71 - cpu_seconds = 0.0  
72 -  
73 - # Simulate events  
74 - sim = ctobssim()  
75 - sim.logFileOpen() # We need this to explicitely open the log file in Python mode  
76 - sim["infile"].filename(model_name)  
77 - sim["outfile"].filename(events_name)  
78 - sim["caldb"].string(caldb)  
79 - sim["irf"].string(irf)  
80 - sim["ra"].real(ra)  
81 - sim["dec"].real(dec)  
82 - sim["rad"].real(rad_sim)  
83 - sim["tmin"].real(tstart)  
84 - sim["tmax"].real(tstop)  
85 - sim["emin"].real(emin)  
86 - sim["emax"].real(emax)  
87 - sim.execute()  
88 - sys.stdout.write("Simulated events ("+str(sim.celapse())+" CPU seconds)\n")  
89 -  
90 - # Update timing  
91 - wall_seconds += sim.telapse()  
92 - cpu_seconds += sim.celapse()  
93 -  
94 - # Bin events into counts map  
95 - bin = ctbin()  
96 - bin.logFileOpen() # We need this to explicitely open the log file in Python mode  
97 - bin["evfile"].filename(events_name)  
98 - bin["outfile"].filename(cntmap_name)  
99 - bin["ebinalg"].string("LOG")  
100 - bin["emin"].real(emin)  
101 - bin["emax"].real(emax)  
102 - bin["enumbins"].integer(enumbins)  
103 - bin["nxpix"].integer(nxpix)  
104 - bin["nypix"].integer(nypix)  
105 - bin["binsz"].real(binsz)  
106 - bin["coordsys"].string(coordsys)  
107 - bin["xref"].real(ra)  
108 - bin["yref"].real(dec)  
109 - bin["proj"].string(proj)  
110 - bin.execute()  
111 - sys.stdout.write("Binned events into counts map ("+str(bin.celapse())+" CPU seconds)\n")  
112 -  
113 - # Update timing  
114 - wall_seconds += bin.telapse()  
115 - cpu_seconds += bin.celapse()  
116 -  
117 - # Perform maximum likelihood fitting  
118 - like = ctlike()  
119 - like.logFileOpen() # We need this to explicitely open the log file in Python mode  
120 - like["infile"].filename(cntmap_name)  
121 - like["srcmdl"].filename(model_name)  
122 - like["outmdl"].filename(result_name)  
123 - like["caldb"].string(caldb)  
124 - like["irf"].string(irf)  
125 - like.execute()  
126 - sys.stdout.write("Maximum likelihood fitting ("+str(like.celapse())+" CPU seconds)\n")  
127 -  
128 - # Update timing  
129 - wall_seconds += like.telapse()  
130 - cpu_seconds += like.celapse()  
131 -  
132 - # Show total times  
133 - sys.stdout.write("Total wall time elapsed: "+str(wall_seconds)+" seconds\n")  
134 - sys.stdout.write("Total CPU time used ...: "+str(cpu_seconds)+" seconds\n")  
135 -  
136 - # Return  
137 - return 47 + """
  48 + # Set script parameters
  49 + model_name = "${CTOOLS}/share/models/crab.xml"
  50 + events_name = "events.fits"
  51 + cntmap_name = "cntmap.fits"
  52 + result_name = "results.xml"
  53 + caldb = "${CALDB}/data/cta/dummy/bcf"
  54 + irf = "cta_dummy_irf"
  55 + ra = 83.63
  56 + dec = 22.01
  57 + rad_sim = 10.0
  58 + tstart = 0.0
  59 + tstop = 1800.0
  60 + emin = 0.1
  61 + emax = 100.0
  62 + enumbins = 20
  63 + nxpix = 200
  64 + nypix = 200
  65 + binsz = 0.02
  66 + coordsys = "CEL"
  67 + proj = "CAR"
  68 +
  69 + # Initialise timing
  70 + wall_seconds = 0.0
  71 + cpu_seconds = 0.0
  72 +
  73 + # Simulate events
  74 + sim = ctools.ctobssim()
  75 + sim.logFileOpen() # We need this to explicitely open the log file in Python mode
  76 + sim["infile"].filename(model_name)
  77 + sim["outfile"].filename(events_name)
  78 + sim["caldb"].string(caldb)
  79 + sim["irf"].string(irf)
  80 + sim["ra"].real(ra)
  81 + sim["dec"].real(dec)
  82 + sim["rad"].real(rad_sim)
  83 + sim["tmin"].real(tstart)
  84 + sim["tmax"].real(tstop)
  85 + sim["emin"].real(emin)
  86 + sim["emax"].real(emax)
  87 + sim.execute()
  88 + sys.stdout.write("Simulated events ("+str(sim.celapse())+" CPU seconds)\n")
  89 +
  90 + # Update timing
  91 + wall_seconds += sim.telapse()
  92 + cpu_seconds += sim.celapse()
  93 +
  94 + # Bin events into counts map
  95 + bin = ctools.ctbin()
  96 + bin.logFileOpen() # We need this to explicitely open the log file in Python mode
  97 + bin["evfile"].filename(events_name)
  98 + bin["outfile"].filename(cntmap_name)
  99 + bin["ebinalg"].string("LOG")
  100 + bin["emin"].real(emin)
  101 + bin["emax"].real(emax)
  102 + bin["enumbins"].integer(enumbins)
  103 + bin["nxpix"].integer(nxpix)
  104 + bin["nypix"].integer(nypix)
  105 + bin["binsz"].real(binsz)
  106 + bin["coordsys"].string(coordsys)
  107 + bin["xref"].real(ra)
  108 + bin["yref"].real(dec)
  109 + bin["proj"].string(proj)
  110 + bin.execute()
  111 + sys.stdout.write("Binned events into counts map ("+str(bin.celapse())+" CPU seconds)\n")
  112 +
  113 + # Update timing
  114 + wall_seconds += bin.telapse()
  115 + cpu_seconds += bin.celapse()
  116 +
  117 + # Perform maximum likelihood fitting
  118 + like = ctools.ctlike()
  119 + like.logFileOpen() # We need this to explicitely open the log file in Python mode
  120 + like["infile"].filename(cntmap_name)
  121 + like["srcmdl"].filename(model_name)
  122 + like["outmdl"].filename(result_name)
  123 + like["caldb"].string(caldb)
  124 + like["irf"].string(irf)
  125 + like.execute()
  126 + sys.stdout.write("Maximum likelihood fitting ("+str(like.celapse())+" CPU seconds)\n")
  127 +
  128 + # Update timing
  129 + wall_seconds += like.telapse()
  130 + cpu_seconds += like.celapse()
  131 +
  132 + # Show total times
  133 + sys.stdout.write("Total wall time elapsed: "+str(wall_seconds)+" seconds\n")
  134 + sys.stdout.write("Total CPU time used ...: "+str(cpu_seconds)+" seconds\n")
  135 +
  136 + # Return
  137 + return
138 138
139 139
140 # ============================= # 140 # ============================= #
141 # Analysis pipeline - version 2 # 141 # Analysis pipeline - version 2 #
142 # ============================= # 142 # ============================= #
143 def pipeline_v2(): 143 def pipeline_v2():
144 - """  
145 - Binned analysis pipeline - keep intermediate results in memory. 144 + """
  145 + Binned analysis pipeline - keep intermediate results in memory.
146 146
147 This function implements an analysis pipeline that successively calls 147 This function implements an analysis pipeline that successively calls
148 ctobssim, ctbin and ctlike without saving the intermediate results as 148 ctobssim, ctbin and ctlike without saving the intermediate results as
149 - FITS files on disk. All data are hold in memory.  
150 -  
151 - At the end, results are plotted (if matplotlib is installed)  
152 - """  
153 - # Set script parameters  
154 - model_name = "${CTOOLS}/share/models/crab.xml"  
155 - caldb = "${CALDB}/data/cta/dummy/bcf"  
156 - irf = "cta_dummy_irf"  
157 - ra = 83.63  
158 - dec = 22.01  
159 - rad_sim = 10.0  
160 - tstart = 0.0  
161 - tstop = 1800.0  
162 - emin = 0.1  
163 - emax = 100.0  
164 - enumbins = 20  
165 - nxpix = 200  
166 - nypix = 200  
167 - binsz = 0.02  
168 - coordsys = "CEL"  
169 - proj = "CAR"  
170 -  
171 - # Initialise timing  
172 - wall_seconds = 0.0  
173 - cpu_seconds = 0.0  
174 -  
175 - # Simulate events  
176 - sim = ctobssim()  
177 - sim["infile"].filename(model_name)  
178 - sim["caldb"].string(caldb)  
179 - sim["irf"].string(irf)  
180 - sim["ra"].real(ra)  
181 - sim["dec"].real(dec)  
182 - sim["rad"].real(rad_sim)  
183 - sim["tmin"].real(tstart)  
184 - sim["tmax"].real(tstop)  
185 - sim["emin"].real(emin)  
186 - sim["emax"].real(emax)  
187 - sim.run()  
188 - sys.stdout.write("Simulated events ("+str(sim.celapse())+" CPU seconds)\n")  
189 -  
190 - # Update timing  
191 - wall_seconds += sim.telapse()  
192 - cpu_seconds += sim.celapse()  
193 -  
194 - # Bin events into counts map  
195 - bin = ctbin(sim.obs())  
196 - bin["ebinalg"].string("LOG")  
197 - bin["emin"].real(emin)  
198 - bin["emax"].real(emax)  
199 - bin["enumbins"].integer(enumbins)  
200 - bin["nxpix"].integer(nxpix)  
201 - bin["nypix"].integer(nypix)  
202 - bin["binsz"].real(binsz)  
203 - bin["coordsys"].string(coordsys)  
204 - bin["xref"].real(ra)  
205 - bin["yref"].real(dec)  
206 - bin["proj"].string(proj)  
207 - bin.run()  
208 - sys.stdout.write("Binned events into counts map ("+str(bin.celapse())+" CPU seconds)\n")  
209 -  
210 - # Update timing  
211 - wall_seconds += bin.telapse()  
212 - cpu_seconds += bin.celapse()  
213 -  
214 - # Perform maximum likelihood fitting  
215 - like = ctlike(bin.obs())  
216 - like.run()  
217 - sys.stdout.write("Maximum likelihood fitting ("+str(like.celapse())+" CPU seconds)\n")  
218 -  
219 - # Update timing  
220 - wall_seconds += like.telapse()  
221 - cpu_seconds += like.celapse()  
222 -  
223 - # Show total times  
224 - sys.stdout.write("Total wall time elapsed: "+str(wall_seconds)+" seconds\n")  
225 - sys.stdout.write("Total CPU time used ...: "+str(cpu_seconds)+" seconds\n")  
226 -  
227 - # Show model fitting results  
228 - #sys.stdout.write(like.obs().models()+"\n")  
229 -  
230 - # Plot counts  
231 - if has_matplotlib:  
232 - plot_counts(bin.obs())  
233 - else:  
234 - sys.stdout.write("Matplotlib is not (correctly) installed on your system. No counts spectra are shown.\n")  
235 -  
236 - # Return  
237 - return 149 + FITS files on disk. All data are hold in memory.
  150 +
  151 + At the end, results are plotted (if matplotlib is installed)
  152 + """
  153 + # Set script parameters
  154 + model_name = "${CTOOLS}/share/models/crab.xml"
  155 + caldb = "${CALDB}/data/cta/dummy/bcf"
  156 + irf = "cta_dummy_irf"
  157 + ra = 83.63
  158 + dec = 22.01
  159 + rad_sim = 10.0
  160 + tstart = 0.0
  161 + tstop = 1800.0
  162 + emin = 0.1
  163 + emax = 100.0
  164 + enumbins = 20
  165 + nxpix = 200
  166 + nypix = 200
  167 + binsz = 0.02
  168 + coordsys = "CEL"
  169 + proj = "CAR"
  170 +
  171 + # Initialise timing
  172 + wall_seconds = 0.0
  173 + cpu_seconds = 0.0
  174 +
  175 + # Simulate events
  176 + sim = ctools.ctobssim()
  177 + sim["infile"].filename(model_name)
  178 + sim["caldb"].string(caldb)
  179 + sim["irf"].string(irf)
  180 + sim["ra"].real(ra)
  181 + sim["dec"].real(dec)
  182 + sim["rad"].real(rad_sim)
  183 + sim["tmin"].real(tstart)
  184 + sim["tmax"].real(tstop)
  185 + sim["emin"].real(emin)
  186 + sim["emax"].real(emax)
  187 + sim.run()
  188 + sys.stdout.write("Simulated events ("+str(sim.celapse())+" CPU seconds)\n")
  189 +
  190 + # Update timing
  191 + wall_seconds += sim.telapse()
  192 + cpu_seconds += sim.celapse()
  193 +
  194 + # Bin events into counts map
  195 + bin = ctools.ctbin(sim.obs())
  196 + bin["ebinalg"].string("LOG")
  197 + bin["emin"].real(emin)
  198 + bin["emax"].real(emax)
  199 + bin["enumbins"].integer(enumbins)
  200 + bin["nxpix"].integer(nxpix)
  201 + bin["nypix"].integer(nypix)
  202 + bin["binsz"].real(binsz)
  203 + bin["coordsys"].string(coordsys)
  204 + bin["xref"].real(ra)
  205 + bin["yref"].real(dec)
  206 + bin["proj"].string(proj)
  207 + bin.run()
  208 + sys.stdout.write("Binned events into counts map ("+str(bin.celapse())+" CPU seconds)\n")
  209 +
  210 + # Update timing
  211 + wall_seconds += bin.telapse()
  212 + cpu_seconds += bin.celapse()
  213 +
  214 + # Perform maximum likelihood fitting
  215 + like = ctools.ctlike(bin.obs())
  216 + like.run()
  217 + sys.stdout.write("Maximum likelihood fitting ("+str(like.celapse())+" CPU seconds)\n")
  218 +
  219 + # Update timing
  220 + wall_seconds += like.telapse()
  221 + cpu_seconds += like.celapse()
  222 +
  223 + # Show total times
  224 + sys.stdout.write("Total wall time elapsed: "+str(wall_seconds)+" seconds\n")
  225 + sys.stdout.write("Total CPU time used ...: "+str(cpu_seconds)+" seconds\n")
  226 +
  227 + # Show model fitting results
  228 + #sys.stdout.write(like.obs().models()+"\n")
  229 +
  230 + # Plot counts
  231 + if has_matplotlib:
  232 + plot_counts(bin.obs())
  233 + else:
  234 + sys.stdout.write("Matplotlib is not (correctly) installed on your system. No counts spectra are shown.\n")
  235 +
  236 + # Return
  237 + return
238 238
239 239
240 # =========== # 240 # =========== #
241 # Plot counts # 241 # Plot counts #
242 # =========== # 242 # =========== #
243 def plot_counts(observations): 243 def plot_counts(observations):
244 - """  
245 - Plot counts.  
246 - """  
247 - # Set legend fontsize  
248 - params = {'legend.fontsize': 10}  
249 - plt.rcParams.update(params)  
250 -  
251 - # Set plot styles  
252 - styles = ['b-', 'g-', 'y-', 'n-']  
253 -  
254 - # Dump header  
255 - sys.stdout.write("\n")  
256 - sys.stdout.write("Make plots (using matplotlib):\n")  
257 - sys.stdout.write("==============================\n")  
258 -  
259 - # Create figure 1  
260 - plt.figure(1,figsize=(12,6))  
261 - plt.subplots_adjust(hspace=.7)  
262 -  
263 - # Create subplot 1  
264 - plt.subplot(121)  
265 - plt.title("Spectrum (summed over all pixels)")  
266 -  
267 - # Loop over observations  
268 - for obs in observations:  
269 -  
270 - # Get event cube  
271 - cube = obs.events()  
272 -  
273 - # Create energy axis  
274 - energy = []  
275 - ebounds = cube.ebounds()  
276 - for i in range(ebounds.size()):  
277 - energy.append(ebounds.elogmean(i).TeV())  
278 -  
279 - # Create spectrum  
280 - sys.stdout.write("Extract data:\n")  
281 - counts = [0.0 for i in range(ebounds.size())]  
282 - for bin in cube:  
283 - index = ebounds.index(bin.energy())  
284 - counts[index] = counts[index] + bin.counts()  
285 -  
286 - # Create error bars  
287 - error = [sqrt(c) for c in counts]  
288 -  
289 - # Plot spectrum  
290 - plt.loglog(energy, counts, 'ro', label='data')  
291 - plt.errorbar(energy, counts, error, fmt=None, ecolor='r')  
292 -  
293 - # Extract models  
294 - sys.stdout.write("Extract models:\n")  
295 - sum_model = [0.0 for i in range(ebounds.size())]  
296 - for k, m in enumerate(observations.models()):  
297 - sys.stdout.write("- "+m.name()+"\n")  
298 - model = [0.0 for i in range(ebounds.size())]  
299 - for bin in cube:  
300 - index = ebounds.index(bin.energy())  
301 - prob = m.eval(bin, obs)  
302 - model[index] = model[index] + prob * bin.size()  
303 - for i in range(ebounds.size()):  
304 - sum_model[i] = sum_model[i] + model[i]  
305 - plt.loglog(energy, model, styles[k], label=m.name())  
306 - plt.loglog(energy, sum_model, 'r-', label='total')  
307 -  
308 - # Put labels  
309 - plt.xlabel("Energy (TeV)")  
310 - plt.ylabel("Counts")  
311 - plt.legend(loc="lower left")  
312 -  
313 - # Create subplot 2  
314 - plt.subplot(122)  
315 - plt.title("Offset (summed over all energies)")  
316 -  
317 - # Set Crab direction  
318 - crab = GSkyDir()  
319 - crab.radec_deg(83.63, 22.01)  
320 -  
321 - # Loop over observations  
322 - for obs in observations:  
323 -  
324 - # Get event cube  
325 - cube = obs.events()  
326 -  
327 - # Create offset histogram  
328 - sys.stdout.write("Extract data:\n")  
329 - nx = 30  
330 - doffset2 = 0.01  
331 - offset2 = [(i+0.5)*doffset2 for i in range(nx)]  
332 - counts = [0.0 for i in range(nx)]  
333 - for bin in cube:  
334 - off = GCTAInstDir(bin.dir()).dir().dist_deg(crab)  
335 - off2 = off*off  
336 - index = int(off2/doffset2)  
337 - if index < nx:  
338 - counts[index] = counts[index] + bin.counts()  
339 -  
340 - # Create error bars  
341 - error = [sqrt(c) for c in counts]  
342 -  
343 - # Plot distribution  
344 - plt.semilogy(offset2, counts, 'ro', label='data')  
345 -  
346 - # Extract models  
347 - sys.stdout.write("Extract models:\n")  
348 - sum_model = [0.0 for i in range(nx)]  
349 - for k, m in enumerate(observations.models()):  
350 - sys.stdout.write("- "+m.name()+"\n")  
351 - model = [0.0 for i in range(nx)]  
352 - for bin in cube:  
353 - off = GCTAInstDir(bin.dir()).dir().dist_deg(crab)  
354 - off2 = off*off  
355 - index = int(off2/doffset2)  
356 - if index < nx:  
357 - prob = m.eval(bin, obs)  
358 - model[index] = model[index] + prob * bin.size()  
359 - for i in range(nx):  
360 - sum_model[i] = sum_model[i] + model[i]  
361 - plt.plot(offset2, model, styles[k], label=m.name())  
362 - plt.plot(offset2, sum_model, 'r-', label='total')  
363 - plt.ylim(ymin=0.1)  
364 -  
365 - # Put labels  
366 - plt.xlabel("Offset (deg^2)")  
367 - plt.ylabel("Counts")  
368 - plt.legend(loc="upper right")  
369 -  
370 - # Show counts spectra  
371 - plt.show()  
372 -  
373 - # Return  
374 - return 244 + """
  245 + Plot counts.
  246 + """
  247 + # Set legend fontsize
  248 + params = {'legend.fontsize': 10}
  249 + plt.rcParams.update(params)
  250 +
  251 + # Set plot styles
  252 + styles = ['b-', 'g-', 'y-', 'n-']
  253 +
  254 + # Dump header
  255 + sys.stdout.write("\n")
  256 + sys.stdout.write("Make plots (using matplotlib):\n")
  257 + sys.stdout.write("==============================\n")
  258 +
  259 + # Create figure 1
  260 + plt.figure(1,figsize=(12,6))
  261 + plt.subplots_adjust(hspace=.7)
  262 +
  263 + # Create subplot 1
  264 + plt.subplot(121)
  265 + plt.title("Spectrum (summed over all pixels)")
  266 +
  267 + # Loop over observations
  268 + for obs in observations:
  269 +
  270 + # Get event cube
  271 + cube = obs.events()
  272 +
  273 + # Create energy axis
  274 + energy = []
  275 + ebounds = cube.ebounds()
  276 + for i in range(ebounds.size()):
  277 + energy.append(ebounds.elogmean(i).TeV())
  278 +
  279 + # Create spectrum
  280 + sys.stdout.write("Extract data:\n")
  281 + counts = [0.0 for i in range(ebounds.size())]
  282 + for bin in cube:
  283 + index = ebounds.index(bin.energy())
  284 + counts[index] = counts[index] + bin.counts()
  285 +
  286 + # Create error bars
  287 + error = [math.sqrt(c) for c in counts]
  288 +
  289 + # Plot spectrum
  290 + plt.loglog(energy, counts, 'ro', label='data')
  291 + plt.errorbar(energy, counts, error, fmt=None, ecolor='r')
  292 +
  293 + # Extract models
  294 + sys.stdout.write("Extract models:\n")
  295 + sum_model = [0.0 for i in range(ebounds.size())]
  296 + for k, m in enumerate(observations.models()):
  297 + sys.stdout.write("- "+m.name()+"\n")
  298 + model = [0.0 for i in range(ebounds.size())]
  299 + for bin in cube:
  300 + index = ebounds.index(bin.energy())
  301 + prob = m.eval(bin, obs)
  302 + model[index] = model[index] + prob * bin.size()
  303 + for i in range(ebounds.size()):
  304 + sum_model[i] = sum_model[i] + model[i]
  305 + plt.loglog(energy, model, styles[k], label=m.name())
  306 + plt.loglog(energy, sum_model, 'r-', label='total')
  307 +
  308 + # Put labels
  309 + plt.xlabel("Energy (TeV)")
  310 + plt.ylabel("Counts")
  311 + plt.legend(loc="lower left")
  312 +
  313 + # Create subplot 2
  314 + plt.subplot(122)
  315 + plt.title("Offset (summed over all energies)")
  316 +
  317 + # Set Crab direction
  318 + crab = gammalib.GSkyDir()
  319 + crab.radec_deg(83.63, 22.01)
  320 +
  321 + # Loop over observations
  322 + for obs in observations:
  323 +
  324 + # Get event cube
  325 + cube = obs.events()
  326 +
  327 + # Create offset histogram
  328 + sys.stdout.write("Extract data:\n")
  329 + nx = 30
  330 + doffset2 = 0.01
  331 + offset2 = [(i+0.5)*doffset2 for i in range(nx)]
  332 + counts = [0.0 for i in range(nx)]
  333 + for bin in cube:
  334 + off = gammalib.GCTAInstDir(bin.dir()).dir().dist_deg(crab)
  335 + off2 = off*off
  336 + index = int(off2/doffset2)
  337 + if index < nx:
  338 + counts[index] = counts[index] + bin.counts()
  339 +
  340 + # Create error bars
  341 + error = [math.sqrt(c) for c in counts]
  342 +
  343 + # Plot distribution
  344 + plt.semilogy(offset2, counts, 'ro', label='data')
  345 +
  346 + # Extract models
  347 + sys.stdout.write("Extract models:\n")
  348 + sum_model = [0.0 for i in range(nx)]
  349 + for k, m in enumerate(observations.models()):
  350 + sys.stdout.write("- "+m.name()+"\n")
  351 + model = [0.0 for i in range(nx)]
  352 + for bin in cube:
  353 + off = gammalib.GCTAInstDir(bin.dir()).dir().dist_deg(crab)
  354 + off2 = off*off
  355 + index = int(off2/doffset2)
  356 + if index < nx:
  357 + prob = m.eval(bin, obs)
  358 + model[index] = model[index] + prob * bin.size()
  359 + for i in range(nx):
  360 + sum_model[i] = sum_model[i] + model[i]
  361 + plt.plot(offset2, model, styles[k], label=m.name())
  362 + plt.plot(offset2, sum_model, 'r-', label='total')
  363 + plt.ylim(ymin=0.1)
  364 +
  365 + # Put labels
  366 + plt.xlabel("Offset (deg^2)")
  367 + plt.ylabel("Counts")
  368 + plt.legend(loc="upper right")
  369 +
  370 + # Show counts spectra
  371 + plt.show()
  372 +
  373 + # Return
  374 + return
375 375
376 376
377 #==========================# 377 #==========================#
378 # Main routine entry point # 378 # Main routine entry point #
379 #==========================# 379 #==========================#
380 if __name__ == '__main__': 380 if __name__ == '__main__':
381 - """  
382 - CTA binned analysis script.  
383 -  
384 - This script implements the analysis steps to perform a binned analysis.  
385 - The steps include:  
386 - - Event simulation  
387 - - Event binning  
388 - - Binned maximum likelihood fitting  
389 -  
390 - Two variations of the script are implemented. One that calls the classes  
391 - like executables, storing the intermediate results on disk, and a second  
392 - that uses the results of the precedent class directly without storing  
393 - any data on disk.  
394 -  
395 - The second variation will also show a counts spectrum of the data after  
396 - fitting (only if matplotlib is available on your system). Note that the  
397 - counts spectrum is summed over the entire data space, i.e. no event selection  
398 - around the Crab has been performed.  
399 - """  
400 - # Dump header  
401 - sys.stdout.write("*************************************\n")  
402 - sys.stdout.write("* CTA binned analysis scripts *\n")  
403 - sys.stdout.write("*************************************\n")  
404 - sys.stdout.write("... this script will take 1-2 minutes\n")  
405 -  
406 - # Remove any existing result files  
407 - list = [glob.glob("*.fits"), glob.glob("*.log"), glob.glob("*.xml")]  
408 - for files in list:  
409 - for file in files:  
410 - os.remove(file)  
411 -  
412 - # Save intermediate results on disk  
413 - sys.stdout.write("\n")  
414 - sys.stdout.write("Executable analysis pipeline:\n")  
415 - sys.stdout.write("=============================\n")  
416 - pipeline_v1()  
417 -  
418 - # Do analysis in memory  
419 - sys.stdout.write("\n")  
420 - sys.stdout.write("In memory analysis pipeline:\n")  
421 - sys.stdout.write("============================\n")  
422 - pipeline_v2()  
423 - sys.stdout.write("\n") 381 + """
  382 + CTA binned analysis script.
  383 +
  384 + This script implements the analysis steps to perform a binned analysis.
  385 + The steps include:
  386 + - Event simulation
  387 + - Event binning
  388 + - Binned maximum likelihood fitting
  389 +
  390 + Two variations of the script are implemented. One that calls the classes
  391 + like executables, storing the intermediate results on disk, and a second
  392 + that uses the results of the precedent class directly without storing
  393 + any data on disk.
  394 +
  395 + The second variation will also show a counts spectrum of the data after
  396 + fitting (only if matplotlib is available on your system). Note that the
  397 + counts spectrum is summed over the entire data space, i.e. no event selection
  398 + around the Crab has been performed.
  399 + """
  400 + # Dump header
  401 + sys.stdout.write("*************************************\n")
  402 + sys.stdout.write("* CTA binned analysis scripts *\n")
  403 + sys.stdout.write("*************************************\n")
  404 + sys.stdout.write("... this script will take 1-2 minutes\n")
  405 +
  406 + # Remove any existing result files
  407 + list = [glob.glob("*.fits"), glob.glob("*.log"), glob.glob("*.xml")]
  408 + for files in list:
  409 + for file in files:
  410 + os.remove(file)
  411 +
  412 + # Save intermediate results on disk
  413 + sys.stdout.write("\n")
  414 + sys.stdout.write("Executable analysis pipeline:\n")
  415 + sys.stdout.write("=============================\n")
  416 + pipeline_v1()
  417 +
  418 + # Do analysis in memory
  419 + sys.stdout.write("\n")
  420 + sys.stdout.write("In memory analysis pipeline:\n")
  421 + sys.stdout.write("============================\n")
  422 + pipeline_v2()
  423 + sys.stdout.write("\n")
examples/make_unbinned_analysis.py
@@ -7,7 +7,7 @@ @@ -7,7 +7,7 @@
7 # Required 3rd party modules: 7 # Required 3rd party modules:
8 # None 8 # None
9 # 9 #
10 -# Copyright (C) 2011 Jurgen Knodlseder 10 +# Copyright (C) 2011-2014 Jurgen Knodlseder
11 # 11 #
12 # This program is free software: you can redistribute it and/or modify 12 # This program is free software: you can redistribute it and/or modify
13 # it under the terms of the GNU General Public License as published by 13 # it under the terms of the GNU General Public License as published by
@@ -23,399 +23,402 @@ @@ -23,399 +23,402 @@
23 # along with this program. If not, see <http://www.gnu.org/licenses/>. 23 # along with this program. If not, see <http://www.gnu.org/licenses/>.
24 # 24 #
25 # ========================================================================== 25 # ==========================================================================
26 -from ctools import *  
27 -from gammalib import *  
28 -from math import * 26 +#from ctools import *
  27 +#from gammalib import *
  28 +#from math import *
  29 +import gammalib
  30 +import ctools
  31 +import math
29 import os 32 import os
30 import glob 33 import glob
31 import sys 34 import sys
32 try: 35 try:
33 - import matplotlib.pyplot as plt  
34 - has_matplotlib = True 36 + import matplotlib.pyplot as plt
  37 + has_matplotlib = True
35 except: 38 except:
36 - has_matplotlib = False 39 + has_matplotlib = False
37 40
38 41
39 # ============================= # 42 # ============================= #
40 # Analysis pipeline - version 1 # 43 # Analysis pipeline - version 1 #
41 # ============================= # 44 # ============================= #
42 def pipeline_v1(): 45 def pipeline_v1():
43 - """  
44 - Unbinned analysis pipeline - save intermediate results. 46 + """
  47 + Unbinned analysis pipeline - save intermediate results.
45 48
46 This function implements an analysis pipeline that successively calls 49 This function implements an analysis pipeline that successively calls
47 ctobssim, ctselect and ctlike by saving the intermediate results as FITS 50 ctobssim, ctselect and ctlike by saving the intermediate results as FITS
48 files on disk. This replicates the way how the analysis would be done 51 files on disk. This replicates the way how the analysis would be done
49 in a ftools-like approach. 52 in a ftools-like approach.
50 - """  
51 - # Set script parameters  
52 - model_name = "${CTOOLS}/share/models/crab.xml"  
53 - events_name = "events.fits"  
54 - selected_events_name = "selected_events.fits"  
55 - result_name = "results.xml"  
56 - caldb = "${CALDB}/data/cta/dummy/bcf"  
57 - irf = "cta_dummy_irf"  
58 - ra = 83.63  
59 - dec = 22.01  
60 - rad_sim = 10.0  
61 - tstart = 0.0  
62 - tstop = 1800.0  
63 - emin = 0.1  
64 - emax = 100.0  
65 - rad_select = 3.0  
66 -  
67 - # Initialise timing  
68 - wall_seconds = 0.0  
69 - cpu_seconds = 0.0  
70 -  
71 - # Simulate events  
72 - sim = ctobssim()  
73 - sim.logFileOpen()  
74 - sim["infile"].filename(model_name)  
75 - sim["outfile"].filename(events_name)  
76 - sim["caldb"].string(caldb)  
77 - sim["irf"].string(irf)  
78 - sim["ra"].real(ra)  
79 - sim["dec"].real(dec)  
80 - sim["rad"].real(rad_sim)  
81 - sim["tmin"].real(tstart)  
82 - sim["tmax"].real(tstop)  
83 - sim["emin"].real(emin)  
84 - sim["emax"].real(emax)  
85 - sim.execute()  
86 - print("Simulated events ("+str(sim.celapse())+" CPU seconds)")  
87 -  
88 - # Update timing  
89 - wall_seconds += sim.telapse()  
90 - cpu_seconds += sim.celapse()  
91 -  
92 - # Select events  
93 - select = ctselect()  
94 - select.logFileOpen()  
95 - select["infile"].filename(events_name)  
96 - select["outfile"].filename(selected_events_name)  
97 - select["ra"].real(ra)  
98 - select["dec"].real(dec)  
99 - select["rad"].real(rad_select)  
100 - select["tmin"].real(tstart)  
101 - select["tmax"].real(tstop)  
102 - select["emin"].real(emin)  
103 - select["emax"].real(emax)  
104 - select.execute()  
105 - print("Select event ("+str(select.celapse())+" CPU seconds)")  
106 -  
107 - # Update timing  
108 - wall_seconds += select.telapse()  
109 - cpu_seconds += select.celapse()  
110 -  
111 - # Perform maximum likelihood fitting  
112 - like = ctlike()  
113 - like.logFileOpen()  
114 - like["infile"].filename(selected_events_name)  
115 - like["srcmdl"].filename(model_name)  
116 - like["outmdl"].filename(result_name)  
117 - like["caldb"].string(caldb)  
118 - like["irf"].string(irf)  
119 - like.execute()  
120 - print("Maximum likelihood fitting ("+str(like.celapse())+" CPU seconds)")  
121 -  
122 - # Update timing  
123 - wall_seconds += like.telapse()  
124 - cpu_seconds += like.celapse()  
125 -  
126 - # Show total times  
127 - print("Total wall time elapsed: "+str(wall_seconds)+" seconds")  
128 - print("Total CPU time used ...: "+str(cpu_seconds)+" seconds")  
129 -  
130 - # Return  
131 - return 53 + """
  54 + # Set script parameters
  55 + model_name = "${CTOOLS}/share/models/crab.xml"
  56 + events_name = "events.fits"
  57 + selected_events_name = "selected_events.fits"
  58 + result_name = "results.xml"
  59 + caldb = "${CALDB}/data/cta/dummy/bcf"
  60 + irf = "cta_dummy_irf"
  61 + ra = 83.63
  62 + dec = 22.01
  63 + rad_sim = 10.0
  64 + tstart = 0.0
  65 + tstop = 1800.0
  66 + emin = 0.1
  67 + emax = 100.0
  68 + rad_select = 3.0
  69 +
  70 + # Initialise timing
  71 + wall_seconds = 0.0
  72 + cpu_seconds = 0.0
  73 +
  74 + # Simulate events
  75 + sim = ctools.ctobssim()
  76 + sim.logFileOpen()
  77 + sim["infile"].filename(model_name)
  78 + sim["outfile"].filename(events_name)
  79 + sim["caldb"].string(caldb)
  80 + sim["irf"].string(irf)
  81 + sim["ra"].real(ra)
  82 + sim["dec"].real(dec)
  83 + sim["rad"].real(rad_sim)
  84 + sim["tmin"].real(tstart)
  85 + sim["tmax"].real(tstop)
  86 + sim["emin"].real(emin)
  87 + sim["emax"].real(emax)
  88 + sim.execute()
  89 + print("Simulated events ("+str(sim.celapse())+" CPU seconds)")
  90 +
  91 + # Update timing
  92 + wall_seconds += sim.telapse()
  93 + cpu_seconds += sim.celapse()
  94 +
  95 + # Select events
  96 + select = ctools.ctselect()
  97 + select.logFileOpen()
  98 + select["infile"].filename(events_name)
  99 + select["outfile"].filename(selected_events_name)
  100 + select["ra"].real(ra)
  101 + select["dec"].real(dec)
  102 + select["rad"].real(rad_select)
  103 + select["tmin"].real(tstart)
  104 + select["tmax"].real(tstop)
  105 + select["emin"].real(emin)
  106 + select["emax"].real(emax)
  107 + select.execute()
  108 + print("Select event ("+str(select.celapse())+" CPU seconds)")
  109 +
  110 + # Update timing
  111 + wall_seconds += select.telapse()
  112 + cpu_seconds += select.celapse()
  113 +
  114 + # Perform maximum likelihood fitting
  115 + like = ctools.ctlike()
  116 + like.logFileOpen()
  117 + like["infile"].filename(selected_events_name)
  118 + like["srcmdl"].filename(model_name)
  119 + like["outmdl"].filename(result_name)
  120 + like["caldb"].string(caldb)
  121 + like["irf"].string(irf)
  122 + like.execute()
  123 + print("Maximum likelihood fitting ("+str(like.celapse())+" CPU seconds)")
  124 +
  125 + # Update timing
  126 + wall_seconds += like.telapse()
  127 + cpu_seconds += like.celapse()
  128 +
  129 + # Show total times
  130 + print("Total wall time elapsed: "+str(wall_seconds)+" seconds")
  131 + print("Total CPU time used ...: "+str(cpu_seconds)+" seconds")
  132 +
  133 + # Return
  134 + return
132 135
133 136
134 # ============================= # 137 # ============================= #
135 # Analysis pipeline - version 2 # 138 # Analysis pipeline - version 2 #
136 # ============================= # 139 # ============================= #
137 def pipeline_v2(show_data): 140 def pipeline_v2(show_data):
138 - """  
139 - Unbinned analysis pipeline - keep intermediate results in memory. 141 + """
  142 + Unbinned analysis pipeline - keep intermediate results in memory.
140 143
141 This function implements an analysis pipeline that successively calls 144 This function implements an analysis pipeline that successively calls
142 ctobssim, ctselect and ctlike without saving the intermediate results as 145 ctobssim, ctselect and ctlike without saving the intermediate results as
143 - FITS files on disk. All data is only hold in memory.  
144 - """  
145 - # Set script parameters  
146 - model_name = "${CTOOLS}/share/models/crab.xml"  
147 - caldb = "${CALDB}/data/cta/dummy/bcf"  
148 - irf = "cta_dummy_irf"  
149 - ra = 83.63  
150 - dec = 22.01  
151 - rad_sim = 10.0  
152 - tstart = 0.0  
153 - tstop = 1800.0  
154 - emin = 0.1  
155 - emax = 100.0  
156 - rad_select = 3.0  
157 -  
158 - # Initialise timing  
159 - wall_seconds = 0.0  
160 - cpu_seconds = 0.0  
161 -  
162 - # Simulate events  
163 - sim = ctobssim()  
164 - sim["infile"].filename(model_name)  
165 - sim["caldb"].string(caldb)  
166 - sim["irf"].string(irf)  
167 - sim["ra"].real(ra)  
168 - sim["dec"].real(dec)  
169 - sim["rad"].real(rad_sim)  
170 - sim["tmin"].real(tstart)  
171 - sim["tmax"].real(tstop)  
172 - sim["emin"].real(emin)  
173 - sim["emax"].real(emax)  
174 - sim.run()  
175 - print("Simulated events ("+str(sim.celapse())+" CPU seconds)")  
176 -  
177 - # Update timing  
178 - wall_seconds += sim.telapse()  
179 - cpu_seconds += sim.celapse()  
180 -  
181 - # Select events  
182 - select = ctselect(sim.obs())  
183 - select["ra"].real(ra)  
184 - select["dec"].real(dec)  
185 - select["rad"].real(rad_select)  
186 - select["tmin"].real(tstart)  
187 - select["tmax"].real(tstop)  
188 - select["emin"].real(emin)  
189 - select["emax"].real(emax)  
190 - select.run()  
191 - print("Select event ("+str(select.celapse())+" CPU seconds)")  
192 -  
193 - # Update timing  
194 - wall_seconds += select.telapse()  
195 - cpu_seconds += select.celapse()  
196 -  
197 - # Perform maximum likelihood fitting  
198 - like = ctlike(select.obs())  
199 - like.run()  
200 - print("Maximum likelihood fitting ("+str(like.celapse())+" CPU seconds)")  
201 -  
202 - # Update timing  
203 - wall_seconds += like.telapse()  
204 - cpu_seconds += like.celapse()  
205 -  
206 - # Show total times  
207 - print("Total wall time elapsed: "+str(wall_seconds)+" seconds")  
208 - print("Total CPU time used ...: "+str(cpu_seconds)+" seconds")  
209 -  
210 - # Show model fitting results  
211 - #print like.obs().models()  
212 -  
213 - # Optionally plot counts  
214 - if show_data:  
215 - if has_matplotlib:  
216 - plot_counts(like.obs())  
217 - else:  
218 - sys.stdout.write("Matplotlib is not (correctly) installed on your system. No counts spectra are shown.\n")  
219 -  
220 - # Return  
221 - return 146 + FITS files on disk. All data is only hold in memory.
  147 + """
  148 + # Set script parameters
  149 + model_name = "${CTOOLS}/share/models/crab.xml"
  150 + caldb = "${CALDB}/data/cta/dummy/bcf"
  151 + irf = "cta_dummy_irf"
  152 + ra = 83.63
  153 + dec = 22.01
  154 + rad_sim = 10.0
  155 + tstart = 0.0
  156 + tstop = 1800.0
  157 + emin = 0.1
  158 + emax = 100.0
  159 + rad_select = 3.0
  160 +
  161 + # Initialise timing
  162 + wall_seconds = 0.0
  163 + cpu_seconds = 0.0
  164 +
  165 + # Simulate events
  166 + sim = ctools.ctobssim()
  167 + sim["infile"].filename(model_name)
  168 + sim["caldb"].string(caldb)
  169 + sim["irf"].string(irf)
  170 + sim["ra"].real(ra)
  171 + sim["dec"].real(dec)
  172 + sim["rad"].real(rad_sim)
  173 + sim["tmin"].real(tstart)
  174 + sim["tmax"].real(tstop)
  175 + sim["emin"].real(emin)
  176 + sim["emax"].real(emax)
  177 + sim.run()
  178 + print("Simulated events ("+str(sim.celapse())+" CPU seconds)")
  179 +
  180 + # Update timing
  181 + wall_seconds += sim.telapse()
  182 + cpu_seconds += sim.celapse()
  183 +
  184 + # Select events
  185 + select = ctools.ctselect(sim.obs())
  186 + select["ra"].real(ra)
  187 + select["dec"].real(dec)
  188 + select["rad"].real(rad_select)
  189 + select["tmin"].real(tstart)
  190 + select["tmax"].real(tstop)
  191 + select["emin"].real(emin)
  192 + select["emax"].real(emax)
  193 + select.run()
  194 + print("Select event ("+str(select.celapse())+" CPU seconds)")
  195 +
  196 + # Update timing
  197 + wall_seconds += select.telapse()
  198 + cpu_seconds += select.celapse()
  199 +
  200 + # Perform maximum likelihood fitting
  201 + like = ctools.ctlike(select.obs())
  202 + like.run()
  203 + print("Maximum likelihood fitting ("+str(like.celapse())+" CPU seconds)")
  204 +
  205 + # Update timing
  206 + wall_seconds += like.telapse()
  207 + cpu_seconds += like.celapse()
  208 +
  209 + # Show total times
  210 + print("Total wall time elapsed: "+str(wall_seconds)+" seconds")
  211 + print("Total CPU time used ...: "+str(cpu_seconds)+" seconds")
  212 +
  213 + # Show model fitting results
  214 + #print like.obs().models()
  215 +
  216 + # Optionally plot counts
  217 + if show_data:
  218 + if has_matplotlib:
  219 + plot_counts(like.obs())
  220 + else:
  221 + sys.stdout.write("Matplotlib is not (correctly) installed on your system. No counts spectra are shown.\n")
  222 +
  223 + # Return
  224 + return
222 225
223 226
224 # =========== # 227 # =========== #
225 # Plot counts # 228 # Plot counts #
226 # =========== # 229 # =========== #
227 def plot_counts(observations): 230 def plot_counts(observations):
228 - """  
229 - Plot counts.  
230 - """  
231 - # Set legend fontsize  
232 - params = {'legend.fontsize': 10}  
233 - plt.rcParams.update(params)  
234 -  
235 - # Set plot styles  
236 - styles = ['b-', 'g-', 'y-', 'n-']  
237 -  
238 - # Dump header  
239 - print("")  
240 - print("Make plots (using matplotlib):")  
241 - print("==============================")  
242 -  
243 - # Create figure 1  
244 - plt.figure(1,figsize=(12,6))  
245 - plt.subplots_adjust(hspace=.7)  
246 -  
247 - # Create subplot 1  
248 - plt.subplot(121)  
249 - plt.title("Spectrum (summed over all pixels)")  
250 -  
251 - # Loop over observations  
252 - for obs in observations:  
253 -  
254 - # Get event list  
255 - list = obs.events()  
256 -  
257 - # Create energy axis  
258 - ebounds = GEbounds(10, GEnergy(0.1, "TeV"), GEnergy(100.0, "TeV"))  
259 - energy = [ebounds.elogmean(i).TeV() for i in range(ebounds.size())]  
260 -  
261 - # Create spectrum  
262 - print("Extract data:")  
263 - counts = [0.0 for i in range(ebounds.size())]  
264 - print(list.size())  
265 - for atom in list:  
266 - index = ebounds.index(atom.energy())  
267 - counts[index] = counts[index] + 1.0  
268 -  
269 - # Create error bars  
270 - error = [sqrt(c) for c in counts]  
271 -  
272 - # Plot spectrum  
273 - plt.loglog(energy, counts, 'ro', label='data')  
274 - #plt.errorbar(energy, counts, error, fmt=None, ecolor='r')  
275 -  
276 - # Extract models  
277 - #print("Extract models:")  
278 - #sum_model = [0.0 for i in range(ebounds.size())]  
279 - #for k, m in enumerate(observations.models()):  
280 - # print("- "+m.name())  
281 - # model = [0.0 for i in range(ebounds.size())]  
282 - # for atom in list:  
283 - # index = ebounds.index(atom.energy())  
284 - # prob = m.eval(atom, obs)  
285 - # model[index] = model[index] + prob * atom.size()  
286 - # for i in range(ebounds.size()):  
287 - # sum_model[i] = sum_model[i] + model[i]  
288 - # plt.loglog(energy, model, styles[k], label=m.name())  
289 - #plt.loglog(energy, sum_model, 'r-', label='total')  
290 -  
291 - # Put labels  
292 - plt.xlabel("Energy (TeV)")  
293 - plt.ylabel("Counts")  
294 - plt.legend(loc="lower left")  
295 -  
296 - # Create subplot 2  
297 - plt.subplot(122)  
298 - plt.title("Offset (summed over all energies)")  
299 -  
300 - # Set Crab direction  
301 - crab = GSkyDir()  
302 - crab.radec_deg(83.63, 22.01)  
303 -  
304 - # Loop over observations  
305 - for obs in observations:  
306 -  
307 - # Get event cube  
308 - list = obs.events()  
309 -  
310 - # Create offset histogram  
311 - print "Extract data:"  
312 - nx = 30  
313 - doffset2 = 0.01  
314 - offset2 = [(i+0.5)*doffset2 for i in range(nx)]  
315 - counts = [0.0 for i in range(nx)]  
316 - for atom in list:  
317 - off = GCTAInstDir(atom.dir()).dir().dist_deg(crab)  
318 - off2 = off*off  
319 - index = int(off2/doffset2)  
320 - if index < nx:  
321 - counts[index] = counts[index] + 1.0  
322 -  
323 - # Create error bars  
324 - error = [sqrt(c) for c in counts]  
325 -  
326 - # Plot distribution  
327 - plt.semilogy(offset2, counts, 'ro', label='data')  
328 -  
329 - # Extract models  
330 - print "Extract models:"  
331 - sum_model = [0.0 for i in range(nx)]  
332 - for k, m in enumerate(observations.models()):  
333 - print "- "+m.name()  
334 - model = [0.0 for i in range(nx)]  
335 - for atom in list:  
336 - off = GCTAInstDir(atom.dir()).dir().dist_deg(crab)  
337 - off2 = off*off  
338 - index = int(off2/doffset2)  
339 - if index < nx:  
340 - prob = m.eval(atom, obs)  
341 - model[index] = model[index] + prob * atom.size()  
342 - for i in range(nx):  
343 - sum_model[i] = sum_model[i] + model[i]  
344 - #plt.plot(offset2, model, styles[k], label=m.name())  
345 - #plt.plot(offset2, sum_model, 'r-', label='total')  
346 - #plt.ylim(ymin=0.1)  
347 -  
348 - # Put labels  
349 - plt.xlabel("Offset (deg^2)")  
350 - plt.ylabel("Counts")  
351 - plt.legend(loc="upper right")  
352 -  
353 - # Show counts spectra  
354 - plt.show()  
355 -  
356 - # Return  
357 - return 231 + """
  232 + Plot counts.
  233 + """
  234 + # Set legend fontsize
  235 + params = {'legend.fontsize': 10}
  236 + plt.rcParams.update(params)
  237 +
  238 + # Set plot styles
  239 + styles = ['b-', 'g-', 'y-', 'n-']
  240 +
  241 + # Dump header
  242 + print("")
  243 + print("Make plots (using matplotlib):")
  244 + print("==============================")
  245 +
  246 + # Create figure 1
  247 + plt.figure(1,figsize=(12,6))
  248 + plt.subplots_adjust(hspace=.7)
  249 +
  250 + # Create subplot 1
  251 + plt.subplot(121)
  252 + plt.title("Spectrum (summed over all pixels)")
  253 +
  254 + # Loop over observations
  255 + for obs in observations:
  256 +
  257 + # Get event list
  258 + list = obs.events()
  259 +
  260 + # Create energy axis
  261 + ebounds = gammalib.GEbounds(10, gammalib.GEnergy(0.1, "TeV"), gammalib.GEnergy(100.0, "TeV"))
  262 + energy = [ebounds.elogmean(i).TeV() for i in range(ebounds.size())]
  263 +
  264 + # Create spectrum
  265 + print("Extract data:")
  266 + counts = [0.0 for i in range(ebounds.size())]
  267 + print(list.size())
  268 + for atom in list:
  269 + index = ebounds.index(atom.energy())
  270 + counts[index] = counts[index] + 1.0
  271 +
  272 + # Create error bars
  273 + error = [math.sqrt(c) for c in counts]
  274 +
  275 + # Plot spectrum
  276 + plt.loglog(energy, counts, 'ro', label='data')
  277 + #plt.errorbar(energy, counts, error, fmt=None, ecolor='r')
  278 +
  279 + # Extract models
  280 + #print("Extract models:")
  281 + #sum_model = [0.0 for i in range(ebounds.size())]
  282 + #for k, m in enumerate(observations.models()):
  283 + # print("- "+m.name())
  284 + # model = [0.0 for i in range(ebounds.size())]
  285 + # for atom in list:
  286 + # index = ebounds.index(atom.energy())
  287 + # prob = m.eval(atom, obs)
  288 + # model[index] = model[index] + prob * atom.size()
  289 + # for i in range(ebounds.size()):
  290 + # sum_model[i] = sum_model[i] + model[i]
  291 + # plt.loglog(energy, model, styles[k], label=m.name())
  292 + #plt.loglog(energy, sum_model, 'r-', label='total')
  293 +
  294 + # Put labels
  295 + plt.xlabel("Energy (TeV)")
  296 + plt.ylabel("Counts")
  297 + plt.legend(loc="lower left")
  298 +
  299 + # Create subplot 2
  300 + plt.subplot(122)
  301 + plt.title("Offset (summed over all energies)")
  302 +
  303 + # Set Crab direction
  304 + crab = gammalib.GSkyDir()
  305 + crab.radec_deg(83.63, 22.01)
  306 +
  307 + # Loop over observations
  308 + for obs in observations:
  309 +
  310 + # Get event cube
  311 + list = obs.events()
  312 +
  313 + # Create offset histogram
  314 + print "Extract data:"
  315 + nx = 30
  316 + doffset2 = 0.01
  317 + offset2 = [(i+0.5)*doffset2 for i in range(nx)]
  318 + counts = [0.0 for i in range(nx)]
  319 + for atom in list:
  320 + off = gammalib.GCTAInstDir(atom.dir()).dir().dist_deg(crab)
  321 + off2 = off*off
  322 + index = int(off2/doffset2)
  323 + if index < nx:
  324 + counts[index] = counts[index] + 1.0
  325 +
  326 + # Create error bars
  327 + error = [math.sqrt(c) for c in counts]
  328 +
  329 + # Plot distribution
  330 + plt.semilogy(offset2, counts, 'ro', label='data')
  331 +
  332 + # Extract models
  333 + print "Extract models:"
  334 + sum_model = [0.0 for i in range(nx)]
  335 + for k, m in enumerate(observations.models()):
  336 + print "- "+m.name()
  337 + model = [0.0 for i in range(nx)]
  338 + for atom in list:
  339 + off = gammalib.GCTAInstDir(atom.dir()).dir().dist_deg(crab)
  340 + off2 = off*off
  341 + index = int(off2/doffset2)
  342 + if index < nx:
  343 + prob = m.eval(atom, obs)
  344 + model[index] = model[index] + prob * atom.size()
  345 + for i in range(nx):
  346 + sum_model[i] = sum_model[i] + model[i]
  347 + #plt.plot(offset2, model, styles[k], label=m.name())
  348 + #plt.plot(offset2, sum_model, 'r-', label='total')
  349 + #plt.ylim(ymin=0.1)
  350 +
  351 + # Put labels
  352 + plt.xlabel("Offset (deg^2)")
  353 + plt.ylabel("Counts")
  354 + plt.legend(loc="upper right")
  355 +
  356 + # Show counts spectra
  357 + plt.show()
  358 +
  359 + # Return
  360 + return
358 361
359 362
360 #==========================# 363 #==========================#
361 # Main routine entry point # 364 # Main routine entry point #
362 #==========================# 365 #==========================#
363 if __name__ == '__main__': 366 if __name__ == '__main__':
364 - """  
365 - CTA unbinned analysis script.  
366 -  
367 - This script implements the analysis steps to perform an unbinned analysis.  
368 - The steps include:  
369 - - Event simulation  
370 - - Event selection  
371 - - Binned maximum likelihood fitting  
372 -  
373 - Two variations of the script are implemented. One that calls the classes  
374 - like executables, storing the intermediate results on disk, and a second  
375 - that uses the results of the precedent class directly without storing  
376 - any data on disk.  
377 - """  
378 - # Initialise flags  
379 - need_help = False  
380 - show_data = True  
381 -  
382 - # Test for command line arguments  
383 - print sys.argv[0]  
384 - if (len(sys.argv) > 1):  
385 - if sys.argv[1] == "-h":  
386 - need_help = True  
387 - elif sys.argv[1] == "-noshow":  
388 - show_data = False  
389 - else:  
390 - need_help = True  
391 -  
392 - # Print help if needed and exit  
393 - if need_help:  
394 - print "Usage: example_binned.py [OPTIONS]"  
395 - print " -h Display this usage message"  
396 - print " -noshow Do not show data using matplotlib"  
397 - sys.exit()  
398 -  
399 - # Dump header  
400 - print "*********************************"  
401 - print "* CTA unbinned analysis scripts *"  
402 - print "*********************************"  
403 -  
404 - # Remove any existing result files  
405 - list = [glob.glob("*.fits"), glob.glob("*.log"), glob.glob("*.xml")]  
406 - for files in list:  
407 - for file in files:  
408 - os.remove(file)  
409 -  
410 - # Save intermediate results on disk  
411 - print ""  
412 - print "Executable analysis pipeline:"  
413 - print "============================="  
414 - pipeline_v1()  
415 -  
416 - # Do analysis in memory  
417 - print ""  
418 - print "In memory analysis pipeline:"  
419 - print "============================"  
420 - pipeline_v2(show_data)  
421 - print "" 367 + """
  368 + CTA unbinned analysis script.
  369 +
  370 + This script implements the analysis steps to perform an unbinned analysis.
  371 + The steps include:
  372 + - Event simulation
  373 + - Event selection
  374 + - Binned maximum likelihood fitting
  375 +
  376 + Two variations of the script are implemented. One that calls the classes
  377 + like executables, storing the intermediate results on disk, and a second
  378 + that uses the results of the precedent class directly without storing
  379 + any data on disk.
  380 + """
  381 + # Initialise flags
  382 + need_help = False
  383 + show_data = True
  384 +
  385 + # Test for command line arguments
  386 + print sys.argv[0]
  387 + if (len(sys.argv) > 1):
  388 + if sys.argv[1] == "-h":
  389 + need_help = True
  390 + elif sys.argv[1] == "-noshow":
  391 + show_data = False
  392 + else:
  393 + need_help = True
  394 +
  395 + # Print help if needed and exit
  396 + if need_help:
  397 + print "Usage: example_binned.py [OPTIONS]"
  398 + print " -h Display this usage message"
  399 + print " -noshow Do not show data using matplotlib"
  400 + sys.exit()
  401 +
  402 + # Dump header
  403 + print "*********************************"
  404 + print "* CTA unbinned analysis scripts *"
  405 + print "*********************************"
  406 +
  407 + # Remove any existing result files
  408 + list = [glob.glob("*.fits"), glob.glob("*.log"), glob.glob("*.xml")]
  409 + for files in list:
  410 + for file in files:
  411 + os.remove(file)
  412 +
  413 + # Save intermediate results on disk
  414 + print ""
  415 + print "Executable analysis pipeline:"
  416 + print "============================="
  417 + pipeline_v1()
  418 +
  419 + # Do analysis in memory
  420 + print ""
  421 + print "In memory analysis pipeline:"
  422 + print "============================"
  423 + pipeline_v2(show_data)
  424 + print ""