diff --git a/doc/source/users/tutorials/howto/howto_advanced_python_fitting.ipynb b/doc/source/users/tutorials/howto/howto_advanced_python_fitting.ipynb index e59be31..9c2afbb 100644 --- a/doc/source/users/tutorials/howto/howto_advanced_python_fitting.ipynb +++ b/doc/source/users/tutorials/howto/howto_advanced_python_fitting.ipynb @@ -11,7 +11,7 @@ "cell_type": "raw", "metadata": {}, "source": [ - "
\"Download
" + "
\"Download
" ] }, { diff --git a/doc/source/users/tutorials/howto/howto_combine_observations.ipynb b/doc/source/users/tutorials/howto/howto_combine_observations.ipynb index 087cb06..d0528b0 100644 --- a/doc/source/users/tutorials/howto/howto_combine_observations.ipynb +++ b/doc/source/users/tutorials/howto/howto_combine_observations.ipynb @@ -11,7 +11,7 @@ "cell_type": "raw", "metadata": {}, "source": [ - "
\"Download
" + "
\"Download
" ] }, { diff --git a/doc/source/users/tutorials/howto/howto_connect_irfs.rst b/doc/source/users/tutorials/howto/howto_connect_irfs.rst index 64ac717..4eb53eb 100644 --- a/doc/source/users/tutorials/howto/howto_connect_irfs.rst +++ b/doc/source/users/tutorials/howto/howto_connect_irfs.rst @@ -66,7 +66,7 @@ observation definition XML file: - + Each observation now has a ``Calibration`` parameter with the attributes ``database`` and ``response``. diff --git a/doc/source/users/tutorials/howto/howto_connect_models.ipynb b/doc/source/users/tutorials/howto/howto_connect_models.ipynb new file mode 100644 index 0000000..bcf0627 --- /dev/null +++ b/doc/source/users/tutorials/howto/howto_connect_models.ipynb @@ -0,0 +1,351 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# How to connect observations to specific models?" + ] + }, + { + "cell_type": "raw", + "metadata": {}, + "source": [ + "
\"Download
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the previous examples there was always a single background model component to describe the residual particle background in the various dataset. This implies that the spatial and spectral shape of the background distribution is assumed to be identical for all observations. This is fine in a simulation, but for a real life situation this assumption will probably not hold.\n", + "\n", + "Start by importing the gammalib, ctools, and cscripts Python modules." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import gammalib\n", + "import ctools\n", + "import cscripts" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Simulating the dataset\n", + "\n", + "Let’s start with creating a pointing definition ASCII file for two 30 min pointings near the Crab." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "f = open('pnt.def', 'wb')\n", + "f.write('name,ra,dec,duration\\n')\n", + "f.write('Crab,83.63,21.51,1800.0\\n')\n", + "f.write('Crab,83.63,22.51,1800.0\\n')\n", + "f.close()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Inspect the file that you just created. For that purpose let's create a `peek()` function that will also be used later to display XML files." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "name,ra,dec,duration\n", + "Crab,83.63,21.51,1800.0\n", + "Crab,83.63,22.51,1800.0\n" + ] + } + ], + "source": [ + "def peek(filename):\n", + " f = open(gammalib.expand_env(filename), 'r')\n", + " for line in f:\n", + " print(line.rstrip())\n", + " f.close()\n", + "peek('pnt.def')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A pointing definition file is an ASCII file in Comma Separated Values (CSV) format that specifies one pointing per row. The file provides the `name` of the observation, the Right Ascension `ra` and Declination `dec` of the pointing, and its `duration`. Additional optional columns are possible (defining for example the energy range or the Instrument Response Function), but for this simulation the provided information is sufficient.\n", + "\n", + "Now transform the pointing definition file into an observation definition XML file using the csobsdef script." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "obsdef = cscripts.csobsdef()\n", + "obsdef['inpnt'] = 'pnt.def'\n", + "obsdef['outobs'] = 'obs.xml'\n", + "obsdef['caldb'] = 'prod2'\n", + "obsdef['irf'] = 'South_0.5h'\n", + "obsdef.execute()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's peek the resulting observation definition XML file" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n" + ] + } + ], + "source": [ + "peek('obs.xml')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The file contains two observations which distinguish by the `id` attributes `000001` and `000002`, which are **unique to each observation for a given instrument**. The `id` attribute can therefore be used to uniquely identify a CTA observation.\n", + "\n", + "The value of the `id` attributes can be controlled by adding specific values to the pointing definition file, but if the values are missing - which is the case in the example - they simply count from `000001` upwards.\n", + "\n", + "Feed now the observation definition XML file into the ctobssim tool to simulate the event data." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "obssim = ctools.ctobssim()\n", + "obssim['inobs'] = 'obs.xml'\n", + "obssim['rad'] = 5.0\n", + "obssim['emin'] = 0.1\n", + "obssim['emax'] = 100.0\n", + "obssim['inmodel'] = '$CTOOLS/share/models/crab_2bkg.xml'\n", + "obssim['outevents'] = 'obs_2bkg.xml'\n", + "obssim.execute()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This will produce the two event files `sim_events_000001.fits` and `sim_events_000002.fits` on disk. Note that a specific model was used to simulate the data and peeking that model definiton file shows that it contains two different background components with a different power law `Prefactor` and `Index`. Both background components also have an `id` attribute which is used to tie them to the two observations. In other words, `Background_000001` will be used for the observation with identifier `000001` and `Background_000002` will be used for the observation with identifier `000002`." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "\n" + ] + } + ], + "source": [ + "peek('$CTOOLS/share/models/crab_2bkg.xml')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Analysing the data\n", + "\n", + "Now run a maximum likelihood fit of the model to the simulated data" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "like = ctools.ctlike()\n", + "like['inobs'] = 'obs_2bkg.xml'\n", + "like['inmodel'] = '$CTOOLS/share/models/crab_2bkg.xml'\n", + "like['outmodel'] = 'crab_results.xml'\n", + "like.run()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "and inspect the model fitting results" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "=== GModels ===\n", + " Number of models ..........: 3\n", + " Number of parameters ......: 14\n", + "=== GModelSky ===\n", + " Name ......................: Crab\n", + " Instruments ...............: all\n", + " Instrument scale factors ..: unity\n", + " Observation identifiers ...: all\n", + " Model type ................: PointSource\n", + " Model components ..........: \"PointSource\" * \"PowerLaw\" * \"Constant\"\n", + " Number of parameters ......: 6\n", + " Number of spatial par's ...: 2\n", + " RA .......................: 83.6331 [-360,360] deg (fixed,scale=1)\n", + " DEC ......................: 22.0145 [-90,90] deg (fixed,scale=1)\n", + " Number of spectral par's ..: 3\n", + " Prefactor ................: 5.72600803344134e-16 +/- 7.26125051929832e-18 [1e-23,1e-13] ph/cm2/s/MeV (free,scale=1e-16,gradient)\n", + " Index ....................: -2.46250488685583 +/- 0.0107342940426319 [-0,-5] (free,scale=-1,gradient)\n", + " PivotEnergy ..............: 300000 [10000,1000000000] MeV (fixed,scale=1000000,gradient)\n", + " Number of temporal par's ..: 1\n", + " Normalization ............: 1 (relative value) (fixed,scale=1,gradient)\n", + "=== GCTAModelIrfBackground ===\n", + " Name ......................: Background_000001\n", + " Instruments ...............: CTA\n", + " Instrument scale factors ..: unity\n", + " Observation identifiers ...: 000001\n", + " Model type ................: \"PowerLaw\" * \"Constant\"\n", + " Number of parameters ......: 4\n", + " Number of spectral par's ..: 3\n", + " Prefactor ................: 0.505907031991183 +/- 0.00781154537579767 [0.001,1000] ph/cm2/s/MeV (free,scale=1,gradient)\n", + " Index ....................: 0.19573511102578 +/- 0.00990457582529741 [-5,5] (free,scale=1,gradient)\n", + " PivotEnergy ..............: 1000000 [10000,1000000000] MeV (fixed,scale=1000000,gradient)\n", + " Number of temporal par's ..: 1\n", + " Normalization ............: 1 (relative value) (fixed,scale=1,gradient)\n", + "=== GCTAModelIrfBackground ===\n", + " Name ......................: Background_000002\n", + " Instruments ...............: CTA\n", + " Instrument scale factors ..: unity\n", + " Observation identifiers ...: 000002\n", + " Model type ................: \"PowerLaw\" * \"Constant\"\n", + " Number of parameters ......: 4\n", + " Number of spectral par's ..: 3\n", + " Prefactor ................: 2.0120585192912 +/- 0.0174910114190175 [0.001,1000] ph/cm2/s/MeV (free,scale=1,gradient)\n", + " Index ....................: -0.199891285321279 +/- 0.00515953660690254 [-5,5] (free,scale=1,gradient)\n", + " PivotEnergy ..............: 1000000 [10000,1000000000] MeV (fixed,scale=1000000,gradient)\n", + " Number of temporal par's ..: 1\n", + " Normalization ............: 1 (relative value) (fixed,scale=1,gradient)\n" + ] + } + ], + "source": [ + "print(like.obs().models())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Obviously the two background models have different best fitting spectral parameters which correspond to the simulated values (see above)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 2", + "language": "python", + "name": "python2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 2 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython2", + "version": "2.7.11" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/doc/source/users/tutorials/howto/howto_connect_models.png b/doc/source/users/tutorials/howto/howto_connect_models.png deleted file mode 100644 index bb5d85e..0000000 --- a/doc/source/users/tutorials/howto/howto_connect_models.png +++ /dev/null diff --git a/doc/source/users/tutorials/howto/howto_connect_models.rst b/doc/source/users/tutorials/howto/howto_connect_models.rst deleted file mode 100644 index 6fa2611..0000000 --- a/doc/source/users/tutorials/howto/howto_connect_models.rst +++ /dev/null @@ -1,134 +0,0 @@ -.. _howto_connect_models: - -How to connecting observations to specific models? --------------------------------------------------- - - .. admonition:: What you will learn - - You will learn how to **connect individual observations to specific - model components**. - -In the previous examples there was always a single background model -component to describe the residual particle background in the -various dataset. -This implies that the spatial and spectral shape of the background -distribution is assumed to be identical for all observations. -This is fine in a simulation, but for a real life situation this -assumption will probably not hold. - -Let's go back to the case of a joint analysis of two observations, -specified by the following observation definition XML file: - -.. code-block:: xml - - - - - - - - - - - -Each observation has a **unique identifier string** and you can use this -unique string to connect an observation to a specific model component. -This is illustrated in the model XML file below that now has two background -components. -Both components have an ``id`` attribute that ties the model to a specific -observation. -Using ``id="00001"``, the first background model is tied to the -``events1.fits`` file, using ``id="00002"``, the second background model is -tied to the ``events2.fits`` file. - -.. code-block:: xml - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -.. note:: - **Model components in a XML file need to have a unique name.** - For this reason the components are named here ``Background_00001`` - and ``Background_00002``. - -Note that both background model components distinguish in their event rate. -``Background_00002`` has a ten times larger background rate than -``Background_00001``. -To illustrate this difference, two observations offset by +/- 2 degrees from -the Crab nebulae have been simulated using :ref:`ctobssim`. -A joint fit of the observations is then performed using :ref:`ctlike` and -a residual map is created using :ref:`csresmap`. - -The result is shown below. -Obviously, the upper observation has larger residuals owing to the ten -times larger background rate that was used in the simulation of the -associated data. - -.. figure:: howto_connect_models.png - :height: 400px - :align: center - - *Residual map of two jointly analysed observations offset by +/- 2 degrees* - -The scheme is even more versatile in that it allows to connect a given -model component to several specific observations. -This is illustrated in the model definition file below where component -``Background_00001`` is now applicable for observations ``00001``, -``00003`` and ``00004``. - -.. code-block:: xml - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/doc/source/users/tutorials/howto/howto_stacked_analysis.ipynb b/doc/source/users/tutorials/howto/howto_stacked_analysis.ipynb index 18b7e57..995f068 100644 --- a/doc/source/users/tutorials/howto/howto_stacked_analysis.ipynb +++ b/doc/source/users/tutorials/howto/howto_stacked_analysis.ipynb @@ -11,7 +11,7 @@ "cell_type": "raw", "metadata": {}, "source": [ - "
\"Download
" + "
\"Download
" ] }, { diff --git a/doc/source/users/tutorials/notebooks.rst b/doc/source/users/tutorials/notebooks.rst index a9dcd9f..5bc79aa 100644 --- a/doc/source/users/tutorials/notebooks.rst +++ b/doc/source/users/tutorials/notebooks.rst @@ -15,6 +15,7 @@ it on your computer. All notebooks are also available for download from this page. * :download:`Using ctools from Python ` -* :download:`Combining observations ` -* :download:`Performing a stacked analysis ` -* :download:`Advanced model manipulation and fitting in Python ` +* :download:`How to combine observations? ` +* :download:`How to connect observations to specific models? ` +* :download:`How to perform a stacked analysis? ` +* :download:`How to do advanced model manipulation and fitting in Python? ` diff --git a/models/Makefile.am b/models/Makefile.am index c46f343..1fdbd91 100644 --- a/models/Makefile.am +++ b/models/Makefile.am @@ -1,7 +1,7 @@ ############################################################################# # Makefile for models # # ------------------------------------------------------------------------- # -# Copyright (C) 2016 Juergen Knoedlseder # +# Copyright (C) 2016-2018 Juergen Knoedlseder # # ------------------------------------------------------------------------- # # # # This program is free software: you can redistribute it and/or modify # @@ -22,6 +22,7 @@ # Models to install models = crab.xml \ crab_beyond.xml \ + crab_2bkg.xml \ crab_poly.xml \ crab_profile.xml \ disk.xml \ diff --git a/models/crab_2bkg.xml b/models/crab_2bkg.xml new file mode 100644 index 0000000..880ba77 --- /dev/null +++ b/models/crab_2bkg.xml @@ -0,0 +1,28 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + +