IC ===== IC stands for `Invisible Cities `_. and can be found on Github as `next-exp/IC `_. Blame for the name to J.J. Gómez-Cadenas, who proposed and started the project. The original developers of IC were J.J. Gómez-Cadenas, J. Generowicz, J.M. Benlloch, G. Martinez-Lema and M. Kekic. The main architect of the code was J. Generowicz. **Language**: Python; **Based**: Numpy, Pandas This section is focused on reviewing how the code is structured and where the installation guides can be found. .. _ICcode: Code ------------ IC is a reconstruction software structured in algorithms/programs named *cities*. The code is written in Python, but it uses `Numpy `_ and `Pandas `_ for data analysis and as a manipulation tool. Each of the *cities* is focused on a specific reconstruction *stage* (e.g. raw waveforms integration, energy correction, deconvolution, etc...), and follow *dataflow* data processing: they resemble a pipeline of data transformations. More details about the dataformat can be found in a previous IC-crash-course in `Dataflow `_ section. A complete review about the IC repository can be found on a previous IC-crash-course under `IC-structure `_. .. _ICinstallation: Installation ------------ There is a *Quickstart guide* at the next-exp/IC repository with anything related to its installation. The simplest way of installing the software is to download the repository with .. code-block:: text git clone git@github.com:next-exp/IC.git Note that the cloning process would be different for users and developers. If someone only wants to use the software they can download from next-exp, but it is recommended for both users and potential developers to fork the repository, and then clone their fork. Check additional details about it in :doc:`workflow` section. Once the repository is downloaded, run .. code-block:: text source manage.sh install_and_check 3.8 from the top level of the IC folder. This will download and install `miniconda `_ if necessary prior to installing IC and its dependencies. Miniconda would be installed in a default location (``$HOME/miniconda``). If you already have an anaconda installation, the first step will be skipped. Usage ------------ After running the installation commands, your shell will be configured to use IC. To set up IC in a new shell run .. code-block:: text source manage.sh work_in_python_version_no_tests 3.8 City Structure ---------------- IC cities follow dataflow data processing: they are designed to resemble a pipeline. By reading this pipeline one should be able to understand what operations are made and in which order. .. image:: images/pipeline.png :width: 650 More details about how this data format works can be found in a previous `IC-crash-course `_. Resembling this pipeline structure, each city needs a specific type of *input* data format, and creates a specific *output* to follow the rest of the production flow: .. image:: images/citystructure.png :width: 350 Configuration files ------------------- Cities may require some parametrisation and, as it is represented on the previous figure, some of them require additional auxiliar (*aux*) data (like maps, PSFs, etc). Configuration files (*config*, ``config_file_city.conf``) provide to the city this specific information they require to run. Examples for each city are located in `IC/invisible_cities/config `_. Official production config files (and production scripts) are located in `next-exp/CERES `_ repository. .. image:: images/configfile.png :width: 850 .. note:: Config files located in `IC/invisible_cities/config `_ are only for testing purposes and **not** realistic. .. _Common arguments to every city: Common arguments to every city :::::::::::::::::::::::::::::: All cities in IC require at least the following arguments .. list-table:: :widths: 40 120 120 :header-rows: 1 * - **Parameter** - **Type(s)** - **Description** * - ``files_in`` - ``str`` or ``Sequence[str]`` - Input file name(s). * - ``file_out`` - ``str`` - Output file name. * - ``compression`` - ``str`` - Compression option. Always ``"ZLIB4"``. * - ``event_range`` - ``int``, ``(int, int)``, ``(int, last)`` or ``all`` - Range of events to process. If an integer N is provided, the first N events are taken. Two integers (N, M) will run from event N to event M. If (N, ``last``) the first N events will be skipped. If ``all``, all events will be processed. * - ``print_mod`` - ``int`` - How frequently to print progress to the std output. * - ``detector_db`` - ``str`` - Name of the detector for database access. * - ``run_number`` - ``int`` - Run number corresponding to the data. Needed to load the appropriate sensor parameters. If negative, the processing is assumed to be a MC run with the corresponding to the detector conditions of run ``abs(run_number)``. How to run a city ----------------- Once it is clear the parametrisation needed to run a city (once IC environment is set), you just need to type: .. code-block:: text city city_name config_file_city.conf where ``config_file_city.conf`` corresponds to the specific configuration file for that city. List of Cities ------------------ IC cities can be categorised depending on their purpose on the following list: MAIN PRODUCTION: .. toctree:: :maxdepth: 1 Irene Penthesilea Sophronia Dorothea Esmeralda Beersheba Isaura Eutropia CALIBRATION: .. toctree:: :maxdepth: 1 Phyllis Trude Berenice ONLY FOR MC: .. toctree:: :maxdepth: 1 Detsim Buffy Diomira Hypathia DEBUGGING/CONTROL: .. toctree:: :maxdepth: 1 Isidora Olivia Each of this cities include a small description in the IC repository (`IC/invisible_cities/cities `_), .. image:: images/cityfunctionality.png :width: 800 and under :doc:`prodflow` a complete review of the IC cities chain can be found for both data and simulations. A tour through the Invisible Cities ----------------------------------- The material from the IC software course that took place in April 2024 is available in `a github repository `_ It contains a series of jupyter notebooks which describe the cities, their inputs and outputs. The user can execute the notebooks, and all inputs are included (config files, data files, etc.). The recording of the course is also available in `this docDB page `_. Contact ------- If you are missing something, or you would like to contribute, contact any of our **Software Manager**: `Gonzalo Martínez-Lema `_ If you have any question, or you would like to discuss something related to NEXUS with other users or developers, you can also write on the **Slack Channels**: * `#IC_support `_: Support channel for users to raise issues and questions.