McStas and Mantid

A note on the McStas Mantid integration

The McStas neutron ray-tracing simulation package is a versatile tool for producing accurate simulations of neutron scattering instruments at reactors and pulsed spallation sources [2, 3, 5]. McStas is extensively used for the design and optimization of instruments, virtual experiments, and user training. McStas is an abbreviation for Monte Carlo Simulation of triple axis spectrometers, but allows for describing and simulating any types of neutron scattering instrument.

The Mantid project provides a framework that supports high-performance computing and visualisation of scientific data [4, 6]. The Mantid framework is a software solution developed for the analysis and visualization of neutron scattering and muon spin measurements. Mantid is an abbreviation for Manipulation and Analysis Toolkit for Instrument Data.

By combining both the McStas neutron transport and the Mantid reduction framework a complete simulation of a neutron scattering experiment can be achieved; where neutron transport and data reduction can be done in one go.

The NeXus data format

A number of large-scale neutron facilities uses the NeXus data format [7] for storing neutron data. NeXus is derived from the Hierarchical Data Format (HDF) [8]. The specific detail of how the neutron counts and metadata are stored in the NeXus files changes from beam line to beam line. However, the NeXus application programming interface (API) is supported on multiple platforms and has library support for multiple programming languages [7]. McStas can write NeXus files and Mantid can read and write NeXus data files.

To use the McStas-Mantid interface the libraries for the NeXus data format must be installed on your computer. See section Install NeXus for how to install NeXus.

McStas Mantid workflow

Generating McStas event data for Mantid is slightly different from running a standard McStas simulation. It requires the following procedure:

Generate the instrument file for Mantid, from the McStas instrument file.
Compile the McStas c code. Remember to link correctly to NeXus libraries.
Run McStas simulation with the NeXus flag "on" to produce event data for Mantid.

Step 1 will generate an xml file for Mantid with a description of the instrument geometry known as the Instrument Definition File (IDF). The IDF is used for time-of-flight (TOF) conversions in Mantid. Importantly, the IDF requires a unique Detector ID for each detector pixel, which are used to link neutron events with detector pixels. Step 2 is used to make sure your McStas simulation can produce a NeXus file as output. Step 3 is used to run the McStas simulation and ask for a NeXus file as output of the simulation, as opposed to several ascii files, which is the default setup in McStas.

The McStas graphical user interface (GUI) is named mcgui. Below we show how to configure mcgui such that McStas event data for Mantid can easily be performed from the McStas GUI. Step 2 and 3 depend on your computers operating system (OS), as we show below.

Setup the McStas instrument to create a Mantid instrument

For McStas to autogenerate a Mantid IDF the McStas instrument file must obey:

McStas instrument file name and the McStas defined name of the instrument must be the same
- E.g. templateSANS_Mantid.instr and DEFINE INSTRUMENT templateSANS_Mantid(…)
In the McStas instrument file the source must be named “sourceMantid”
- E.g. COMPONENT sourceMantid = Source_simple(…)
In the McStas instrument file the sample must be named “sampleMantid”
- E.g. COMPONENT sampleMantid = Sans_spheres(…)
In the McStas instrument file the event monitors must be named “nD_Mantid_#”
- E.g. COMPONENT nD_Mantid_1 = Monitor_nD(…) for the first event detector bank
- E.g. COMPONENT nD_Mantid_2 = Monitor_nD(…) for the second event detector bank

To use a square shaped monitor_nD the COMPONENT nD_Mantid_1 must be called with the argument: options ="mantid square x limits=[-0.2] bins=128 y limits=[-0.2] bins=128, neutron pixel min=0 t, list all neutrons". The number of bins and limits can be chosen freely, but "list all neutrons" is required. 'min=0' specifies the detector IDs required for Mantid, where 'min=0' means that the first detector ID will be 0 the next 1 and so on. Each detector pixel in Mantid must have associated with it a unique detector ID. Hence if you if have a mcstas instrument with several detector banks then you much ensure that these are unique. However these numbers don't have to be in increments of one.

For a simple example of an instrument with one bank see: templateSANS_Mantid.instr.

For a simple example of an instrument setup to study multiple scattering see: templateVanadiumMultipleScat_Mantid.instr. Multiple scattering is a special case since exactly the same Mcstas detector components are specified more than once in the McStas instrument file. In the simple example this is:

%{
if (SCATTERED == 1) single_flag =1 ;
if (SCATTERED > 1) multi_flag =1 ;
%}

COMPONENT nD_Mantid_0 = Monitor_nD(
    options ="Mantid square, x limits=[-2.5 2.5] bins=50 y limits=[-2.5 2.5] bins=50, neutron pixel min=0 t limits [0.002,0.005] list all neutrons, file=multi",
    xwidth = 5, yheight = 5, restore_neutron = 1)
  WHEN (multi_flag ==1)
  AT (0, 0, 1) RELATIVE sampleMantid

 
COMPONENT Mantid_1 = Monitor_nD(
    options ="square, x limits=[-2.5 2.5] bins=50 y limits=[-2.5 2.5] bins=50, neutron pixel min=0 t limits [0.002,0.005] list all neutrons, file=single",
    xwidth = 5, yheight = 5, restore_neutron = 1)
  WHEN (single_flag ==1)
  AT (0, 0, 1) RELATIVE sampleMantid

As is seen above exactly the same component is specified twice with particles recorded in file=multi and file=single respectively. To avoid that a Mantid instrument is created with two identical components sitting on top of each other, the Mantid key-words are only used for the first component. That way only this first component is added to the Mantid IDF and Mantid will use this IDF to create workspaces for both file=multi and file=single.

A banana shaped monitor can be added as follows: nD_Mantid_1 = Monitor_nD(xwidth=2*radius, yheight=0.8, options="mantid banana, theta limits=[-173 173] bins=100, y limits=[-0.4 0.4] bins=100, neutron pixel min=0 t, list all neutrons"). A full example of how to such a component can be specified is:

COMPONENT nD_Mantid_1 = Monitor_nD(
     xwidth=sampdet, yheight=0.1,
     options="mantid banana, theta limits=[0 27] bins=256, y bins=1, neutron pixel min=0 t, list all neutrons",
	 filename = "bank01_events.dat")
  AT(0.0, 0.0, 0.0) RELATIVE PREVIOUS
```c

McStas also supports event monitors defined by [OFF](https://en.wikipedia.org/wiki/OFF_(file_format)) geometry files. Contact the McStas developers on how to use OFF detectors at <[email protected]>.



## McStas GUI
Generating McStas event data for Mantid can be achieved from the McStas GUI `mcgui`. Below we show how to setup the simulation on Windows 7, OSX 10.12, and Ubuntu 16.04. For [McStas](http://mcstas.org/download) we use version 2.4.1. For [Mantid](http://download.mantidproject.org) use version 3.4 or later.


**Windows users** need to download the latest version of [mccode-r.c](https://raw.githubusercontent.com/McStasMcXtrace/McCode/master/common/lib/share/mccode-r.c) in order to generate IDF data in the mccode.h5 file which Mantid can read. Copy the downloaded file to the directory where your McStas instrument file is located. 


1. Open the McStas configuration file. In `mcgui` go to: File -> Configuration
2. Change the setting as shown in figures below:
   * In the section `mcrun` select `mcrun --format=NeXus`
   * In the section `mcplot` select `mcplot-pyqtgraph`
   * In the section `mcdisplay` select `mcdisplay-pl --format=Mantid` (Windows) or `mcdisplay.pl --format=Mantid` (OSX or Ubuntu)
   * In the section `C flags` select this line depending on your OS
     * Windows: `-g -O2 -lm -DUSE_NEXUS -llibNeXus-0 -I "C:/Program Files (x86)/NeXus Data Format/include/nexus" -L "C:/Program Files (x86)/NeXus Data Format/lib/nexus"` 
     * OS X: `-g -O2 -lm -DUSE_NEXUS -lNeXus -I/usr/local/include/nexus`
     * Ubuntu: `-g -O2 -lm -DUSE_NEXUS -lNeXus` 


<figure>
<img src="https://github.com/McStasMcXtrace/McCode/blob/master/doc/Wiki/McStas_Mantid/Win_config.png" height="495">
<img src="https://github.com/McStasMcXtrace/McCode/blob/master/doc/Wiki/McStas_Mantid/OSX_config.png" height="495">
<img src="https://github.com/McStasMcXtrace/McCode/blob/master/doc/Wiki/McStas_Mantid/Ubuntu_config.png" height="495">
<br>
<figcaption><b>Figure 1:</b> McStas GUI configuration widget. Windows 7 (top left), OS X 10.12 (top right), Ubuntu 16.04 (bottom).</figcaption>
</figure>

<hr />


In Fig. 1 the `mcgui` configuration widgets for Windows, Mac OS X, and Ubuntu are shown. Press `Save` to keep the configuration settings.
 

You should check your local installation if the above mentioned paths to libraries and header files are equal – otherwise modify accordingly to your laptop settings.

3. Load instrument file:
   * Download and use the latest version of [templateSANS_Mantid.inst](https://raw.githubusercontent.com/McStasMcXtrace/McCode/master/mcstas-comps/examples/templateSANS_Mantid.instr). Select `Open` and choose the downloaded instrument file.   
4. Compile the instrument file: Simulation -> Compile Instrument
5. Generate the IDF: Press `Run` in `mcgui`:
   * In the section `Simulation/Trace` choose `Trace`
   * In the section `Particle count` type `1`
   * Run simulation: Press `Start`

<figure>
<img src="https://github.com/McStasMcXtrace/McCode/blob/master/doc/Wiki/McStas_Mantid/IDF_gen.png" height="500">
<br>
<figcaption><b>Figure 2:</b> McStas GUI for generating the IDF xml file for Mantid.</figcaption>
</figure>

<hr />


A new file is written to disk: templateSANS_Mantid.instr.xml

6. Run the McStas simulation: Press `Run` in `mcgui`:
   * In the section `Simulation/Trace:` choose `Simulation`
   * In the section `Particle count:` type `1E8`
   * In the section `Output subdir(optional):` type `1E8`
   * Run simulation: Press `Start`


<figure>
<img src="https://github.com/McStasMcXtrace/McCode/blob/master/doc/Wiki/McStas_Mantid/Run_NeXus.png" height="450">
<img src="https://github.com/McStasMcXtrace/McCode/blob/master/doc/Wiki/McStas_Mantid/Run_NeXus_output.png" height="450">
<br>
<figcaption><b>Figure 3:</b> McStas GUI run widget (top). McStas GUI simulation started (bottom).</figcaption>
</figure>

<hr />


A new folder is written to disk `1E8` with two files: `mccode.h5` and `templateSANS_Mantid.instr`. Notice: NeXus files cannot be plotted from McStas.


## McStas CLI
Generating McStas event data for Mantid can also be controlled from the terminal, using this command line interface (CLI). _Below we show how to setup the simulation for OSX 10.12. The procedure for Windows and Ubuntu is similar._ For McStas we use version 2.4.1. 

**Windows users** need to download the latest version of [mccode-r.c](https://raw.githubusercontent.com/McStasMcXtrace/McCode/master/common/lib/share/mccode-r.c) in order to generate IDF data in the mccode.h5 file which Mantid can read. Copy the file to the directory where your McStas instrument file is located.


### Mac OS X
1. Compile McStas instrument file
```{r, engine='bash', count_lines}
$ mcstas templateSANS_Mantid.instr --trace

Generate the IDF for Mantid

$ mcdisplay.pl templateSANS_Mantid.instr --format=Mantid -n0

Generate the executable

$ gcc -o templateSANS_Mantid.out templateSANS_Mantid.c -lm -DUSE_NEXUS -lNeXus -I/usr/local/include/nexus/

Run the executable and use the default parameters

$ ./templateSANS_Mantid.out --format=NeXus -d 1E8

MantidPlot view of McStas event data

Below we show how the simulated data from templateSANS_Mantid.instr can be viewed in Mantid by just using the Mantid GUI. Use Mantid version 3.4 or later.

Open MantidPlot
Press load and select the just created Mcstas file mccode.h5 (created following the above procedure)

Figure 4 shows a typical view of McStas data in Mantid. Use the following steps:

In Workspaces left-click on the McStas data mccode
Right-click on EventData_mccode choose Show Instrument
Zoom in/out and rotate to find the detector
Right-click on Edet0.dat_mccode choose Plot Spectrum. (Use log scale for y-axis.)
Right-click on PSD.dat_mccode choose Color fill plot. (Use log scale for colour plot.)

Figure 4: Mantidplot used to display event and histogram data stored in the McStas file mccode.h5

Mantid reduction of McStas event data

Once McStas event data has been loaded into Mantid, all of Mantid analysis methods can be applied to the dataset. This makes already developed and tested methods from Mantid directly applicable to McStas data, and will in turn make the analysis faster as the users do not have to reinvent and re-implement reduction and analysis methods.

To show how the reduction methodology works in Mantid we shall consider the McStas event data set generated with the templateSANS_Mantid.instr file. The goal of this small exercise is to compare the reduced scattering curve evaluated by Mantid to the scattering kernel used in the McStas sample component. By a proper set of operations on the original McStas event data [1], Mantid can eventually return the scattering curve I(q) for the data recorded on the detector, and hence the scattering cross section of the sample can be deduced; including sample specific parameters.

Use the steps and python script listed below in Mantid to evaluate I(q):

In the Script Interpreter window (i.e. the IPython Console) navigate to the folder with the McStas NeXus event data: cd /path/to/folder
Open the Python script editor: View -> Script Window
Copy the following Python code to the Script Window
Run reduction: Execute -> Execute All

from mantid.simpleapi import *

# load McStas NeXus file
ws = Load('mccode.h5')

# Read the event data. Rebin and TOF conversion
sample = Rebin(ws[0],'10', False)
sample = ConvertUnits(sample, 'Wavelength')
binning = '1.75,0.05,16.5'
sample  = Rebin(sample, binning)

# Read normalizaion data
mcstasMonitor = CloneWorkspace(mtd['Edet0.dat_ws'])
mcstasMonitor = ConvertToHistogram(mcstasMonitor)
mcstasMonitor = rebin(mcstasMonitor, binning)
monitor = CreateWorkspace(mcstasMonitor.dataX(0), mcstasMonitor.dataY(0), mcstasMonitor.dataE(0), UnitX='Wavelength')

# Reduce data to scattering cuvre I(q)
binningMASK = '0.0035,0.001,0.7'
Iq_1D = Q1D(sample, binningMASK, WavelengthAdj='monitor')

See Ref. [1] for full explanations of the specific Mantid methods used in this tutorial. For comparison, we have also included the scattering curve evaluated by SasView, a SANS data analysis tool [9], using the same material parameters as given in the McStas scattering sample “Sans_spheres”.

Figure 5: Mantidplot used reduced McStas event data. The Python script is copied to Script Window (left) and the associated reduced data I(q) is plotted and compared with a theoretical prediction using SasView (right).

ISIS SANS2D

In this section, we shall consider a more realistic McStas description of a neutron scattering instrument: the ISIS SANS2d beam line. A McStas instrument file of the base line for SANS2d can be found in the McStas neutron suites of instruments: ISIS_SANS2d.instr. To generate McStas event data for SANS2d ready to be loaded by Mantid we need to change the ISIS_SANS2d.instr file according to the McStas conventions (described above in the section Setup the McStas instrument to create a Mantid instrument). This implies adding the keywords and the associated McStas components: soruceMantid, sampleMantid and nD_Mantid_1. The resulting McStas instrument file we will denote ISIS_SANS2d_Mantid.instr. It can be downloaded from the McStas repository. For this tutorial, we shall also need to download the latest version of SANS_benchmark2.comp. It is the SANS scattering sample we will use for the instrument simulation. Run the McStas simulation as described above.

Figure 6: Mantid view on the neutron optics geometry used in the McStas instrument file. Use the method LoadEmptyInstrument in Mantid to display the IDF file. The IDF file must be generated by using mcdisplay with the option "--complete". This will render the complete instrument file in IDF xml file.

The SANS2D instrument has a curved neutron guide. See Fig. 6. The bending optics is included in the McStas description, but it poses a problem for Mantid, since the time-of-flight (TOF) conversion in Mantid assumes a straight flight path from source to sample. To by-pass this, we simply artificially introduce the sourceMantid component as a specific point in space without any neutron transport duties associated (i.e. just as an Arm). At the same time, we keep the ISIS source component at the origin, see Fig.7. In summary we have then kept the neutron transport correct, but adjusted the geometry (the IDF file) passed on to Mantid, such that the flight path from source to sample appears as a straight line to Mantid. See the instrument file ISIS_SANS2d_Mantid.instr for implementation details.

Figure 7: Mantid view on the neutron optics geometry used in the McStas instrument file with a zoom-in on the origin around the source.

Figure 8 below shows the resulting reduced data I(q) for running two simulations of the ISIS_SANS2D_Mantid.instr file. In both cases the sample consists of identical nano spheres with a radius of 150Å; i.e. monodisperse, homogeneous particles in dilute solution, where the particles are located randomly in the sample. In the first simulation, Sim1, we have kept the default instrument settings and run a McStas simulation with 1E8 rays. In the second simulation, Sim2, we have change the radius of slit S6 and collimation slit A2 to 1mm and run a McStas simulation with 3E10 rays. The McStas event data can then be reduced to get the scattering curve I(q) using the Python script as described above in the section Mantid reduction of McStas event data.

Figure 8: Plots of reduced data I(q) using Mantid for simulation Sim1 and Sim2 compared with the theoretical predictions from SasView.

Install NeXus

In order for McStas to produce Mantid readable output, NeXus needs to be installed.

Windows

The latest binary NeXus installer for Windows can be found at the McStas web site. Download the NeXus file and follow the install instructions.

Mac OS X

NeXus can be obtained from the Homebrew page. Installation is done from the command line.

Open a browser and go to the webpage https://brew.sh. If brew is installed, then jump to item 3.
Follow the install instructions. Type in a terminal:

$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Add “science” to search path

$ brew tap homebrew/science

Install NeXus data format

$ brew install nexusformat

Ubuntu 16.04 LTS

NeXus can be installed from the command line. (NeXus for other Linux distributions are installed in a similar manner.)

Search for NeXus

$ apt-cache search libnexus

Install NeXus

$ sudo apt-get install libnexus0v5

Optionally you may download the NeXus source code (https://github.com/nexusformat/code) and compile the library locally.

References

[1] Nielsen, T.R.; Markvardsen, A.J. and Willendrup, P. "McStas and Mantid integration". Journal of Neutron Research, vol. 18, no. 2-3, pp. 79-92, 2015. http://content.iospress.com/articles/journal-of-neutron-research/jnr026

[2] Willendrup P.; Farhi E. and Lefmann K. "McStas 1.7 - A New Version of the Flexible Monte Carlo Neutron Scattering Package", Physica B: Condensed Matter. Volume 350, Issues 1–3, Supplement, 15 July 2004, Pages E735–E737. https://doi.org/10.1016/j.physb.2004.03.193

[3] Willendrup, P.; Farhi E.; Knudsen E.; Filges U. and Lefmann K; "McStas: past, present and future". Journal of Neutron Research, vol. 17, no. 1, pp. 35-43, 2014. http://content.iospress.com/articles/journal-of-neutron-research/jnr004

[4] O. Arnold; et al., "Mantid—Data analysis and visualization package for neutron scattering and μSR experiments", Nuclear Instruments and Methods in Physics Research Section A, Volume 764, 11 November 2014, Pages 156-166. http://dx.doi.org/10.1016/j.nima.2014.07.029

[5] http://www.mcstas.org

[6] http://www.mantidproject.org

[7] http://www.nexusformat.org

[8] https://www.hdfgroup.org/HDF5

[9] http://www.sasview.org