Skip to content

0.6.2 -- Calibration via dreamBeam, mmap reader, back-end cleanup, Stokes U fix
Compare
Choose a tag to compare
@David-McKenna David-McKenna released this 03 Feb 15:28
0.6.2
3d396cc

0.6.0 introduces support for calibrating data via Jones matrices calculated by Tobia Carozzi's dreamBeam (2baOrNot2ba/dreamBeam). This will correct voltages to be in J2000-coordinates, though can be configured with any input coordinate system supported by casacore (J2000, SUN, JUPITER, AZELGO...). Applying calibration to your data will result in the output data type always being a 32-bit float as we do not apply type conversion to the results after applying the Jones matrix.

Similarly, we now use mmap(2) to read data for the zstandard reader, which has shown 5-15% improvements in read times over the old fread(3) methodology. The madvise(2) calls to de-allocate the memory are fairly slow (10-40ms on REALTA, without the library will use all available memory on all CPUs), but there is still a net gain from the process.

This release also cleans up parts of the back-end so that they are easier to manage and expand. This include having a single C-C++ bridge function rather than one per mode, introducing inline functions for calculating standard offsets in the processing kernels (input, time-major output, frequency-major output), and adjusting input types to be an enum rather than a true/false check on zstandard compression. GCC-compiled outputs now also use the task based processing that was used with ICC before as it no longer takes an eternity (but it is still fairly slow compared to ICC).

Calibration process does need a bit more documentation, but time is short at the moment. Tl;dr, posix_spawnp -> python -> pipe -> C function

0.6.1 introduces a fix for an incorrect definition of Stokes U spotted by Tobia Carozzi (@2baOrNot2ba )

0.6.2 hides some verbose messages that were always printed on some execution paths rather than checking if the DEBUG flag was set.

Minor additions/changes:

  • Added a 'stationID' meta parameter with the name of the station, as read from the CEP packets.
  • Added support for time-major Stokes outputs, though this currently not exposed to the reader.
  • Added VERBOSEP for printing an error that mentions the function name (TODO: convert verbose messages over to this define)
  • CLI: -c (clock mode) moved to -z to allow for calibration to use -c
  • Stokes functions now use ints instead of floats to improve SIMD throughput (future: multiple functions of different types to improve speeds even more? Not sure how to get templates and typedefs working together though.)
  • Add an extra verbose more (meta->verbose = 2) for printing the outputs during the kernel processing
  • lofar_udp_misc and the initialisation functions now operate directly on the data rather than using a union struct to get ints from chars
  • Moved a lot of common defines/includes to lofar_udp_general.h
  • Number of OpenMP threads can now be set by reader->ompThreads rather than being fixed at compile time
  • Added a few more UDPHDROFF offsets to packet checks; not complete though.
  • We no longer reset the number of OpenMP threads on every reader call

Minor fixes:

  • Heavily desyn'd ports (> 1 iteration worth of packets) will now be correctly aligned rather than cause a segfault on the first iteration (fixes #5)
  • Several smaller bounds checks to make sure we don't overrun our buffer when dealing with packet loss while skipping to/from packets (fixes #3)
  • Fix the ascii_hdr_manager (GUPPI RAW output) not correctly resetting fargc for parsing metadata parameters (fixes #4)
  • Error if the clock types are mixed
Assets 2
Loading