Main class for access to the flagger functionality. More...

#include <aoflagger.h>

Public Member Functions
	AOFlagger ()
	Create and initialize the flagger main class.

	~AOFlagger ()
	Destructor.

void	CollectStatistics (QualityStatistics &destination, const ImageSet &imageSet, const FlagMask &rfiFlags, const FlagMask &correlatorFlags, size_t antenna1, size_t antenna2)
	Collect statistics from time-frequency images and masks. More...

Strategy	LoadStrategy (const std::string &filename)
	Load a strategy from disk. More...

FlagMask	MakeFlagMask (size_t width, size_t height)
	Create a new uninitialized FlagMask with specified dimensions. More...

FlagMask	MakeFlagMask (size_t width, size_t height, bool initialValue)
	Create a new initialized FlagMask with specified dimensions. More...

ImageSet	MakeImageSet (size_t width, size_t height, size_t count)
	Create a new uninitialized ImageSet with specified specs. More...

ImageSet	MakeImageSet (size_t width, size_t height, size_t count, float initialValue)
	Create a new initialized ImageSet with specified specs. More...

ImageSet	MakeImageSet (size_t width, size_t height, size_t count, float initialValue, size_t widthCapacity)
	Create a new initialized ImageSet with specified specs. More...

ImageSet	MakeImageSet (size_t width, size_t height, size_t count, size_t widthCapacity)
	Create a new uninitialized ImageSet with specified specs. More...

QualityStatistics	MakeQualityStatistics (const double scanTimes, size_t nScans, const double channelFrequencies, size_t nChannels, size_t nPolarizations)
	Create a new object for collecting statistics. More...

QualityStatistics	MakeQualityStatistics (const double scanTimes, size_t nScans, const double channelFrequencies, size_t nChannels, size_t nPolarizations, bool computeHistograms)
	Create a new object for collecting statistics, possibly with histograms. More...

Strategy	MakeStrategy (enum TelescopeId telescopeId=GENERIC_TELESCOPE, unsigned strategyFlags=StrategyFlags::NONE, double frequency=0.0, double timeRes=0.0, double frequencyRes=0.0)
	Initialize a strategy for a specific telescope. More...

FlagMask	Run (Strategy &strategy, const ImageSet &input)
	Run the flagging strategy on the given data. More...

FlagMask	Run (Strategy &strategy, const ImageSet &input, const FlagMask &existingFlags)
	Run the flagging strategy on the given data with existing flags. More...

void	SetStatusListener (StatusListener *statusListener)
	Set a handler for progress updates and exceptions. More...

void	WriteStatistics (const QualityStatistics &statistics, const std::string &measurementSetPath)
	Write collected statistics in standard tables to a measurement set. More...

Static Public Member Functions
static void	GetVersion (short &major, short &minor, short &subMinor)
	Get the AOFlagger version number separated in major, minor and subminor fields. More...

static std::string	GetVersionDate ()
	Get the date this version was released as a string. More...

static std::string	GetVersionString ()
	Get the AOFlagger version number as a string. More...

Detailed Description

Main class for access to the flagger functionality.

Software using the flagger should first create an instance of the AOFlagger class, from which other actions can be initiated.

Overview

To flag a data set:

Create the AOFlagger instance
Specify a strategy with MakeStrategy() or LoadStrategy()
Make a data buffer with MakeImageSet()
For each correlated baseline or dish:
- Fill the images with data from this correlated baseline or dish
- Call Run() with the created Strategy and ImageSet
- Process the data that was returned in the FlagMask.

Optionally, it is possible to assemble quality statistics, that can be written to the measurement set in the standard format that e.g. the aoqplot tool can read. To do this:

Create (once) a quality statistics object with MakeQualityStatistics().
After flagging a baseline, add it to the statistics object with CollectStatistics(). A "correlator mask" can be specified that describes which flags are not due to RFI but caused by different things.
When a full set is processed, store the statistics with WriteStatistics().

To flag multiple baselines, the Strategy can be stored and the same instance can be used again.

Thread safety

The Run() method is thread-safe, as long as different ImageSet instances are specified. It is okay to call Run() from different threads with the same Strategy, and it is recommended to do so for multi-threaded implementations. CollectStatistics() is also thread safe, as long as different QualityStatistics instances are passed. For multi-threading, each thread should collect into its own QualityStatistics object. When finished, these can be combined with QualityStatistics::operator+=().

It is okay to create multiple AOFlagger instances, but not recommended.

Data order

A common problem for integrating the flagger, is that data are stored in a different order: the time dimension is often the direction with the slowest increasing indices. Because the flagger needs one baseline at a time, this requires reordering the data. As long as the data fits in memory, this reordering is quite straightforward. When this is not the case, the data could be split into sub-bands and/or time windows. Next, these parts can be passed to the flagger and recombined later (if desired).

To decide how to split, keep in mind that the flagger works best when both a lot of channels and a lot of timesteps are available. As an example: LOFAR splits into subbands of 256 channels, and the default processing with NDPPP loads as many as possible timesteps in memory for flagging. Typically, this means at least a few hundred of timesteps are processed at a time (with 1-3s per timestep), and this seems to work fine.

The 'aoflagger' executable always flags on the full measurement set, which is the most accurate way. For sets that are larger than memory, a mode is used in which the data is reordered to disk before the actual flagging starts. It turns out that this is much faster than reading each baseline directly from the set, and it simultaneously produces the best flagging accuracy, so if enough processing power is available to do so, that should be the preferred way.

Member Function Documentation

◆ CollectStatistics()

void aoflagger::AOFlagger::CollectStatistics	(	QualityStatistics &	destination,
		const ImageSet &	imageSet,
		const FlagMask &	rfiFlags,
		const FlagMask &	correlatorFlags,
		size_t	antenna1,
		size_t	antenna2
	)

Collect statistics from time-frequency images and masks.

This will update the statistics in the destination object so that it represents the combination of previous collected data and the newly given data.

This function can be called from different thread context, as long as the destination is different. See the QualityStatistics class documentation for further multithreading info.

Parameters

destination	Object holding the statistics to which the data will be added
imageSet	Data to collect statistics from
rfiFlags	Flags set by the automatic RFI detector
correlatorFlags	Flags that were set prior to RFI detector, e.g. because of a broken antenna or correlator hickup.
antenna1	Index of the first antenna involved in this baseline.
antenna2	Index of the second antenna involved in this baseline.

◆ GetVersion()

static void aoflagger::AOFlagger::GetVersion	(	short &	major,
		short &	minor,
		short &	subMinor
	)

static

Get the AOFlagger version number separated in major, minor and subminor fields.

Parameters

major	Most significant number of the version, e.g. '1' for version '1.2.3'. This number is only incremented in major changes of the flagger.
minor	Minor number of the version, e.g. '2' for version '1.2.3'. This number is incremented for every public release.
subMinor	Subminor number of the version, e.g. '3' for version '1.2.3', or zero if the current version has no subminor number. This number is incremented for internal releases or small bug fixes.

Since: Version 2.6

◆ GetVersionDate()

static std::string aoflagger::AOFlagger::GetVersionDate ( )

static

Get the date this version was released as a string.

Returns: The version date formatted like "1982-05-08".

Since: Version 2.6

◆ GetVersionString()

static std::string aoflagger::AOFlagger::GetVersionString ( )

static

Get the AOFlagger version number as a string.

Returns: The version number, formatted like '1.2.3'.

Since: Version 2.6

◆ LoadStrategy()

Strategy aoflagger::AOFlagger::LoadStrategy ( const std::string & filename )

inline

Load a strategy from disk.

The best way to create strategies is to use the rfigui tool. In case you have optimized strategies for an unlisted telescope or for new parameters, please let me know so that I can further optimize the flagger.

Parameters

filename Full pathname to .rfis strategy file.

Returns: The new Strategy.

◆ MakeFlagMask() [1/2]

FlagMask aoflagger::AOFlagger::MakeFlagMask	(	size_t	width,
		size_t	height
	)

inline

Create a new uninitialized FlagMask with specified dimensions.

Parameters

width	Width of mask (number of timesteps)
height	Height of mask (number of frequency channels)

Returns: A new FlagMask.

◆ MakeFlagMask() [2/2]

FlagMask aoflagger::AOFlagger::MakeFlagMask	(	size_t	width,
		size_t	height,
		bool	initialValue
	)

inline

Create a new initialized FlagMask with specified dimensions.

Parameters

width	Width of mask (number of timesteps)
height	Height of mask (number of frequency channels)
initialValue	Value to initialize the mask to.

Returns: A new FlagMask.

◆ MakeImageSet() [1/4]

ImageSet aoflagger::AOFlagger::MakeImageSet	(	size_t	width,
		size_t	height,
		size_t	count
	)

inline

Create a new uninitialized ImageSet with specified specs.

The float values will not be initialized.

Parameters

width	Number of time steps in images
height	Number of frequency channels in images
count	Number of images in set (see class description of ImageSet for image order).

Returns: A new ImageSet.

◆ MakeImageSet() [2/4]

ImageSet aoflagger::AOFlagger::MakeImageSet	(	size_t	width,
		size_t	height,
		size_t	count,
		float	initialValue
	)

inline

Create a new initialized ImageSet with specified specs.

Parameters

width	Number of time steps in images
height	Number of frequency channels in images
count	Number of images in set (see class description of ImageSet for image order).
initialValue	Initialize all pixels with this value.

Returns: A new ImageSet.

◆ MakeImageSet() [3/4]

ImageSet aoflagger::AOFlagger::MakeImageSet	(	size_t	width,
		size_t	height,
		size_t	count,
		float	initialValue,
		size_t	widthCapacity
	)

inline

Create a new initialized ImageSet with specified specs.

Parameters

width	Number of time steps in images
height	Number of frequency channels in images
count	Number of images in set (see class description of ImageSet for image order).
initialValue	Initialize all pixels with this value.
widthCapacity	Allow for enlarging image to this size,

See also: ImageSet::ResizeWithoutReallocation()

Returns: A new ImageSet.

Since: 2.6

◆ MakeImageSet() [4/4]

ImageSet aoflagger::AOFlagger::MakeImageSet	(	size_t	width,
		size_t	height,
		size_t	count,
		size_t	widthCapacity
	)

inline

Create a new uninitialized ImageSet with specified specs.

The float values will not be initialized.

Parameters

width	Number of time steps in images
height	Number of frequency channels in images
count	Number of images in set (see class description of ImageSet for image order).
widthCapacity	Allow for enlarging image to this size,

See also: ImageSet::ResizeWithoutReallocation()

Returns: A new ImageSet.

Since: 2.6

◆ MakeQualityStatistics() [1/2]

QualityStatistics aoflagger::AOFlagger::MakeQualityStatistics	(	const double *	scanTimes,
		size_t	nScans,
		const double *	channelFrequencies,
		size_t	nChannels,
		size_t	nPolarizations
	)

Create a new object for collecting statistics.

See the QualityStatistics class description for info on multithreading and/or combining statistics with different meta data. The meta data that is passed to this method will be used for all calls to CollectStatistics() if this class is specified. No histograms will be computed.

◆ MakeQualityStatistics() [2/2]

QualityStatistics aoflagger::AOFlagger::MakeQualityStatistics	(	const double *	scanTimes,
		size_t	nScans,
		const double *	channelFrequencies,
		size_t	nChannels,
		size_t	nPolarizations,
		bool	computeHistograms
	)

Create a new object for collecting statistics, possibly with histograms.

See the QualityStatistics class description for info on multithreading and/or combining statistics with different meta data. The meta data that is passed to this method will be used for all calls to CollectStatistics() if this class is specified.

Since: Version 2.6

◆ MakeStrategy()

Strategy aoflagger::AOFlagger::MakeStrategy	(	enum TelescopeId	telescopeId = `GENERIC_TELESCOPE`,
		unsigned	strategyFlags = `StrategyFlags::NONE`,
		double	frequency = `0.0`,
		double	timeRes = `0.0`,
		double	frequencyRes = `0.0`
	)

inline

Initialize a strategy for a specific telescope.

All parameters are hints to optimize the strategy, but need not actual alter the strategy (or even have correct effects), but the returned strategy should for most common cases be fine. Some properties conflict, e.g., specifying StrategyFlags::HIGH_TIME_RESOLUTION as flag and giving a low timeRes value. In these cases it is not defined which parameter takes precedence, thus it should obviously be avoided.

If frequency value, time resolution or frequency resolution or not known, they can be left at their default values. Currently, they have no effect, but might have effect in later versions. Therefore, if they are known, it is recommended to specify them. They could even identify problematic cases and report this.

Parameters

telescopeId	Identifies the telescope to optimize the strategy for.
strategyFlags	Flags to optimize the strategy further.
frequency	The observation frequency in Hz, or zero if unknown.
timeRes	The time resolution (distance between two consecutive time steps) in s, or zero if unknown.
frequencyRes	The frequency resolution (distance between to channels) in Hz, or zero if unknown.

◆ Run() [1/2]

FlagMask aoflagger::AOFlagger::Run	(	Strategy &	strategy,
		const ImageSet &	input
	)

Run the flagging strategy on the given data.

It is okay to call this function from multiple threads as long as they call Run with a different instance for the input parameter. The strategy parameter can be the same for different threads.

Parameters

strategy	The flagging strategy that will be used.
input	The data to run the flagger on.

Returns: The flags identifying bad (RFI contaminated) data.

◆ Run() [2/2]

FlagMask aoflagger::AOFlagger::Run	(	Strategy &	strategy,
		const ImageSet &	input,
		const FlagMask &	existingFlags
	)

Run the flagging strategy on the given data with existing flags.

This method is similar to Run(Strategy& strategy, const ImageSet&), except that it will pass existing flags (e.g. as set by the correlator) to the flagging strategy, which in the case of bad data can do a better job of finding RFI in the good data. input parameter. The strategy parameter can be the same for different threads.

Parameters

strategy	The flagging strategy that will be used.
input	The data to run the flagger on.
existingFlags	Flags that indicate what data are bad.

Returns: A flag mask that identifies bad (RFI contaminated) data.

Since: 2.12

◆ SetStatusListener()

void aoflagger::AOFlagger::SetStatusListener ( StatusListener * statusListener )

inline

Set a handler for progress updates and exceptions.

By default, exceptions will be reported to stderr and progress updates will be ignored. If an application needs to handle either of these themselves, they can override a StatusListener that handles these events and call this method to enable receiving the events. This method is not thread safe.

Parameters

statusListener The handler that will receive the status updates.

Since: Version 2.7

◆ WriteStatistics()

void aoflagger::AOFlagger::WriteStatistics	(	const QualityStatistics &	statistics,
		const std::string &	measurementSetPath
	)

Write collected statistics in standard tables to a measurement set.

Parameters

statistics	The collected statistics
measurementSetPath	Path to measurement set to which the statistics will be written.

The documentation for this class was generated from the following file:

aoflagger.h

Public Member Functions

Static Public Member Functions

Detailed Description

Overview

Thread safety

Data order

Member Function Documentation

◆ CollectStatistics()

◆ GetVersion()

◆ GetVersionDate()

◆ GetVersionString()

◆ LoadStrategy()

◆ MakeFlagMask() [1/2]

◆ MakeFlagMask() [2/2]

◆ MakeImageSet() [1/4]

◆ MakeImageSet() [2/4]

◆ MakeImageSet() [3/4]

◆ MakeImageSet() [4/4]

◆ MakeQualityStatistics() [1/2]

◆ MakeQualityStatistics() [2/2]

◆ MakeStrategy()

◆ Run() [1/2]

◆ Run() [2/2]

◆ SetStatusListener()

◆ WriteStatistics()