MuonDataLib Tutorial 4: Time Filtering

In the previous tutorial we used the sample logs to filter the event data. However, it is also possible to filter the events based on time. Lets start by loading the data.

from MuonDataLib.data.loader.load_events import load_events
from MuonDataLib.plot.basic import Figure
import os

file_name = 'HIFI00195790.nxs'
input_file = os.path.join('..', '..', '..', '..', 'test', 'data_files', file_name)
data = load_events(input_file, 64)

WARNING: The metadata **RUN** is missing. Using fallback values
WARNING: The metadata **TITLE** is missing. Using fallback values
WARNING: The metadata **EXPERIMENT IDENTIFIER** is missing. Using fallback values

Using the get_frame_start_times method will allow us to know that we are filtering times that exist within our data set.

frame_start_times = data.get_frame_start_times()
print(f'start time: {frame_start_times[0]}, end time: {frame_start_times[-1]}')

start time: 0.99375522, end time: 3.1737624

The events occur from about \(0.994\) until about \(3.174\) seconds.

We will add a single sample log to show how the time filters impact the data.

def linear(x, m, c):
    return m*x + c
from MuonDataLib.data.utils import create_data_from_function

start = frame_start_times[0]
end = frame_start_times[-1]+1
step = (frame_start_times[-1]-frame_start_times[0])/40

x, y = create_data_from_function(start, end, step, [3.1, 0.1], linear, seed=1)
data.add_sample_log("field", x, y)

Lets start by creating an unfiltered histogram

no_filter_hist, bins = data.histogram()
fig = Figure(y_label='Counts')
fig.plot_from_histogram(bins, no_filter_hist, [0])
fig.show()

Data type cannot be displayed: application/vnd.plotly.v1+json

Time filters - Keeping data between a range of times

The first type of time filter we will look at is one that keeps data between two user specified time stamps. However, the removed data may be slightly before or after these time stamps. This is because the filters will remove all of the data contained within the frame that the filter is placed within.

data.only_keep_data_time_between('filter_1', 1.1, 1.5)

The only_keep_data_time_between command is used to add a time filter that keeps the data between the two user specified values. The first argument is the name we want to give to the filter and the remaining arguments define the range of times that we want to keep. Lets look at how this changes a sample log, but to apply the filter we must generate (or save) a histogram first.

hist_keep_between, bins = data.histogram()
fig = Figure(y_label='Field (MHz)', x_label='Time (seconds)')
fig.plot_sample_log(data, 'field')
fig.show()

WARNING: The target 0.0 is before the first frame start time 0.99375522 seconds. Difference is 0.99375522 seconds
WARNING: The target 4.1737624 is after the last frame start time 3.1737624 seconds. Difference is 1.0 seconds

Data type cannot be displayed: application/vnd.plotly.v1+json

The only data that has been kept is between \(1.1\) and \(1.5\) seconds. We can add additional regions of data that we want to keep by using the only_keep_data_time_between method. However, the names for each filter must be unique.

data.only_keep_data_time_between('filter_2', 2.6, end + 0.01)
hist_keep_between, bins = data.histogram()
fig = Figure(y_label='Field (MHz)', x_label='Time (seconds)')
fig.plot_sample_log(data, 'field')
fig.show()

WARNING: The target 0.0 is before the first frame start time 0.99375522 seconds. Difference is 0.99375522 seconds
WARNING: The target 4.1837624 is after the last frame start time 3.1737624 seconds. Difference is 1.0099999999999998 seconds
WARNING: The target 4.1737624 is after the last frame start time 3.1737624 seconds. Difference is 1.0 seconds

Data type cannot be displayed: application/vnd.plotly.v1+json

There are now two bands of kept data. The second filter used a value larger than the end of the frame start times so we would include the end of the data collection.

Lets see how the time filters alter the histogram.

fig = Figure(y_label='Counts')
fig.plot_from_histogram(bins, no_filter_hist, [0], 'unfiltered, ')
fig.plot_from_histogram(bins, hist_keep_between, [0], 'filtered time between, ')
fig.show()

Data type cannot be displayed: application/vnd.plotly.v1+json

We can see that the filter has removed some counts, as expected.

If we want to remove a time between filter, we can remove it with the command

data.delete_only_keep_data_time_between('filter_1')
data.delete_only_keep_data_time_between('filter_2')

where the argument is the name of the filter to be deleted. We have removed both filters in this example, so the data should look like the unfiltered results.

hist_check, bins = data.histogram()
fig = Figure(y_label='Counts')
fig.plot_from_histogram(bins, no_filter_hist, [0], 'unfiltered, ')
fig.plot_from_histogram(bins, hist_check, [0], 'Check filter removed, ')
fig.show()

Data type cannot be displayed: application/vnd.plotly.v1+json

As you can see from the plot, the two histograms match.

Time filters - Removing data between times

The second type of time filter allows the us to specify the times we want to remove from the data. For example, if we know that the sample logs were not recorded for a period of time.

data.remove_data_time_between('filter_3', 2, 2.6)

The remove_data_time_between command is used to add a filter that removes all of the data between the second and third arguments. Once again the first argument must be a unique name to identify the filter. Just like before we can add multiple filters by repeating the above command.

data.remove_data_time_between('filter_4', start, 1.1)

In this case we want to remove the data from the begining of the collection, so we specify the value for the first frame. Lets check the sample log values

hist_rm_data, bins = data.histogram()
fig = Figure(y_label='Field (MHz)', x_label='Time (seconds)')
fig.plot_sample_log(data, 'field')
fig.show()

Data type cannot be displayed: application/vnd.plotly.v1+json

These look as expected. So lets look at the histogram.

fig = Figure(y_label='Counts')
fig.plot_from_histogram(bins, no_filter_hist, [0], 'unfiltered, ')
fig.plot_from_histogram(bins, hist_rm_data, [0], 'filtered remove ranges, ')
fig.show()

Data type cannot be displayed: application/vnd.plotly.v1+json

Lets remove the filters,

data.delete_remove_data_time_between('filter_3')
data.delete_remove_data_time_between('filter_4')