Generic Filters

class myoverse.datasets.filters.generic.ApplyFunctionFilter(input_is_chunked=None, is_output=False, name=None, function=None, **function_kwargs)[source]

Bases: FilterBaseClass

Filter that applies a function to the input array.

Parameters:

  • input_is_chunked (bool) – Whether the input is chunked or not.

  • function (callable) – The function to apply. This can be any function that accepts a numpy array as input and returns a numpy array as output. Example: np.mean or lambda x: x + 1.

  • is_output (bool) – Whether the filter is an output filter. If True, the resulting signal will be outputted by and dataset pipeline.

  • name (str)

__call__(input_array: np.ndarray) → np.ndarray

Apply the function to the input array.

class myoverse.datasets.filters.generic.ChunkizeDataFilter(input_is_chunked=False, chunk_size=None, chunk_shift=None, chunk_overlap=None, is_output=False, name=None)[source]

Bases: FilterBaseClass

Filter that chunks the input array.

Parameters:

  • chunk_size (int) – The size of each chunk.

  • chunk_shift (int) – The shift between each chunk. If provided, the chunk_overlap parameter is ignored.

  • chunk_overlap (int) – The overlap between each chunk. If provided, the chunk_shift parameter is ignored.

  • input_is_chunked (bool) – Whether the input is chunked or not.

  • is_output (bool) – Whether the filter is an output filter. If True, the resulting signal will be outputted by and dataset pipeline.

  • name (str)

__call__(input_array: np.ndarray) → np.ndarray

Filters the input array. Input shape is determined by whether the allowed_input_type is “both”, “chunked” or “not chunked.

class myoverse.datasets.filters.generic.IdentityFilter(input_is_chunked=None, is_output=False, name=None)[source]

Bases: FilterBaseClass

Filter that returns the input array unchanged.

This filter is useful for debugging and testing purposes.

Parameters:

  • input_is_chunked (bool) – Whether the input is chunked or not.

  • is_output (bool) – Whether the filter is an output filter. If True, the resulting signal will be outputted by and dataset pipeline.

  • name (str)

__call__(input_array: np.ndarray) → np.ndarray

Returns the input array unchanged. If the input_array attribute is not None, this array will be returned.

class myoverse.datasets.filters.generic.IndexDataFilter(input_is_chunked=None, indices=None, is_output=False)[source]

Bases: FilterBaseClass

Filter that indexes the input array.

Parameters:

  • indices (Sequence[Union[int, slice]]) – The indices to use for indexing the input array. Example: [0, 1, slice(2, 4)] will select the first two elements of the first dimension and the third and fourth elements of the second dimension.

  • input_is_chunked (bool) – Whether the input is chunked or not.

  • is_output (bool) – Whether the filter is an output filter. If True, the resulting signal will be outputted by and dataset pipeline.

__call__(input_array: np.ndarray) → np.ndarray

Filters the input array. Input shape is determined by whether the allowed_input_type is “both”, “chunked” or “not chunked”.