module: base

class annotlib.base.BaseAnnot

Bases: abc.ABC

This class is an abstract representation of base annotators. It defines the required methods and implements methods to evaluate the labelling performance and to illustrate the samples labelled by the annotators.

class_labels(self, X, annotator_ids, query_value, **kwargs)

Method returning the class labels of the given samples.

Parameters:

X: array-like, shape (n_samples, n_features)

Samples whose class labels are queried.

annotator_ids: array-like, shape (n_queried_annotators)

The indices of the annotators whose class labels are queried.

query_value: int

The query_value represents the increment of the query statistics of the queried annotators.

Returns:

Y: numpy.ndarray, shape (n_samples, n_annotators)

Class labels of the given samples which were provided by the queried annotators. The non queried annotators should return np.nan values.

confidence_scores(self, X, annotator_ids, **kwargs)

Method returning the confidence scores for labelling the given samples.

Parameters:

X: array-like, shape (n_samples, n_features)

Samples whose class labels are queried.

annotator_ids: array-like, shape (n_queried_annotators)

The indices of the annotators whose confidence scores are queried.

Returns:

C: numpy.ndarray, shape (n_samples, n_annotators)

confidence scores of the queried annotators for labelling the given samples. The non queried annotators should return np.nan values.

labelling_performance(self, X, y_true, perf_func=None, **kwargs)

Method computing the labelling performance of each annotator.

Parameters:

X: array-like, shape (n_samples, n_features)

Samples on which the labelling performances of the annotators is evaluated.

y_true: array-like, shape (n_samples)

True class labels of the given samples.

perf_func: callable(y_true, y_pred)

Function evaluating the performance depending on true and predicted class labels. The default function is the accuracy score.

Returns:

perf: list, shape (n_annotators)

Label accuracy of each annotator.

n_annotators(self)

Method returning the number of annotators.

Returns:

n_annotators: int

Number of annotators.

n_queries(self)

Method returning the number of queries posed to an annotator.

Returns:

n_queries: numpy.ndarray, shape (n_annotators)

An entry n_queries[a_idx] indicates how many queries annotator with id a_idx has processed.

plot_class_labels(self, X, features_ids=None, annotator_ids=None, plot_confidences=5, y_true=None, figsize=(5, 3), dpi=150, fontsize=7, **kwargs)

Method creating scatter plots of the given samples for each annotator. In each scatter plot, the samples are colored according the class labels provided by the corresponding annotator.

Parameters:

X: array-like, shape (n_samples, n_features)

Samples which are plotted.

features_ids: array-like, shape (2)

The feature indices to be plotted. The array is limited to two indices.

annotator_ids: array-like, shape (n_queried_annotators)

The indices of the annotators whose class label distributions are plotted.

plot_confidences: boolean

If true, the size of the markers is plotted according to the given confidence scores.

y_true: array-like, shape (n_samples)

This is a optional parameter. If the true class labels are given, the samples are marked according to correctness of their predicted class label

figsize: 2-tuple of floats, default: (5, 3*n_annotators)

Figure dimension (width, height*n_annotators) in inches.

dpi: float, default: 150

Dots per inch.

fontsize: int

Font size of plotted text.

Returns:

fig: matplotlib.axes.Axes object

Created figure.

ax: array-like, shape (n_annotator_ids).

The array ax is a collection of matplotlib.axes.Axes instances representing the plots of the annotators.

plot_labelling_accuracy(self, X, y_true, annotator_ids=None, figsize=(4, 4), dpi=150, fontsize=12)

Method plotting the labelling accuracy of each desired annotator.

Parameters:

X: array-like, shape (n_samples, n_features)

Samples on which the labelling accuracies of the annotators is evaluated.

y_true: array-like, shape (n_samples)

True class labels of the given samples.

annotator_ids: array-like, shape (n_queried_annotators)

The indices of the annotators whose labelling accuracies are plotted.

figsize: 2-tuple of floats, default: (4, 4

Figure dimension (width, height) in inches.

dpi: float, default: 150

Dots per inch.

fontsize: int

Font size of plotted text.

Returns:

fig: matplotlib.figure.Figure object

Created figure.

ax: matplotlib.axes.Axes object

Created axes.

plot_labelling_confusion_matrices(self, X, y_true, y_unique, annotator_ids=None, figsize=(4, 4), dpi=150, fontsize=12)

Method plotting the labelling confusion matrix of each desired annotator.

Parameters:

X: array-like, shape (n_samples, n_features)

Samples on which the labelling confusion matrices of the annotators is evaluated.

y_true: array-like, shape (n_samples)

True class labels of the given samples.

annotator_ids: array-like, shape (n_queried_annotators)

The indices of the annotators whose labelling confusion matrices are plotted.

figsize: 2-tuple of floats, default: (4, 4*n_annotators)

Figure dimension (width, height*n_annotators) in inches.

dpi: float, default: 150

Dots per inch.

fontsize: int

Font size of plotted text.

Returns:

fig: matplotlib.figure.Figure object

Created figure.

ax: array-like, shape (n_annotator_ids).

The array ax is a collection of matplotlib.axes.Axes instances representing the plots of the annotators.

queried_samples(self)

Method returning the samples for which the annotators were queried to provide class labels.

Returns:

X_queried: list, shape (n_annotators, n_queried_samples, n_features)

An entry X_queried[a_idx] represents the samples for which the annotator with id a_idx was queried to provide class labels.