module: standard

class annotlib.standard.StandardAnnot(X, Y, C=None, confidence_noise=None, random_state=None, probabilistic=False)

Bases: annotlib.base.BaseAnnot

Standard annotators are represented by the class StandardAnnot, which enables to define arbitrary annotators. In a real-world scenario, an annotator is often a human who is asked to provide class labels for samples. An instance of the StandardAnnotators class aims at representing such a human within a Python environment.

Parameters:

X: array-like, shape (n_samples, n_features)

Samples of the whole data set.

Y: array-like, shape (n_samples, n_annotators)

Class labels of the given samples X.

C: array-like, shape (n_samples)

confidence score for labelling the given samples x.

confidence_noise : array-like, shape (n_annotators)

An entry of confidence_noise defines the interval from which the noise is uniformly drawn, e.g. confidence_noise[a] = 0.2 results in sampling n_samples times from U(-0.2, 0.2) and adding this noise to the confidence scores. Zero noise is the default value for each annotator.

random_state: None | int | numpy.random.RandomState

The random state used for generating class labels of the annotators.

Examples

>>> import numpy as np
>>> from sklearn.datasets import load_iris
>>> X, y_true = load_iris(return_X_y=True)
>>> # generate confidence scores
>>> C = np.ones((len(X), 1))
>>> # annotator is always correct
>>> Y = y_true.reshape(-1,1)
>>> annotator = StandardAnnot(X=X, Y=Y, C=C)
>>> # number of annotators
>>> print(annotator.n_annotators())
1
>>> # labelling performance of annotator
>>> print(annotator.labelling_performance(X=X, y_true=y_true))
[1.0]

Attributes:

X_: numpy.ndarray, shape (n_samples, n_features)

Samples of the whole data set.

Y_: numpy.ndarray, shape (n_samples, n_annotators)

Class labels of the given samples X.

C_: numpy.ndarray, shape (n_samples, n_annotators)

confidence score for labelling the given samples x.

C_noise_: numpy.ndarray, shape (n_samples, n_annotators)

The uniformly noise for each annotator and each sample, e.g. C[x_idx, a_idx] indicates the noise for the confidence score of annotator with id a_idx in labelling sample with id x_idx.

n_annotators_: int

Number of annotators.

n_queries_: numpy.ndarray, shape (n_annotators)

An entry n_queries_[a_idx] indicates how many queries annotator with id a_idx has processed.

queried_flags_: numpy.ndarray, shape (n_samples, n_annotators)

An entry queried_flags_[a_idx, x_idx] is a boolean indicating whether annotator with id a_idx has provided a class label for sample with id x_idx.

random_state_: None | int | numpy.random.RandomState

The random state used for generating class labels of the annotators.

class_labels(self, X, annotator_ids=None, query_value=1, **kwargs)

Method returning the class labels of the given samples. If the query value is greater than zero, it updates the n_queries and queried sample statistics

Parameters:

X: array-like, shape (n_samples, n_features)

Samples whose class labels are queried.

annotator_ids: array-like, shape (n_queried_annotators)

The indices of the annotators whose class labels are queried.

query_value: int

The query value represents the increment of the query statistics of the queried annotators.

Returns:

Y: numpy.ndarray, shape (n_samples, n_annotators)

Class labels of the given samples which were provided by the queried annotators. The non queried annotators return np.nan values.

confidence_scores(self, X, annotator_ids=None, **kwargs)

Method returning the confidence scores for labelling the given samples.

Parameters:

X: array-like, shape (n_samples, n_features)

Samples whose class labels are queried.

annotator_ids: array-like, shape (n_queried_annotators)

The indices of the annotators whose confidence scores are queried.

Returns:

C: numpy.ndarray, shape (n_samples, n_annotators)

confidence scores of the queried annotators for labelling the given samples. The non queried annotators should return np.nan values.

n_annotators(self)

Method returning the number of annotators.

Returns:

n_annotators: int

Number of BaseAnnot.

n_queries(self)

Method returning the number of queries posed to an annotator.

Returns:

n_queries_: numpy.ndarray, shape (n_annotators)

An entry n_queries_[a] indicates how many queries annotator a has processed.

queried_samples(self)

Method returning the samples for which the annotators were queried to provide class labels.

Returns:

X_queried: list, shape (n_annotators, n_samples, n_features)

An entry X_queried_[a] represents the samples for which the annotator a was queried to provide class labels.