\section{Introduction}\label{sec:introduction}
\subsection{Motivation}\label{subsec:motivation}
For most supervised learning tasks lots of training samples are essential.
With too less training data the model will not generalize well and not fit a real world task.
Labeling datasets is commonly seen as an expensive task and wants to be avoided as much as possible.
That's why there is a machine-learning field called active learning.
The general approach is to train a model that predicts within every iteration a ranking metric or Pseudo-Labels which then can be used to rank the importance of samples to be labeled.

The goal of this practical work is to test active learning within a simple classification task and evaluate its performance.
\subsection{Research Questions}\label{subsec:research-questions}

\subsubsection{Does Active-Learning benefit the learning process?}

Should Active-learning be used for classification tasks to improve learning performance?
Furthermore, how does the sample-selection process impact the learning?
\subsubsection{Is Dagster and Label-Studio a proper tooling to build an AL Loop?}
Is combining Dagster with Label-Studio a good match for building scalable and reliable Active-Learning loops?
\subsubsection{Does balancing the learning samples improve performance?}
The sample-selection metric might select samples just from one class by chance.
Does balancing this distribution help the model performance?
\subsection{Outline}\label{subsec:outline}

The model is defined as $g(\pmb{x};\pmb{w})$ where $\pmb{w}$ are the model weights and $\mathcal{X}$ the input samples.
We define two hyperparameters, the batch size $\mathcal{B}$ and the sample size $\mathcal{S}$ where $\mathcal{B} < \mathcal{S}$.
In every active learning loop iteration we sample $\mathcal{S}$ random samples~\eqref{eq:batchdef} from our total unlabeled sample set $\mathcal{X}_U \subset \mathcal{X}$.

\begin{equation}
    \label{eq:batchdef}
    \pmb{x} \coloneqq (\pmb{x}_0,\dots,\pmb{x}_\mathcal{S}) \sim \mathcal{X}_U
\end{equation}

The model with the weights of the current loop iteration predicts pseudo predictions.

\begin{equation}\label{eq:equation2}
z = g(\pmb{x};\pmb{w})
\end{equation}

Those predictions might have any numerical value and have to be squeezed into a proper distribution which sums up to 1.
The Softmax function has exactly this effect: $\sum^\mathcal{S}_{i=1}\sigma(z)_i=1$.
Since we have a two class problem the Softmax results in two result values, the two probabilities of how certain one class is a match.
We want to calculate the distance to the class center and the more far away a prediction is from the center the more certain it is.
Vice versa, the more centered the predictions are the more uncertain the prediction is.
Labels $0$ and $1$ result in a class center of $\frac{0+1}{2}=\frac{1}{2}$.
That means taking the absolute value of the prediction minus the class center results in the certainty of the sample~\eqref{eq:certainty}.

\begin{align}
    \label{eq:certainty}
    S(z) = | 0.5 - \sigma(\mathbf{z})_0|  \; \textit{or}  \; S(z) = \max \sigma(\mathbf{z}) - 0.5
\end{align}

\cite{activelearning}

With the help of this metric the pseudo predictions can be sorted by the score $S(z)$.
We define $\text{min}_n(S)$ and $\text{max}_n(S)$ respectively in~\ref{eq:minnot} and~\ref{eq:maxnot} to define a short form of taking a subsection of the minimum or maximum of a set.

\begin{equation}\label{eq:minnot}
\text{min}_n(S) \coloneqq a \subset S \mid \text{where } a \text{ are the } n \text{ smallest numbers of } S
\end{equation}

\begin{equation}\label{eq:maxnot}
\text{max}_n(S) \coloneqq a \subset S \mid \text{where } a \text{ are the } n \text{ largest numbers of } S
\end{equation}

This notation helps to define which subsets of samples to give the user for labeling.
There are different ways how this subset can be chosen.
In this PW we do the obvious experiments with High-Certainty first~\ref{subsec:low-certainty-first}, Low-Certainty first~\ref{subsec:high-certainty-first}.
Furthermore, the two mixtures between them, half-high and half-low certain and only the middle section of the sorted certainty scores.

\subsubsection{Low certainty first}\label{subsec:low-certainty-first}
We take the samples with the lowest certainty score first and give it to the user for labeling.
This is the most intuitive way to do active learning and might also be the most beneficial.

\begin{equation}
    \mathcal{X}_t  = \text{min}_\mathcal{B}(S(z))
\end{equation}

\subsubsection{High certainty first}\label{subsec:high-certainty-first}
We take the samples with the highest certainty score first and give it to the user for labeling.
The idea behind this is that the model is already very certain about the prediction and the user can confirm this.
This might help ignoring labels which are irrelevant for the model.

\begin{equation}
    \mathcal{X}_t  =\text{max}_\mathcal{B}(S(z))
\end{equation}

\subsubsection{Low and High certain first}

We take half the batch-size $\mathcal{B}$ of low certainty and the other half with high certainty samples.
Benefit from both, low and high certainty samples.

\begin{equation}
    \mathcal{X}_t  =\text{max}_{\mathcal{B}/2}(S(z)) \cup  \text{max}_{\mathcal{B}/2}(S(z))
\end{equation}

\subsubsection{Mid certain first}

We take the middle section of the certainty scores.
To close the gap and have also the fourth variation included in the experiments.
This is expected to perform the worst but might still be better than random sampling in some cases.

\begin{equation}
    \mathcal{X}_t  =S(z) \setminus (\text{min}_{\mathcal{S}/2 - \mathcal{B}/2}(S(z)) \cup  \text{max}_{\mathcal{S}/2 - \mathcal{B}/2}(S(z)))
\end{equation}

\subsubsection{Model training}
So now we have defined the samples we want to label with $\mathcal{X}_t$ and the user starts labeling this samples.
After labelling the model $g(\pmb{x};\pmb{w})$ is trained with the new samples and the weights $\pmb{w}$ are updated with the labeled samples $\mathcal{X}_t$.
The loop starts again with the new model and draws new unlabeled samples from $\mathcal{X}_U$.

\subsubsection{Further improvement by class balancing}
An intuitive improvement step might be the balancing of the class predictions.
The selected samples of the active learning step above from $\mathcal{X}_t$ might all be from one class.
This is bad for the learning process because the model overfits to one class if always the same class is selected.

Since nobody knows the true label during the sample selection process we cannot just sort by the true label and balance the samples.
The simplest solution to this is using the models predicted class and balance the selection by using half of the samples from one predicted class and the other one from the other.
Afterwards apply the selected scoring metric from above to do uncertainty sampling or similar to the balanced selection.

This process can be shown mathematically for low certainty sampling as in~\eqref{eq:balancedlowcertainty}.

\begin{equation}\label{eq:balancedlowcertainty}
    \mathcal{X}_t  = \text{min}_{\mathcal{B}/2}(\left\{\alpha \in S(z) : \alpha_0 < 0.5\right\}) \cup \text{min}_{\mathcal{B}/2}(\left\{\alpha \in S(z) : \alpha_1 < 0.5\right\})
\end{equation}