2024-12-30 18:34:43 +01:00
|
|
|
#import "@preview/fletcher:0.5.3" as fletcher: diagram, node, edge
|
|
|
|
#import fletcher.shapes: rect, diamond
|
|
|
|
#import "utils.typ": todo
|
|
|
|
|
2024-10-28 16:02:53 +01:00
|
|
|
= Implementation
|
2024-12-30 18:34:43 +01:00
|
|
|
The three methods described (ResNet50, CAML, P>M>F) were implemented in a Jupyter notebook and compared to each other.
|
|
|
|
|
|
|
|
== Experiments
|
|
|
|
For all of the three methods we test the following use-cases:#todo[maybe write more to each test]
|
|
|
|
- Detection of anomaly class (1,3,5 shots)
|
|
|
|
- Inbalanced target class prediction (5,10,15,30 good shots, 5 bad shots)
|
|
|
|
- 2 Way classification (1,3,5 shots)
|
|
|
|
- Inbalanced 2 Way classification (5,10,15,30 good shots, 5 bad shots)
|
|
|
|
- Detect only anomaly classes (1,3,5 shots)
|
|
|
|
|
|
|
|
Those experiments were conducted on the MVTEC AD dataset on the bottle and cable classes.
|
|
|
|
|
|
|
|
== ResNet50
|
|
|
|
=== Approach
|
|
|
|
The simplest approach is to use a pre-trained ResNet50 model as a feature extractor.
|
|
|
|
From both the support and query set the features are extracted to get a downprojected representation of the images.
|
|
|
|
The support set embeddings are compared to the query set embeddings.
|
|
|
|
To predict the class of a query the class with the smallest distance to the support embedding is chosen.
|
|
|
|
If there are more than one support embedding within the same class the mean of those embeddings is used (class center).
|
|
|
|
This approach is similar to a prototypical network @snell2017prototypicalnetworksfewshotlearning.
|
|
|
|
|
|
|
|
In this bachelor thesis a pre-trained ResNet50 (IMAGENET1K_V2) pytorch model was used.
|
|
|
|
It is pretrained on the imagenet dataset and has 50 residual layers.
|
|
|
|
|
|
|
|
To get the embeddings the last layer of the model was removed and the output of the second last layer was used as embedding output.
|
|
|
|
In the following diagram the ResNet50 architecture is visualized and the cut-point is marked.
|
|
|
|
|
|
|
|
#diagram(
|
|
|
|
spacing: (5mm, 5mm),
|
|
|
|
node-stroke: 1pt,
|
|
|
|
node-fill: eastern,
|
|
|
|
edge-stroke: 1pt,
|
|
|
|
|
|
|
|
// Input
|
|
|
|
node((1, 1), "Input", shape: rect, width: 30mm, height: 10mm, name: <input>),
|
|
|
|
|
|
|
|
// Conv1
|
|
|
|
node((1, 0), "Conv1\n7x7, 64", shape: rect, width: 30mm, height: 15mm, name: <conv1>),
|
|
|
|
edge(<input>, <conv1>, "->"),
|
|
|
|
|
|
|
|
// MaxPool
|
|
|
|
node((1, -1), "MaxPool\n3x3", shape: rect, width: 30mm, height: 15mm, name: <maxpool>),
|
|
|
|
edge(<conv1>, <maxpool>, "->"),
|
|
|
|
|
|
|
|
// Residual Blocks
|
|
|
|
node((3, -1), "Residual Block 1\n3x [64, 64, 256]", shape: rect, width: 40mm, height: 15mm, name: <res1>),
|
|
|
|
edge(<maxpool>, <res1>, "->"),
|
|
|
|
|
|
|
|
node((3, 0), "Residual Block 2\n4x [128, 128, 512]", shape: rect, width: 40mm, height: 15mm, name: <res2>),
|
|
|
|
edge(<res1>, <res2>, "->"),
|
|
|
|
|
|
|
|
node((3, 1), "Residual Block 3\n6x [256, 256, 1024]", shape: rect, width: 40mm, height: 15mm, name: <res3>),
|
|
|
|
edge(<res2>, <res3>, "->"),
|
|
|
|
|
|
|
|
node((3, 2), "Residual Block 4\n3x [512, 512, 2048]", shape: rect, width: 40mm, height: 15mm, name: <res4>),
|
|
|
|
edge(<res3>, <res4>, "->"),
|
|
|
|
|
|
|
|
// Cutting Line
|
|
|
|
edge(<res4>, <avgpool>, marks: "..|..>", stroke: 1pt, label: "Cut here", label-pos: 0.5, label-side: left),
|
|
|
|
|
|
|
|
// AvgPool + FC
|
|
|
|
node((7, 2), "AvgPool\n1x1", shape: rect, width: 30mm, height: 10mm, name: <avgpool>),
|
|
|
|
//edge(<res4>, <avgpool>, "->"),
|
|
|
|
|
|
|
|
node((7, 1), "Fully Connected\n1000 classes", shape: rect, width: 40mm, height: 10mm, name: <fc>),
|
|
|
|
edge(<avgpool>, <fc>, "->"),
|
|
|
|
|
|
|
|
// Output
|
|
|
|
node((7, 0), "Output", shape: rect, width: 30mm, height: 10mm, name: <output>),
|
|
|
|
edge(<fc>, <output>, "->")
|
|
|
|
)
|
|
|
|
|
|
|
|
After creating the embeddings for the support and query set the euclidean distance is calculated.
|
|
|
|
The class with the smallest distance is chosen as the predicted class.
|
|
|
|
|
|
|
|
=== Results
|
|
|
|
|
|
|
|
|
|
|
|
== CAML
|
|
|
|
|
|
|
|
== P>M>F
|
2024-10-28 16:02:53 +01:00
|
|
|
|
|
|
|
== Experiment Setup
|
|
|
|
% todo
|
|
|
|
todo setup of experiments, which classes used, nr of samples
|
|
|
|
kinds of experiments which lead to graphs
|
|
|
|
|
|
|
|
== Jupyter
|
|
|
|
|
|
|
|
To get accurate performance measures the active-learning process was implemented in a Jupyter notebook first.
|
|
|
|
This helps to choose which of the methods performs the best and which one to use in the final Dagster pipeline.
|
|
|
|
A straight forward machine-learning pipeline was implemented with the help of Pytorch and RESNet-18.
|
|
|
|
|
|
|
|
Moreover, the Dataset was manually imported with the help of a custom torch dataloader and preprocessed with random augmentations.
|
|
|
|
After each loop iteration the Area Under the Curve (AUC) was calculated over the validation set to get a performance measure.
|
|
|
|
All those AUC were visualized in a line plot, see section~\ref{sec:experimental-results} for the results.
|