add result images for each method
	
		
			
	
		
	
	
		
	
		
			All checks were successful
		
		
	
	
		
			
				
	
				Build Typst document / build_typst_documents (push) Successful in 1m1s
				
			
		
		
	
	
				
					
				
			@@ -1,6 +1,7 @@
 | 
				
			|||||||
#import "@preview/fletcher:0.5.3" as fletcher: diagram, node, edge
 | 
					#import "@preview/fletcher:0.5.3" as fletcher: diagram, node, edge
 | 
				
			||||||
#import fletcher.shapes: rect, diamond
 | 
					#import fletcher.shapes: rect, diamond
 | 
				
			||||||
#import "utils.typ": todo
 | 
					#import "utils.typ": todo
 | 
				
			||||||
 | 
					#import "@preview/subpar:0.1.1"
 | 
				
			||||||
 | 
					
 | 
				
			||||||
= Implementation
 | 
					= Implementation
 | 
				
			||||||
The three methods described (ResNet50, CAML, P>M>F) were implemented in a Jupyter notebook and compared to each other.
 | 
					The three methods described (ResNet50, CAML, P>M>F) were implemented in a Jupyter notebook and compared to each other.
 | 
				
			||||||
@@ -8,16 +9,19 @@ The three methods described (ResNet50, CAML, P>M>F) were implemented in a Jupyte
 | 
				
			|||||||
== Experiments
 | 
					== Experiments
 | 
				
			||||||
For all of the three methods we test the following use-cases:#todo[maybe write more to each test]
 | 
					For all of the three methods we test the following use-cases:#todo[maybe write more to each test]
 | 
				
			||||||
- Detection of anomaly class (1,3,5 shots)
 | 
					- Detection of anomaly class (1,3,5 shots)
 | 
				
			||||||
- Inbalanced target class prediction (5,10,15,30 good shots, 5 bad shots)
 | 
					 | 
				
			||||||
- 2 Way classification (1,3,5 shots)
 | 
					- 2 Way classification (1,3,5 shots)
 | 
				
			||||||
- Inbalanced 2 Way classification (5,10,15,30 good shots, 5 bad shots)
 | 
					 | 
				
			||||||
- Detect only anomaly classes (1,3,5 shots)
 | 
					- Detect only anomaly classes (1,3,5 shots)
 | 
				
			||||||
 | 
					- Inbalanced 2 Way classification (5,10,15,30 good shots, 5 bad shots)
 | 
				
			||||||
 | 
					- Inbalanced target class prediction (5,10,15,30 good shots, 5 bad shots)
 | 
				
			||||||
 | 
					
 | 
				
			||||||
Those experiments were conducted on the MVTEC AD dataset on the bottle and cable classes.
 | 
					Those experiments were conducted on the MVTEC AD dataset on the bottle and cable classes.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					
 | 
				
			||||||
== Experiment Setup
 | 
					== Experiment Setup
 | 
				
			||||||
#todo[Setup of experiments, which classes used, nr of samples]
 | 
					All the experiments were done on the bottle and cable classes of the MVTEC AD dataset.
 | 
				
			||||||
 | 
					The correspoinding number of shots were randomly selected from the dataset.
 | 
				
			||||||
 | 
					The rest of the images were used to test the model and measure the accuracy.
 | 
				
			||||||
 | 
					#todo[Maybe add real number of samples per classes]
 | 
				
			||||||
 | 
					
 | 
				
			||||||
== ResNet50
 | 
					== ResNet50
 | 
				
			||||||
=== Approach
 | 
					=== Approach
 | 
				
			||||||
@@ -83,13 +87,78 @@ After creating the embeddings for the support and query set the euclidean distan
 | 
				
			|||||||
The class with the smallest distance is chosen as the predicted class.
 | 
					The class with the smallest distance is chosen as the predicted class.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
=== Results
 | 
					=== Results
 | 
				
			||||||
This method perofrmed better than expected wich such a simple method.
 | 
					This method performed better than expected wich such a simple method.
 | 
				
			||||||
 | 
					As in @resnet50bottleperfa with a normal 5 shot / 4 way classification the model achieved an accuracy of 75%.
 | 
				
			||||||
 | 
					When detecting only if there occured an anomaly or not the performance is significantly better and peaks at 81% with 5 shots / 2 ways.
 | 
				
			||||||
 | 
					Interestintly the model performed slightly better with fewer shots in this case.
 | 
				
			||||||
 | 
					Moreover in @resnet50bottleperfa, the detection of the anomaly class only (3 way) shows a similar pattern as the normal 4 way classification.
 | 
				
			||||||
 | 
					The more shots the better the performance and it peaks at around 88% accuracy with 5 shots.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
#todo[Add images of graphs with ResNet50 stuff only]
 | 
					In @resnet50bottleperfb the model was tested with inbalanced class distributions.
 | 
				
			||||||
 | 
					With [5,10,15,30] good shots and 5 bad shots the model performed worse than with balanced classes.
 | 
				
			||||||
 | 
					The more good shots the worse the performance.
 | 
				
			||||||
 | 
					The only exception is the faulty or not detection (2 way) where the model peaked at 15 good shots with 83% accuracy.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					#subpar.grid(
 | 
				
			||||||
 | 
					  figure(image("rsc/resnet/ResNet50-bottle.png"), caption: [
 | 
				
			||||||
 | 
					    Normal [1,3,5] shots
 | 
				
			||||||
 | 
					  ]), <resnet50bottleperfa>,
 | 
				
			||||||
 | 
					  figure(image("rsc/resnet/ResNet50-bottle-inbalanced.png"), caption: [
 | 
				
			||||||
 | 
					    Inbalanced [5,10,15,30] shots
 | 
				
			||||||
 | 
					  ]), <resnet50bottleperfb>,
 | 
				
			||||||
 | 
					  columns: (1fr, 1fr),
 | 
				
			||||||
 | 
					  caption: [ResNet50 performance on bottle class],
 | 
				
			||||||
 | 
					  label: <resnet50bottleperf>,
 | 
				
			||||||
 | 
					)
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					The same experiments were conducted on the cable class and the results are shown in @resnet50cableperfa and @resnet50cableperfb.
 | 
				
			||||||
 | 
					The results are very similar to the bottle class.
 | 
				
			||||||
 | 
					Generally the more shots the better the accuracy.
 | 
				
			||||||
 | 
					But the overall reached max accuracy is lower than on the bottle class,
 | 
				
			||||||
 | 
					but this is expected as the cable class consists of 8 faulty classes.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					#subpar.grid(
 | 
				
			||||||
 | 
					  figure(image("rsc/resnet/ResNet50-cable.png"), caption: [
 | 
				
			||||||
 | 
					    Normal [1,3,5] shots
 | 
				
			||||||
 | 
					  ]), <resnet50cableperfa>,
 | 
				
			||||||
 | 
					  figure(image("rsc/resnet/ResNet50-cable-inbalanced.png"), caption: [
 | 
				
			||||||
 | 
					    Inbalanced [5,10,15,30] shots
 | 
				
			||||||
 | 
					  ]), <resnet50cableperfb>,
 | 
				
			||||||
 | 
					  columns: (1fr, 1fr),
 | 
				
			||||||
 | 
					  caption: [ResNet50 performance on cable class],
 | 
				
			||||||
 | 
					  label: <resnet50cableperf>,
 | 
				
			||||||
 | 
					)
 | 
				
			||||||
 | 
					
 | 
				
			||||||
== P>M>F
 | 
					== P>M>F
 | 
				
			||||||
=== Approach
 | 
					=== Approach
 | 
				
			||||||
=== Results
 | 
					=== Results
 | 
				
			||||||
 | 
					The results of P>M>F look very promising and improve by a large margin over the ResNet50 method.
 | 
				
			||||||
 | 
					In @pmfbottleperfa the model reached an accuracy of 79% with 5 shots / 4 way classification.
 | 
				
			||||||
 | 
					#todo[write bit more here]
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					#subpar.grid(
 | 
				
			||||||
 | 
					  figure(image("rsc/pmf/P>M>F-bottle.png"), caption: [
 | 
				
			||||||
 | 
					    Normal [1,3,5] shots
 | 
				
			||||||
 | 
					  ]), <pmfbottleperfa>,
 | 
				
			||||||
 | 
					  figure(image("rsc/pmf/P>M>F-bottle-inbalanced.png"), caption: [
 | 
				
			||||||
 | 
					    Inbalanced [5,10,15,30] shots
 | 
				
			||||||
 | 
					  ]), <pmfbottleperfb>,
 | 
				
			||||||
 | 
					  columns: (1fr, 1fr),
 | 
				
			||||||
 | 
					  caption: [P>M>F performance on bottle class],
 | 
				
			||||||
 | 
					  label: <pmfbottleperf>,
 | 
				
			||||||
 | 
					)
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					#subpar.grid(
 | 
				
			||||||
 | 
					  figure(image("rsc/pmf/P>M>F-cable.png"), caption: [
 | 
				
			||||||
 | 
					    Normal [1,3,5] shots
 | 
				
			||||||
 | 
					  ]), <pmfcableperfa>,
 | 
				
			||||||
 | 
					  figure(image("rsc/pmf/P>M>F-cable-inbalanced.png"), caption: [
 | 
				
			||||||
 | 
					    Inbalanced [5,10,15,30] shots
 | 
				
			||||||
 | 
					  ]), <pmfcableperfb>,
 | 
				
			||||||
 | 
					  columns: (1fr, 1fr),
 | 
				
			||||||
 | 
					  caption: [P>M>F performance on cable class],
 | 
				
			||||||
 | 
					  label: <pmfcableperf>,
 | 
				
			||||||
 | 
					)
 | 
				
			||||||
 | 
					
 | 
				
			||||||
== CAML
 | 
					== CAML
 | 
				
			||||||
=== Approach
 | 
					=== Approach
 | 
				
			||||||
@@ -106,4 +175,18 @@ This might be caused by the fact that the model was not fine-tuned for any indus
 | 
				
			|||||||
The model was trained on a large number of general purpose images and is not fine-tuned at all.
 | 
					The model was trained on a large number of general purpose images and is not fine-tuned at all.
 | 
				
			||||||
It might not handle very similar images well.
 | 
					It might not handle very similar images well.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
#todo[Add images of graphs with CAML stuff only]
 | 
					Compared the the other two methods CAML performed poorly in almost all experiments.
 | 
				
			||||||
 | 
					The normal few-shot classification reached only 40% accuracy in @camlperfa at best.
 | 
				
			||||||
 | 
					The only test it did surprisingly well was the detection of the anomaly class for the cable class in @camlperfb were it reached almost 60% accuracy.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					#subpar.grid(
 | 
				
			||||||
 | 
					  figure(image("rsc/caml/CAML-bottle.png"), caption: [
 | 
				
			||||||
 | 
					    Normal [1,3,5] shots - Bottle
 | 
				
			||||||
 | 
					  ]), <camlperfa>,
 | 
				
			||||||
 | 
					  figure(image("rsc/caml/CAML-cable.png"), caption: [
 | 
				
			||||||
 | 
					    Normal [1,3,5] shots - Cable
 | 
				
			||||||
 | 
					  ]), <camlperfb>,
 | 
				
			||||||
 | 
					  columns: (1fr, 1fr),
 | 
				
			||||||
 | 
					  caption: [CAML performance],
 | 
				
			||||||
 | 
					  label: <camlperf>,
 | 
				
			||||||
 | 
					)
 | 
				
			||||||
 
 | 
				
			|||||||
							
								
								
									
										
											BIN
										
									
								
								rsc/caml/CAML-bottle.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 32 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								rsc/caml/CAML-cable.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 36 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								rsc/pmf/P>M>F-bottle-inbalanced.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 34 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								rsc/pmf/P>M>F-bottle.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 38 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								rsc/pmf/P>M>F-cable-inbalanced.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 31 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								rsc/pmf/P>M>F-cable.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 35 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								rsc/resnet/ResNet50-bottle-inbalanced.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 35 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								rsc/resnet/ResNet50-bottle.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 37 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								rsc/resnet/ResNet50-cable-inbalanced.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 28 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								rsc/resnet/ResNet50-cable.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						| 
		 After Width: | Height: | Size: 34 KiB  |