Statistics: Results
Experiment 1
download csv
|
|
|
|
|
|
|
Agreements |
Disagreements |
Kappa |
|
|
Detector |
Projects |
Synthetics |
Misuses |
Potential Hits |
Open Reviews |
Open '?' |
Yes |
No |
Total |
Yes/No |
No/Yes |
Total |
p0 |
pe |
Score |
Hits |
Recall |
DMMC |
26 |
3 |
122 |
62 |
0 |
0 |
13 |
49 |
62 |
0 |
0 |
0 |
1 |
0.166 |
1 |
13 |
10.7% |
GrouMiner |
26 |
3 |
122 |
88 |
0 |
0 |
56 |
29 |
85 |
1 |
2 |
3 |
0.966 |
0.21 |
0.957 |
57 |
46.7% |
Jadet |
26 |
3 |
122 |
23 |
0 |
0 |
14 |
9 |
23 |
0 |
0 |
0 |
1 |
0.238 |
1 |
14 |
11.5% |
MuDetect |
55 |
4 |
171 |
149 |
0 |
0 |
119 |
25 |
144 |
1 |
4 |
5 |
0.966 |
0.134 |
0.961 |
123 |
71.9% |
Tikanga |
26 |
3 |
122 |
15 |
0 |
0 |
3 |
10 |
13 |
0 |
2 |
2 |
0.867 |
0.133 |
0.846 |
3 |
2.5% |
Total |
55 |
4 |
171 |
337 |
0 |
0 |
205 |
122 |
327 |
2 |
8 |
10 |
0.97 |
0.22 |
0.962 |
210 |
24.6% |
Experiment 2
download csv
|
|
|
|
|
Agreements |
Disagreements |
Kappa |
|
|
Detector |
Projects |
Potential Hits |
Open Reviews |
Open '?' |
Yes |
No |
Total |
Yes/No |
No/Yes |
Total |
p0 |
pe |
Score |
Hits |
Precision |
DMMC |
5 |
80 |
40 |
2 |
1 |
53 |
54 |
1 |
0 |
1 |
0.675 |
0.008 |
0.672 |
1 |
1.3% |
GrouMiner |
5 |
90 |
50 |
0 |
0 |
61 |
61 |
1 |
0 |
1 |
0.678 |
0 |
0.678 |
1 |
1.1% |
Jadet |
5 |
52 |
0 |
0 |
2 |
47 |
49 |
2 |
1 |
3 |
0.942 |
0.036 |
0.94 |
4 |
7.7% |
MuDetect |
10 |
146 |
0 |
0 |
29 |
112 |
141 |
2 |
3 |
5 |
0.966 |
0.153 |
0.96 |
32 |
21.9% |
MuDetectXP |
10 |
91 |
26 |
3 |
24 |
45 |
69 |
0 |
5 |
5 |
0.758 |
0.13 |
0.722 |
25 |
27.5% |
Tikanga |
5 |
41 |
0 |
0 |
2 |
33 |
35 |
0 |
6 |
6 |
0.854 |
0.039 |
0.848 |
2 |
4.9% |
Total |
10 |
500 |
116 |
5 |
58 |
351 |
409 |
6 |
15 |
21 |
0.818 |
0.082 |
0.802 |
65 |
13% |
Experiment 3
download csv
|
|
|
|
|
|
Agreements |
Disagreements |
Kappa |
|
|
Detector |
Projects |
Misuses |
Potential Hits |
Open Reviews |
Open '?' |
Yes |
No |
Total |
Yes/No |
No/Yes |
Total |
p0 |
pe |
Score |
Hits |
Recall |
DMMC |
30 |
179 |
42 |
0 |
0 |
13 |
27 |
40 |
0 |
2 |
2 |
0.952 |
0.199 |
0.941 |
13 |
7.3% |
GrouMiner |
29 |
175 |
14 |
0 |
0 |
7 |
7 |
14 |
0 |
0 |
0 |
1 |
0.25 |
1 |
7 |
4% |
Jadet |
29 |
175 |
12 |
0 |
0 |
11 |
0 |
11 |
1 |
0 |
1 |
0.917 |
0 |
0.917 |
12 |
6.9% |
MuDetect |
56 |
226 |
73 |
0 |
0 |
47 |
26 |
73 |
0 |
0 |
0 |
1 |
0.229 |
1 |
47 |
20.8% |
MuDetectXP |
51 |
217 |
152 |
0 |
0 |
92 |
55 |
147 |
2 |
3 |
5 |
0.967 |
0.219 |
0.958 |
95 |
43.8% |
Tikanga |
30 |
179 |
11 |
0 |
0 |
8 |
1 |
9 |
2 |
0 |
2 |
0.818 |
0.066 |
0.805 |
10 |
5.6% |
Total |
56 |
226 |
304 |
0 |
0 |
178 |
116 |
294 |
5 |
5 |
10 |
0.967 |
0.224 |
0.958 |
184 |
13.6% |