Survey of Active Learning Hyperparameters

Type of the data
datacite.resourceTypeGeneral

Dataset

Total size of the dataset
datacite.size

344254188472

Author
dc.contributor.author

Gonsior, Julius

Upload date
dc.date.accessioned

2025-06-02T12:10:20Z

Publication date
dc.date.available

2025-06-02T12:10:20Z

Publication date
dc.date.issued

2025-06-02

Abstract of the dataset
dc.description.abstract

Annotating data is a time-consuming and costly task, but it is inherently required for supervised machine learning. Active Learning (AL) is an established method that minimizes human labeling effort by iteratively selecting the most informative unlabeled samples for expert annotation, thereby improving the overall classification performance. Even though AL has been known for decades [1], AL is still rarely used in real-world applications. As indicated in the two community web surveys among the NLP community about AL [2], [3], two main reasons continue to hold practitioners back from using AL: first, the complexity of setting AL up, and second, a lack of trust in its effectiveness. We hypothesize that both reasons share the same culprit: the large hyperparameter space of AL. This mostly unexplored hyperparameter space often leads to misleading and irreproducible AL experiment results. In this study, we first compiled a large hyperparameter grid of over 4.6 million hyperparameter combinations, second, recorded the performance of all combinations in the so-far biggest conducted AL study, and third, analyzed the impact of each hyperparameter in the experiment results. Besides our publication we are making our raw experiment results publically available for other researchers to built upon. [1] B. Settles, “Active Learning Literature Survey,” University of Wisconsin-Madison Department of Computer Sciences, Technical Report, 2009. [2] K. Tomanek and F. Olsson, “A web survey on the use of active learning to support annotation of text data,” in Proceedings of the NAACL HLT 2009 Workshop on Active Learning for Natural Language Processing, E. Ringger, R. Haertel, and K. Tomanek, Eds. Boulder, Colorado: Association for Computational Linguistics, 2009, pp. 45–48. [3] J. Romberg, C. Schröder, J. Gonsior, K. Tomanek, and F. Olsson, “Have LLMs Made Active Learning Obsolete? Surveying the NLP Community,” 2025.

Public reference to this page
dc.identifier.uri

https://opara.zih.tu-dresden.de/handle/123456789/1501

Public reference to this page
dc.identifier.uri

https://doi.org/10.25532/OPARA-862

Publisher
dc.publisher

Technische Universität Dresden

Licence
dc.rights

Attribution-NonCommercial 4.0 Internationalen

URI of the licence text
dc.rights.uri

http://creativecommons.org/licenses/by-nc/4.0/

Specification of the discipline(s)
dc.subject.classification

4::44::409::409-06

Title of the dataset
dc.title

Survey of Active Learning Hyperparameters
Files
Original bundle
Now showing 1 - 2 of 2
Loading...
Thumbnail Image
Name:
archive_listing.txt
Size:
183.88 MB
Format:
Plain Text
Description:
Loading...
Thumbnail Image
Name:
full_exp_jan.zip
Size:
320.43 GB
Format:
Unknown data format
Description:
License bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
license.txt
Size:
4.66 KB
Format:
Item-specific license agreed to upon submission
Description:
Attribution-NonCommercial 4.0 International