HUME: Measuring the Human-Model Performance Gap in Text Embedding Task
Paper • 2510.10062 • Published • 10
HUME
The HUME benchmark is designed to evaluate the performance of text embedding models and humans on a comparable set of tasks. This captures areas wh...
-
-
mteb/HUMEEmotionClassification
Viewer • Updated • 16k • 40 -
mteb/HUMEToxicConversationsClassification
Viewer • Updated • 8.05k • 36 -
mteb/HUMETweetSentimentExtractionClassification
Viewer • Updated • 27.5k • 33 -
mteb/HUMEMultilingualSentimentClassification
Viewer • Updated • 15.2k • 27 -
mteb/mteb-human-arxiv-clustering
Viewer • Updated • 1 • 23 -
mteb/mteb-human-reddit-clustering
Viewer • Updated • 1 • 25 -
mteb/mteb-human-wikicities-clustering
Viewer • Updated • 1 • 21 -
mteb/mteb-human-sib200-clustering
Viewer • Updated • 10 • 42 -
mteb/HUMECore17InstructionReranking
Viewer • Updated • 320 • 54 -
mteb/HUMENews21InstructionReranking
Viewer • Updated • 496 • 21 -
mteb/HUMERobust04InstructionReranking
Viewer • Updated • 784 • 26 -
mteb/HUMEWikipediaRerankingMultilingual
Viewer • Updated • 1.44k • 37 -
mteb/mteb-human-sickr-sts
Viewer • Updated • 40 • 471 -
mteb/mteb-human-sts12-sts
Viewer • Updated • 50 • 509 -
mteb/mteb-human-stsbenchmark-sts
Viewer • Updated • 50 • 469 -
mteb/mteb-human-sts22-sts
Viewer • Updated • 240 • 133