Adi Raz Goldfarb [email protected]
commited on
Commit
·
4f3aaec
1
Parent(s):
f6e6df1
update readme
Browse files
README.md
CHANGED
|
@@ -12,7 +12,7 @@ Granite-vision-3.3-2b-embedding is an efficient embedding model based on granite
|
|
| 12 |
By removing the need for OCR-based text extractions, granite-vision-3.3-2b-embedding can help simplify and accelerate RAG pipelines.
|
| 13 |
|
| 14 |
**Evaluations:**
|
| 15 |
-
We evaluated granite-vision-3.3-2b-embedding alongside other top colBERT style multi-modal embedding models in the 1B-4B parameter range using two benchmark: Vidore2 and [Real-MM-RAG-Bench](https://arxiv.org/abs/2502.12342) which aim to specifically address complex multimodal document retrieval tasks.
|
| 16 |
|
| 17 |
## **NDCG@5 - ViDoRe V2**
|
| 18 |
| Collection \ Model | ColPali-v1.3 | ColQwen2.5-v0.2 | ColNomic-3b | ColSmolvlm-v0.1 | granite-vision-3.3-2b-embedding |
|
|
@@ -102,7 +102,7 @@ print(f"📊 Similarity between image and text: {similarity.item():.4f}")
|
|
| 102 |
print("=" * 50)
|
| 103 |
```
|
| 104 |
### Use granite-vision-embedding-3.3-2b for MM RAG
|
| 105 |
-
For an example of MM-RAG using granite-vision-3.3-2b-embedding refer to [this notebook](
|
| 106 |
|
| 107 |
**Model Architecture:**
|
| 108 |
The architecture of granite-vision-3.3-2b-embedding follows ColPali(https://arxiv.org/abs/2407.01449) approach and consists of the following components:
|
|
|
|
| 12 |
By removing the need for OCR-based text extractions, granite-vision-3.3-2b-embedding can help simplify and accelerate RAG pipelines.
|
| 13 |
|
| 14 |
**Evaluations:**
|
| 15 |
+
We evaluated granite-vision-3.3-2b-embedding alongside other top colBERT style multi-modal embedding models in the 1B-4B parameter range using two benchmark: [Vidore2] (https://github.com/illuin-tech/vidore-benchmark/) and [Real-MM-RAG-Bench](https://arxiv.org/abs/2502.12342)([dataset](https://huggingface.co/collections/ibm-research/real-mm-rag-bench-67d2dc0ddf2dfafe66f09d34)) which aim to specifically address complex multimodal document retrieval tasks.
|
| 16 |
|
| 17 |
## **NDCG@5 - ViDoRe V2**
|
| 18 |
| Collection \ Model | ColPali-v1.3 | ColQwen2.5-v0.2 | ColNomic-3b | ColSmolvlm-v0.1 | granite-vision-3.3-2b-embedding |
|
|
|
|
| 102 |
print("=" * 50)
|
| 103 |
```
|
| 104 |
### Use granite-vision-embedding-3.3-2b for MM RAG
|
| 105 |
+
For an example of MM-RAG using granite-vision-3.3-2b-embedding refer to [this notebook](https://github.com/ibm-granite/granite-vision-models/tree/main/cookbooks/GraniteVisionEmbedding_MM-RAG_Notebook).
|
| 106 |
|
| 107 |
**Model Architecture:**
|
| 108 |
The architecture of granite-vision-3.3-2b-embedding follows ColPali(https://arxiv.org/abs/2407.01449) approach and consists of the following components:
|