def vit_giant2(patch_size=16, **kwargs):
    """Close to ViT-giant, with embed-dim 1536 and 24 heads => embed-dim per head 64."""
    model = DinoVisionTransformer(
        patch_size=patch_size,
        embed_dim=1536,
        depth=40,
        num_heads=24,
        mlp_ratio=4,
        block_fn=partial(Block, attn_class=MemEffAttention),
        **kwargs,
    )
    return model
Close to ViT-giant, with embed-dim 1536 and 24 heads => embed-dim per head 64.
kornia/feature/dedode/transformer/dinov2.py