…nto mieb

* make lint
* wip
* add TinyImageNet and run
* type hints
* add accuracy
* lint

* add ImageTextPairClassification abstask and evaluator

* dataset transform into sequence of images for each sample

* fix processing logic; list of list images compatability

* lint and docstrings

* make lint

* fix failing tests in TaskMetadata

* add tests for mieb

* skip gated repo

---------

Co-authored-by: gowitheflow-1998 <jsbs54@durham.ac.uk>

…1101)

* fix task metadata

* use overrideable column names

* add CIFAR datasets

* add caltech101 dataset

* add FGVC aircraft dataset

* add food 101 dataset

* add OxfordPets dataset

* remove comments

* correct cifar100 path

* update cifar100 classification results

* cifar zero shot results

* add caltech101 zero shot

* matching CLIP paper implementation

* add aircraft and food zero shot

* add oxford pets zero shot

add CIFAR clustering

…asets (#1103)

* update category to i2t

* add MNIST linear probe and zero shot

* add FER2013 linear probe and zero shot

* add stanford cars linear probe and zero shot

* add birdsnap linear probe and zero shot

* add eurosat linear probe and zero shot

* lint

* add STL10 linear probe and zero shot

* add RESISC45 linear probe and zeor shot

* add Describable textures linear probe and zero shot

* fix spacing lint

* add SUN397 linear probe and zero shot

* Add Country211 classification task

* Add imagenet1k classification task

* Add UCF101 classification task

* Add PatchCamelyon Classification task

* Add GTSRB classification task

* Add GSTRB Zero Shot Classification

* Add country211 zero shot classification

* Add results for classification tasks

* Add zero shot classification tasks

* Add PatchCamelyon tasks and results

* Add linting

* Add results and fix prompts for zero shot

* Add results

* Add results and linting

* add jina clip and mscoco i2t and t2i results

* make lint

* fix instruction retrival (#1072)

* fix instruction retrival

* fix test

* add points

* make nested results

* add test

* skip instruction test

* fix instruction passes

* fix unions

* move do_length_ablation

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>

---------

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>

* Update points table

* fix: fix bug-causing spelling error in function name of e5-mistral-instruct (#1106)

found bug

* 1.12.85

Automatically generated by python-semantic-release

* fix: MultilingualSentimentClassification (#1109)

* Update points table

* fix: Avoid spaces in dataset name for CQADupstack and ignore speed tasks

* 1.12.86

Automatically generated by python-semantic-release

* fix: Ensure that MLSUMClusteringP2P.v2 use the fast implementation as was intended (#1112)

* fix: Ensure that MLSUMClusteringP2P.v2 use the fast implementation as was intended

* fix: fixed formatting for cli

* docs: improve searchability in the advanced usage documentation

* 1.12.87

Automatically generated by python-semantic-release

* docs: improve searchability in the advanced usage documentation (#1113)

* docs: improve searchability in the advanced usage documentation

* docs: update based on corrections

* fix: export type for `mteb create_meta` (#1114)

* fix export type

* fix dataset version too

* 1.12.88

Automatically generated by python-semantic-release

* fix: Simplify models implementations (#1085)

* Merge

* Adapt

* Simplify

* Check for rev again

* Rmv cmmnt

* Simplify

* simplify

* Rmv comment

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>

* Use logging; change try except; add info

* Lint

* Rmv results

* Update rev

* format

* Simplify models; Allow instructions

* Jobs

* Fix merge

* Format

* Adapt models

* fix: ensure that e5 ignores the NQ

* format

---------

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>

* 1.12.89

Automatically generated by python-semantic-release

* fix: nomic models using prefix correctly (#1125)

* fix: nomic models using prefix correctly

* chore: remove comment

* fix: handling in case not torch tensor

* Fix typo

---------

Co-authored-by: Niklas Muennighoff <n.muennighoff@gmail.com>

* 1.12.90

Automatically generated by python-semantic-release

* refactor vista model wrapper to contain lib import

* python 38 type hints

---------

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com>
Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: anpalmak2003 <73543260+anpalmak2003@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Niklas Muennighoff <n.muennighoff@gmail.com>
Co-authored-by: Zach Nussbaum <zanussbaum@gmail.com>
Co-authored-by: chenghao xiao <85804993+gowitheflow-1998@users.noreply.github.com>

* Add CLEVER and SciMMIR

* Update metadata

* remove useless comment

* Add linting

* fix typo and tests

* Add CLEVR count task

* add linting

* wip

* add results

* make lint

* change back the order

* add cub200 and results

* add skip_first_result

* skipped self and rerun results