版本发布 5
Draft a new release, for internal evaluation use.
## What's Changed * [Feature] Support multi_generate by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/1 * [Tool] Minor updates 1205 by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/3 * TranCore-M 20231208 by @PCIResearch in https://github.com/open-compass/VLMEvalKit/pull/8 * [Result] update TransCore Results by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/9 * COREMM Evaluation Benchmark by @youngfly11 in https://github.com/open-compass/VLMEvalKit/pull/6 * Adding MMVet by @llllIlllll in https://github.com/open-compass/VLMEvalKit/pull/7 * [Doc] README Update 12.11 by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/10 * Updating mmvet_eval by @llllIlllll in https://github.com/open-compass/VLMEvalKit/pull/11 * [Feature] Add run.py and simplify the evaluation. by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/12 * [Doc] Refine README by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/13 * [Fix] Fix README by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/14 * add dataset md5 check integrity by @FangXinyu-0913 in https://github.com/open-compass/VLMEvalKit/pull/15 * [Feature] Support two Vision APIs by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/19 * Add COCO dataset by @FangXinyu-0913 in https://github.com/open-compass/VLMEvalKit/pull/16 * [Fix] Fix 1221 by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/21 * [Feature] More Robust API Evaluation by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/22 * [Refactor] Refactor Custom Prompt & Fix mPLUG-Owl2 acc by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/23 * [Dataset] VQA Datasets by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/25 * [Fix] Bug Fix by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/26 * Add MMMU dataset by @llllIlllll in https://github.com/open-compass/VLMEvalKit/pull/18 * Add QwenVLPlus API by @llllIlllll in https://github.com/open-compass/VLMEvalKit/pull/27 * [Result] Update MMMU Acc by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/30 * [Feature] Support `LLaVA_XTuner` models by @LZHgrla in https://github.com/open-compass/VLMEvalKit/pull/17 * [Result] Update XTuner Performance by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/31 * [Result] Update COCO Caption Results by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/35 * add dataset ChartQA by @FangXinyu-0913 in https://github.com/open-compass/VLMEvalKit/pull/28 * [Feature]: Add ScienceQA by @YuanLiuuuuuu in https://github.com/open-compass/VLMEvalKit/pull/24 * [Dataset] MathVista dataset by @llllIlllll in https://github.com/open-compass/VLMEvalKit/pull/29 * [Dataset] HallusionBench by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/38 * Add sharedcaptioner and cogvlm by @fitzpchao in https://github.com/open-compass/VLMEvalKit/pull/37 * [Fix] Fix GPT error with parallel calling by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/40 * [Eval] multiple_choice.py: E->Z by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/41 * [Eval] Use exact matching for Y/N and multi-choice when OPENAI_API_KEY not set by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/44 * add VLM: monkey by @ShuoZhang2003 in https://github.com/open-compass/VLMEvalKit/pull/45 * [Fix] Fix multiple choice evaluation when OPENAI_API_KEY missing by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/48 * [Improvement] Support non-contiguous choices by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/49 * Add Emu2 and Emu2_chat by @llllIlllll in https://github.com/open-compass/VLMEvalKit/pull/47 * [Dataset] Add DocVQA by @llllIlllll in https://github.com/open-compass/VLMEvalKit/pull/50 * [Benchmark] support AI2D by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/51 * add monkey-chat by @ShuoZhang2003 in https://github.com/open-compass/VLMEvalKit/pull/54 * Add `LLaVA-InternLM2` by @LZHgrla in https://github.com/open-compass/VLMEvalKit/pull/53 * [Dataset] Support LLaVABench by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/55 * Support torchrun for emu2&emu2_chat and fix bug by @llllIlllll in https://github.com/open-compass/VLMEvalKit/pull/52 * support sharegpt4v-13b by @xiaoachen98 in https://github.com/open-compass/VLMEvalKit/pull/56 * Fix bug in file.py by @Ezra-Yu in https://github.com/open-compass/VLMEvalKit/pull/58 * [Result] Update Evaluation Results by @kennymckormick in https://github.com/open-compass/VLMEvalKit/pull/60 ## New Contributors * @kennymckormick made their first contribution in https://github.com/open-compass/VLMEvalKit/pull/1 * @PCIResearch made their first contribution in https://github.com/open-compass/VLMEvalKit/pull/8 * @youngfly11 made their first contribution in https://github.com/open-compass/VLMEvalKit/pull/6 * @llllIlllll made their first contribution in https://github.com/open-compass/VLMEvalKit/pull/7 * @FangXinyu-0913 made their first contribution in https://github.com/open-compass/VLMEvalKit/pull/15 * @LZHgrla made their first contribution in https://github.com/open-compass/VLMEvalKit/pull/17 * @YuanLiuuuuuu made their first contribution in https://github.com/open-compass/VLMEvalKit/pull/24 * @fitzpchao made their first contribution in https://github.com/open-compass/VLMEvalKit/pull/37 * @ShuoZhang2003 made their first contribution in https://github.com/open-compass/VLMEvalKit/pull/45 * @xiaoachen98 made their first contribution in https://github.com/open-compass/VLMEvalKit/pull/56 * @Ezra-Yu made their first contribution in https://github.com/open-compass/VLMEvalKit/pull/58 **Full Changelog**: https://github.com/open-compass/VLMEvalKit/commits/v0.1