QServe-benchmarks

# QServe benchmarks This huggingface repository contains configurations and tokenizer files for all models benchmarked in our [QServe](https://github.com/mit-han-lab/qserve) project: - Llama-3-8B - Llama-2-7B - Llama-2-13B - Llama-2-70B - Llama-30B - Mistral-7B - Yi-34B - Qwen1.5-72B Please clone this repository if you wish to run our QServe benchmark code without cloning full models. Please consider citing our paper if it is helpful: ``` @article{lin2024qserve, title={QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving}, author={Lin*, Yujun and Tang*, Haotian and Yang*, Shang and Zhang, Zhekai and Xiao, Guangxuan and Gan, Chuang and Han, Song}, year={2024} } ```# QServe 基准测试集 本Hugging Face仓库包含了我们[QServe](https://github.com/mit-han-lab/qserve)项目中所有参与基准测试的模型的配置文件与分词器文件： - Llama-3-8B - Llama-2-7B - Llama-2-13B - Llama-2-70B - Llama-30B - Mistral-7B - Yi-34B - Qwen1.5-72B 若您希望运行QServe基准测试代码且无需克隆完整模型，请克隆本仓库。 若本项目对您的研究有所帮助，请考虑引用我们的论文： @article{lin2024qserve, title={QServe：面向高效大语言模型（LLM）服务的W4A8KV4量化与系统协同设计}, author={Lin*, Yujun and Tang*, Haotian and Yang*, Shang and Zhang, Zhekai and Xiao, Guangxuan and Gan, Chuang and Han, Song}, year={2024} }

搜集汇总

背景与挑战

背景概述

该数据集提供了QServe项目中基准测试模型的配置和分词器文件，包括Llama、Mistral等多个系列模型。用户可克隆此仓库以运行QServe基准测试代码，无需下载完整模型。