The Impact of Hyperparameters on Large Language Model Inference Performance: An Evaluation of vLLM and HuggingFace Pipelines
Paper
•
2408.01050
•
Published
•
9
Research • Archive
Long-term archive of papers, models, datasets, and tools worth revisiting. Curated for reference, replication, and future deep dives.
-
-
Agent-as-a-Judge: Evaluate Agents with Agents
Paper • 2410.10934 • Published • 23 -
Agent-SafetyBench: Evaluating the Safety of LLM Agents
Paper • 2412.14470 • Published • 13 -
IntellAgent: A Multi-Agent Framework for Evaluating Conversational AI Systems
Paper • 2501.11067 • Published • 13 -
Seesaw: High-throughput LLM Inference via Model Re-sharding
Paper • 2503.06433 • Published -
PRIMA.CPP: Speeding Up 70B-Scale LLM Inference on Low-Resource Everyday Home Clusters
Paper • 2504.08791 • Published • 139 -
Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Models
Paper • 2412.09645 • Published • 36 -
EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents
Paper • 2501.11858 • Published • 7 -
Evaluation Guidebook
📝267Explore LLM benchmark trends over time
-
Open VLM Leaderboard
🌎985VLMEvalKit Evaluation Results Collection