数据集以 Google Sheets 形式发布，需复制到自己的 Google Drive 并另存新文件名；FreshEval 依赖 LLM 做评判，需自行准备可调用的模型；README 未给出本地安装命令，主要走 Colab 与表格流程
freshqa
freshllmsApache-2.0更新放缓FreshQA 是评估大模型回答时效性问题能力的基准数据集与代码，配套论文 FreshLLMs，用于测试模型在动态知识下的检索增强问答表现。
Data and code for FreshLLMs (https://arxiv.org/abs/2310.03214)
暂无版本404star21fork同步于 2026-11-07 07:29 UTC
README 里没有找到可直接复制的安装命令，请到 README 或官方文档查看安装方式。
硬件要求按 README 说明
README 未说明
FreshLLMs 论文的数据与代码，用搜索引擎增强让大模型回答时效性问题
FreshQA 是评估大模型回答时效性问题能力的基准数据集与代码，配套论文 FreshLLMs，用于测试模型在动态知识下的检索增强问答表现。
核心能力
- 提供每周更新的 FreshQA 时效性问答数据集
- FreshPrompt 用搜索引擎检索结果增强 LLM 提示
- FreshEval 用少样本上下文学习自动评判回答
- 支持 Relaxed 与 Strict 两种评估模式
- 配套 Colab notebook 可直接试用 FreshPrompt
容易踩的坑
内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README
暂无 release 数据（可能没有发版，或尚未同步）。
用户点评
登录用户的真实使用记录，可按版本和硬件筛选共 0 条
还没有人点评。第一个说出真实体验的人最有价值。
FreshLLMs
Data and code for our paper FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation.
:star: Our data and method have inspired or been used for the development of recent large language models (LLMs) including Google's Gemini, Perplexity.AI's Online LLMs, You.com, and Contextual AI's RAG 2.0 :star:
:star: We are excited to introduce SealQA, a new evaluation benchmark that builds on FreshQA with a stronger focus on reasoning. SealQA is available at https://huggingface.co/datasets/vtllms/sealqa and has been used by Google's Gemini, DeepSeek and Kimi :star:
Quick links
FreshQA
Next update: May 11, 2026
We update our dataset weekly or upon request. If you find any updates or misclassifications in our FreshQA questions or answers that we may have overlooked, please notify us by commenting on the dataset spreadsheet above or sending an email to [email protected].
Older versions:
FreshPrompt
FreshPrompt notebook: 
FreshEval
We believe that human evaluators possess the expertise and common sense required to detect issues like hallucinations, making them more reliable than automated evaluation metrics for assessing LLMs' factuality. However, researchers have the flexibility to adjust their evaluation methods if human evaluation proves challenging. An easily implemented alternative is to use standard metrics like F1/exact match or recall, which assess the overlap between the model response and the ground truth answer(s) (e.g., see You.com's recent blog where they report FreshQA recall). Researchers can also use LLM-based automatic evaluation metrics such as FactScore or our FreshEval metric below.
Automatic evaluation
To facilitate future evaluations, we have developed FreshEval, a simple automatic metric that uses few-shot in-context learning to teach an LLM to judge model responses, which achieved high agreement with human raters (see Appendix B in our paper for details).
To use FreshEval under a specific evaluation mode (Relaxed or Strict), please follow the instructions below:
- Make a copy of our latest data spreadsheet and store it in your Google Drive with a new filename (e.g.,
fresheval_relaxedorfresheval_strict). - Insert 3 new columns
D,E,Fin the new spreadsheet for model responses, evaluation rating, evaluation explanation, respectively and save your model's responses in columnD(see our sample evaluation spreadsheet below). - Run the associated
FreshEvalnotebook with the evaluation mode. Note that for demonstration purposes, we evaluated only the first 10 model responses. You can adjust the number as needed.
Note: Currently, we recommend gpt-4-1106-preview over gpt-4-0125-preview for FreshEval as it yielded slightly better agreement with human annotations in our small-scale evaluation.
Here are our FreshEval notebooks.
FreshEval (Relaxed) notebook:
FreshEval (Strict) notebook:
After obtaining TRUE/FALSE ratings for model responses, follow the instructions below to calculate the accuracy for each question category:
- Download our latest data spreadsheet in a Comma Separated Values (.csv) format and store it as
freshqa.csv. - Open this notebook and upload the
freshqa.csvfile to the session storage (Files > Upload file to session storage). - Replace the existing ratings in the notebook with your ratings and run the notebook.
Acknowledgements
We thank Hailey Joren, William Zhang, Varun Singh, Peter Hart, and Filipe Mesquita for their help in updating our FreshQA questions/answers.
We are grateful to the following people for their contributions to creating our original FreshQA dataset: Marzena Karpinska, Dustin Tran, Daniel Cer, Sam Fullerton, Elizabeth Clark, Nishant Raj, Xiaoyu Song, Yapei Chang, Yixiao Song, Nader Akoury, Ankita Gupta, Bill Ray, Chau Pham, Wenlong Zhao, Maximilian Mozes, Simeng Sun, Ronan Salz, Kalpesh Krishna, Katherine Thai, Kanishka Misra, Salaheddin Alzu'bi, Erica Cai, Thibault Sellam, Jiao Sun, Dhruv Agarwal, Tessa Masis, Andrew Drozdov, Brian Lester, George Wei, Naveen Jafer Nizar, Shufan Wang, Youngwoo Kim, and Shib Sankar Dasgupta.
We are also grateful to SerpApi for their generous sponsorship of 10,000 searches for FreshPrompt's users upon its release.
Citation
If you use our data or method, please cite our paper:
@misc{vu2023freshllms,
title={FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation},
author={Tu Vu and Mohit Iyyer and Xuezhi Wang and Noah Constant and Jerry Wei and Jason Wei and Chris Tar and Yun-Hsuan Sung and Denny Zhou and Quoc Le and Thang Luong},
year={2023},
eprint={2310.03214},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
langchain
langchain-ai·Python
用 Python 搭建 LLM 应用和智能体的开发框架，把模型调用、工具、检索和记忆串成可组合的链路，省去自己拼装底层胶水代码。
智能体开发+4RAG 框架
149312+17025klangchain-typesafe==0.0.22026-11-10 16:25 UTC
awesome-llm-apps
Shubhamsaboo·Python
收录 100 多个可直接上手的 AI Agent、Agent Skills 和 RAG 应用示例，帮开发者快速找到能跑起来的 LLM 项目参考。
AI Agent+4RAG 框架
144351+28421k2026-11-10 01:54 UTC
hello-agents
datawhalechina·Python
从零开始手把手教你构建 LLM 智能体的中文教程，覆盖 Agent 原理、RAG 等核心实践，适合入门与动手复现。
智能体教程+4RAG 框架
86618+30911kV1.0.32026-10-13 22:28 UTC
serviette
pathwaycom·Python
一个免代码的 RAG 服务器，能随文档更新自动同步答案，支持 8 种向量数据库，基于 Pathway 构建。
RAG+4RAG 框架
587191.5kv0.1.82026-11-10 16:12 UTC