easy-dataset

如果遇到 Github 下载慢的问题可以使用网盘下载：https://pan.quark.cn/s/194b7eedf16e

🔧 修复（Fixes）

1. 修复自定义评估题目 Prompt 无法生效的问题 #676
2. 修复筛选重置时仍使用旧条件发起请求的竞态问题，确保任务进度、状态展示更稳定
3. 修复多轮对话与评估功能切换时页面卡顿/假死的问题

⚡ 优化（Optimizations）

1. 模型提供商交互体验

• 新增/接入多家模型提供商（provider）图标资源，统一 provider logo 映射工具，标准化视觉展示
• 调整模型设置、项目模型配置、会话列表展示等页面与接口的配置逻辑，优化提供商选择流程与展示一致性

2. 页面性能与响应速度

• 解决多轮对话/评估功能切换卡顿问题，提升页面切换、高频筛选场景的响应速度
• 前端新增请求取消机制（AbortController），避免快速切换操作时旧请求堆积导致的页面卡顿

3. 接口与数据请求轻量化

• 优化多轮对话列表接口：分页查询不再返回 rawMessages 等大字段，显著降低列表请求数据体积
• 优化批量任务“全选”逻辑：获取待操作数据时仅查询 ID，避免拉取整条记录，减少数据传输量
• 优化评估数据集接口：
  • 需统计数据时，并发查询列表与统计信息，提升加载效率；
  • 筛选/翻页场景默认仅拉取列表数据，减少重复查询压力

感谢 @AdamPlatin123 的贡献

🔧 Fixes

1. Fixed the issue where custom evaluation question Prompts failed to take effect #676
2. Fixed the race condition where old filter conditions were still used for requests after reset, ensuring stable display of task and progress status
3. Fixed freezing/unresponsiveness when switching between multi-turn dialogue and evaluation features

⚡ Optimizations

1. Model Provider Display & Selection Experience

• Added/connected icon resources for multiple model providers, unified the provider logo mapping tool to standardize visual display
• Adjusted configuration logic of related pages and interfaces (covering model settings, project model configuration, session list display, etc.), optimizing provider selection experience

2. Page Performance & Response Speed

• Resolved freezing/unresponsiveness when switching between multi-turn dialogue and evaluation features, improved response speed for page switching and high-frequency filtering scenarios
• Added request cancellation mechanism (AbortController) on the frontend to avoid page lag caused by accumulated old requests during rapid switching

3. Interface & Data Request Lightweight

• Optimized multi-turn dialogue list interface: paginated queries no longer return large fields such as rawMessages, significantly reducing request data volume
• Optimized batch task "select all" logic: only query IDs of data to be operated on when fetching data, avoiding pulling entire records and reducing data transmission
• Optimized evaluation dataset interface:
  • Concurrently queries list and statistical data when statistics are needed to improve loading efficiency;
  • Only pulls list data by default for filtering/pagination scenarios to reduce repeated query pressure

如果遇到 Github 下载慢的问题可以使用网盘下载：https://pan.quark.cn/s/194b7eedf16e

🔧 Fixes

1. Fixed the synchronization issue between text block full selection status and filter conditions
2. Fixed the problem where model API call failures were incorrectly displayed as successful in asynchronous tasks
3. Fixed the filter box experience issues in the dataset management module (e.g., invalid conditions, style anomalies)

⚡ Optimizations

1. Added batch deletion support for text blocks to improve efficiency in multi-block scenarios
2. Increased the maximum upload file size limit from 50MB to 300MB, enabling processing of larger documents or images
3. Exposed image answer generation prompts in custom prompt configurations for fine-grained control of VQA generation logic #658
4. Optimized export speed for extremely large datasets to reduce waiting time #667

🔧 修复

1. 修复文本块全选状态与筛选条件不同步的问题
2. 修复模型API调用失败，但异步任务状态仍显示为成功的异常
3. 修复数据集管理模块筛选框的交互体验问题（如筛选条件不生效、样式异常等）

⚡ 优化

1. 文本块支持批量删除，提升多文本块场景下的操作效率
2. 上传文件大小限制从 50MB 放宽至 300MB，支持处理更大体积的文档或图像
3. 自定义提示词配置中开放图像答案生成的提示词，支持精细化控制图像问答生成逻辑 #658
4. 优化超大量数据集的导出速度，减少导出等待时间 #667

In v1.7.0, we focus on addressing the core pain points of model evaluation: difficult test set construction, lack of quantitative metrics, and the gap between automation and manual testing. This release introduces a brand-new Evaluation Module, creating a closed loop from Test Set Generation to Automated Scoring and Human Blind Testing.

Here are the detailed updates:

🎉 Core Highlights

• One-Stop Evaluation Loop: Support automatic test question generation from raw documents, one-click multi-model automated scoring, and visualized comparison reports.
• LMArena Mode Integration: Built-in "Chatbot Arena" style human blind testing to quantify subjective "feelings" into actionable data.
• Multi-Dimensional Question Support: Covers 5 core question types, satisfying evaluation needs ranging from fact-checking to logical reasoning.

🚀 New Features

1. Intelligent Test Set Generation

Stop worrying about the lack of QA pairs. You can now quickly build high-quality evaluation datasets through multiple methods:

• Document Extraction: Automatically slice and extract questions from PDF/Markdown/Docx domain literature.

• Dataset Variants: Generate variants based on existing training sets (e.g., converting multiple-choice questions to true/false) to expand testing diversity.

• 5 Core Question Types:

• ✅ True/False: Detect model hallucinations.

• ✅ Single/Multiple Choice: Test knowledge extraction and discrimination.

• ✅ Short Answer: Test the ability to capture and express core knowledge concisely.

• ✅ Open-Ended: Evaluate long-text reasoning and summarization skills.

• Ratio Configuration: Support custom distribution ratios for different question types during the generation task.

2. Automated Evaluation Tasks (Auto-Evaluation)

Conduct concurrent assessments on multiple models just like an "exam". The system supports two grading modes:

• Rule-Based Scoring (Objective): For Choice and True/False questions. No LLM is required; the system uses direct code-level comparison for zero cost and zero error.
• Teacher Model Scoring (LLM-as-a-Judge): For Short Answer and Open-Ended questions. Configure a "Teacher Model" to score and comment on answers with customizable criteria (Prompts).

3. Human Blind Test Arena

Return to the authentic "Side-by-Side" evaluation experience:

• Anonymous Battle: Model names are hidden. Answers are displayed side-by-side (supports streaming output).
• Intuitive Voting: Simply click "Left is better", "Right is better", or "Tie".
• Win Rate Statistics: Automatically generate win-rate comparison charts to eliminate brand bias.

4. Data & Ecosystem

• Multi-Format Import/Export: Fully supports JSON, XLS, and XLSX formats for test set management.
• Built-in Domain Question Banks: Pre-loaded standard test sets across multiple disciplines and domains, ready to use out of the box.
• Fully Open Prompts: Complete access to configure prompts for the evaluation system (Question Generation, Answer Extraction, Scoring Standards), allowing for high customization.

在 v1.7.0 版本中，我们重点解决了模型评估中 “测试集难构造”、“量化指标难获取” 以及 “自动化与人工割裂” 的痛点。新增全新的 「评估」 模块，打通了从 测试集生成 到 自动化评分 再到 人工盲测 的全流程闭环。

以下是本次更新的详细内容：

🎉 核心亮点

• 一站式评估闭环：支持从原始文档自动生成测试题，一键发起多模型自动化评分，并提供可视化的对比报告。
• LMArena 模式集成：内置类似 Chatbot Arena 的人工盲测竞技场，解决“感觉”无法量化的问题。
• 多维题型支持：覆盖 5 种核心题型，满足从事实核查到逻辑推理的全方位评估需求。

🚀 新增功能 (New Features)

1. 智能测试集构造 (Test Set Generation)

不再为没有 QA 对而发愁，现在支持通过多种方式快速构建高质量评估数据集：

• 文档提取：支持从 PDF/Markdown/Docx 领域文献中自动切片并提取题目。

• 数据集变体：支持基于现有训练集生成变体（如将选择题改为判断题），扩充测试多样性。

• 五大题型支持：

• ✅ 判断题：检测模型幻觉。

• ✅ 单选/多选题：考察知识提取与辨析。

• ✅ 简答题：考察核心知识点的精简表达。

• ✅ 开放题：考察长文本推理与逻辑总结。

• 比例配置：支持自定义生成任务中各题型的分布比例。

2. 自动化评估任务 (Auto-Evaluation)

像“考试”一样对多个模型进行并发测评，支持两种阅卷模式：

• 客观题规则评分：针对选择、判断

如果遇到 Github 下载慢的问题可以使用网盘下载：https://pan.quark.cn/s/194b7eedf16e

🔧 修复

1. 修复导入数据删除报错的Bug #645
2. 修复部分OPEN AI模型max_completion_tokens设置失效的问题 #623
3. 修复部分阿里云百炼视觉模型无法识别图片的问题 #622
4. 修复首页卡片未统计图片数据集数量的问题 #611 #607
5. 修复MAC图片设计不规范的问题 #630
6. 修复大量标签数据时，标签树查询接口阻塞导致问题管理列表加载异常的问题 #629
7. 修复大量标签数据情况下，自动蒸馏任务构建标签速度极慢的问题 #629

✨ 新功能

1. 支持单独导出问题 #644
2. 支持批量选择文件并自定义添加GA #643
3. 支持全选批量删除已上传文件 #636
4. 问题管理支持“不匹配关键字”筛选，可快速过滤不符合需求的问题 #613
5. 增加 Token 统计面板（从首页，右上角统计图标可进入此功能） #133

⚡ 优化

1. 优化项目选择弹框的关闭方式

🔧 Fixes

1. Fixed the error when deleting imported data #645
2. Fixed the invalid max_completion_tokens setting issue for some OPEN AI models #623
3. Fixed the image unrecognition problem of some Alibaba Cloud Bailian visual models #622
4. Fixed the homepage card's failure to count image dataset quantities #611 #607
5. Fixed the irregular design issue of MAC images #630
6. Fixed the question management list loading exception caused by label tree query interface blocking (with large label data) #629
7. Fixed the slow label construction speed of automatic distillation tasks (with large label data) #629

✨ New Features

1. Supported exporting questions individually #644
2. Supported batch selecting files and custom adding GA #643
3. Supported selecting all and batch deleting uploaded files #636
4. Added "exclude keyword" filtering in question management to quickly filter out unwanted questions #613

⚡ Optimizations

1. Optimized the closing method of the project selection pop-up