Vibe-Skills

Quick start ·

SkillsBench is a benchmark designed to evaluate whether AI agents can effectively use Skills to complete professional tasks across diverse domains. Its purpose is to measure how much a model’s ability to solve complex real-world tasks improves when it is equipped with specialized Skills.

To evaluate performance in production-like environments where a large number of Skills are installed simultaneously, we adapted SkillsBench into a more realistic large-scale multi-Skill setting. In the original SkillsBench setup, each task is provided only with the specialized Skill associated with that task. In our modified setting, every task is evaluated in a global environment containing all 195 specialized Skills, while all other experimental conditions remain unchanged. This setting is intended to assess whether an agent can autonomously discover, select, and orchestrate the relevant Skills from a large installed Skill pool, and organize them into an effective workflow for completing complex tasks.

vibeskills v4.1.0 was benchmarked on SkillsBench (https://www.skillsbench.ai/) using DeepSeekV4Flash-VE and OpenHands as the baseline evaluation setup. Compared with the baseline without vibeskills, vibeskills increased the mean verifier reward by 21.12 percentage points, while reducing token consumption by 29.6% and tool calls by 33.1%.

Analysis of the logs from the original benchmark shows that VibeSkills achieves better task performance not by invoking more Skills.

Instead, it first clarifies the task objective and delivery requirements, then decomposes a complex task into several verifiable subtasks. It subsequently selects only a small number of truly relevant capabilities from a large pool of candidate Skills and executes them in an order determined by their dependencies. This helps reduce misunderstandings of the task, omitted steps, and incorrect Skill selection, thereby improving overall task performance.

In terms of token cost and tool usage, this workflow also eliminates a substantial amount of ineffective trial and error. Native agents are more likely to repeatedly invoke tools in unproductive directions, reread the same context, and redo previous work. In contrast, VibeSkills converges more quickly on the critical steps through clearer planning and pre-delivery checks. As a result, it not only improves task quality, but also significantly reduces tool-call loops and the token overhead caused by repeated context processing.

Task

Use public data to complete a reproducible classification experiment and deliver a data audit, statistical review, 4 result figures, a scientific report, and a 7-slide group-meeting deck.

The diagram shows what happened after the requirement and plan were approved: how the task was executed, what it produced, and how the result was checked.

The task used the L workflow and proceeded in order. During publication preparation, the configured folders on the same host contained more than 100 Skills. VibeSkills reviewed the candidates and their SKILL.md files, selected 7 for this task, and arranged the work into 5 groups and 10 work units. Those units covered environment setup, data audit, modeling, statistical review, figures, the report, and the slide deck.

After the work finished, VibeSkills ran 17 checks across the data, experiment results, figures, report, and slides. The task passed final acceptance after the required files, cross-deliverable consistency, and core reproduction all passed.

%%{init: {"flowchart": {"curve": "monotoneX", "nodeSpacing": 18, "rankSpacing": 36}}}%%
flowchart LR
    subgraph DISC["Skill discovery"]
        direction TB
        A["Configured Skill folders<br/>100+ Skills"]
        B["Shortlist candidates<br/>Read SKILL.md"]
        SEL["Skill selection<br/>7 Skills assigned"]
        A --> B
        B --> SEL
    end

    subgraph EXEC["Execution · 5 work groups · 10 work units"]
        direction TB

        subgraph G1["G1 · 01 Environment and data"]
            direction LR
            u01["U01<br/>Environment setup"]
            u02["U02<br/>Data audit"]
            u01 --> u02
        end

        subgraph G2["G2 · 02 Modeling and reproduction"]
            direction LR
            u03["U03<br/>Baseline experiment"]
        end

        subgraph G3["G3 · 03 Statistics and scientific review"]
            direction LR
            u04["U04<br/>Statistical analysis"]
            u05["U05<br/>Scientific review"]
            u04 --> u05
        end

        subgraph G4["G4 · 04 Figures and report"]
            direction LR
            u06["U06<br/>Result figures"]
            u07["U07<br/>Report draft"]
            u08["U08<br/>Report review"]
            u06 --> u07
            u07 --> u08
        end

        subgraph G5["G5 · 05 Slides and acceptance"]
            direction LR
            u09["U09<br/>Group-meeting slides"]
            u10["U10<br/>Case package and consistency"]
            u09 --> u10
        end

        G1 --> G2
        G2 --> G3
        G3 --> G4
        G4 --> G5
    end

    subgraph MID["Run and outputs"]
        direction TB
        S(["Run status<br/>10 / 10 completed<br/>0 failed · 0 blocked"])
        D["Real outputs<br/>4 figures · Scientific report<br/>7-slide deck"]
        S --> D
    end

    subgraph VERIFY["Verification · 17 checks"]
        direction TB

        subgraph V1["V1 · Foundation and plan"]
            direction LR
            t01["T01<br/>required-files"]
            t02["T02 module-output-<br/>patterns"]
            t03["T03 runtime-plan-<br/>binding"]
            t04["T04 environment-<br/>contract"]
            t01 --> t02
            t02 --> t03
            t03 --> t04
        end

        subgraph V2["V2 · Data, model, and reproduction"]
            direction LR
            t05["T05<br/>dataset-contract"]
            t06["T06 split-and-model-<br/>contract"]
            t07["T07<br/>baseline-results"]
            t08["T08 exact-<br/>reproduction"]
            t05 --> t06
            t06 --> t07
            t07 --> t08
        end

        subgraph V3["V3 · Statistics and deliverables"]
            direction LR
            t09["T09 uncertainty-<br/>consistency"]
            t10["T10 statistics-write-<br/>protection"]
            t11["T11 figure-<br/>traceability"]
            t12["T12 report-<br/>consistency"]
            t13["T13 slides-<br/>consistency"]
            t09 --> t10
            t10 --> t11
            t11 --> t12
            t12 --> t13
        end

        subgraph V4["V4 · Publication and boundaries"]
            direction LR
            t14["T14 bilingual-summary-<br/>consistency"]
            t15["T15 visual-material-<br/>guidance"]
            t16["T16 manifest-<br/>boundary"]
            t17["T17 artifact-path-<br/>boundary"]
            t14 --> t15
            t15 --> t16
            t16 --> t17
        end

        V1 --> V2
        V2 --> V3
        V3 --> V4
    end

    E(["Final acceptance<br/>17 / 17 checks passed<br/>PASS"])

    DISC --> EXEC
    EXEC --> MID
    MID --> VERIFY
    VERIFY --> E

    classDef source fill:#EAF3F3,stroke:#2B6F73,color:#182026;
    classDef selected fill:#F5EBEE,stroke:#8A5363,color:#182026;
    classDef unit fill:#FFFFFF,stroke:#5B7F83,color:#182026;
    classDef status fill:#F7EEF1,stroke:#8A5363,color:#182026,stroke-width:2px;
    classDef output fill:#E8F2F0,stroke:#2D7F75,color:#182026;
    classDef check fill:#FFFFFF,stroke:#8A9AA7,color:#182026;
    classDef result fill:#EAF4EE,stroke:#2F7A4B,color:#182026,stroke-width:2px;
    class A,B source;
    class SEL selected;
    class u01,u02,u03,u04,u05,u06,u07,u08,u09,u10 unit;
    class S status;
    class D output;
    class t01,t02,t03,t04,t05,t06,t07,t08,t09,t10,t11,t12,t13,t14,t15,t16,t17 check;
    class E result;

    style DISC fill:transparent,stroke:#AAB7C4,stroke-width:1px,stroke-dasharray:4 3;
    style EXEC fill:transparent,stroke:#AAB7C4,stroke-width:1px,stroke-dasharray:4 3;
    style MID fill:transparent,stroke:#AAB7C4,stroke-width:1px,stroke-dasharray:4 3;
    style VERIFY fill:transparent,stroke:#AAB7C4,stroke-width:1px,stroke-dasharray:4 3;
    style G1 fill:#FFFFFF,stroke:#DCE4EA,stroke-width:1px;
    style G2 fill:#FFFFFF,stroke:#DCE4EA,stroke-width:1px;
    style G3 fill:#FFFFFF,stroke:#DCE4EA,stroke-width:1px;
    style G4 fill:#FFFFFF,stroke:#DCE4EA,stroke-width:1px;
    style G5 fill:#FFFFFF,stroke:#DCE4EA,stroke-width:1px;
    style V1 fill:#FFFFFF,stroke:#DCE4EA,stroke-width:1px;
    style V2 fill:#FFFFFF,stroke:#DCE4EA,stroke-width:1px;
    style V3 fill:#FFFFFF,stroke:#DCE4EA,stroke-width:1px;
    style V4 fill:#FFFFFF,stroke:#DCE4EA,stroke-width:1px;
    linkStyle default stroke:#6D878B,stroke-width:1px;

VibeSkills gives an Agent one process from receiving a task to checking the delivery.

Each stage answers a concrete question: what needs to be done, how the work should proceed, which Skills should take part, what actually happened, and whether the result is ready to deliver.

Level	Best for	How it works
L	Multi-step work of manageable size	Splits the task, then works through the parts in order with less time and context overhead
XL	Larger work with several relatively independent parts	Uses a more detailed breakdown and can run up to two non-conflicting parts at the same time, with additional coordination and result collection

Local Skills can store tool usage, working steps, decision rules, and checking methods.

VibeSkills reviews the local Skill folders you configure, then shortlists the Skills that fit the work required by each part of the task.

The left side shows the different kinds of work in the task, VibeSkills makes the assignment in the middle, and the local Skill folders are on the right. A selected Skill is tied to concrete work, expected delivery, and a check. The current Agent then follows the shared plan.

VibeSkills does something straightforward: it first makes the whole task clear, then assigns the right Skills to the relevant parts. It coordinates the work and checks the combined result at the end. The task uses the Skills it needs; the rest of the local library stays available without entering the plan.

You can keep adding your own Skills, team Skills, and third-party Skills. VibeSkills does not call every installed Skill automatically; it selects the Skills that fit the current task. The size of the library defines the available choices, not a list that every task must use.

VibeSkills checks the Skill folders you configure, but finding files locally and placing their full contents in the model context are different operations.

Discovery and index generation happen locally. VibeSkills first extracts compact information such as each Skill's name, description, intended use, and boundaries, then uses that information to shortlist candidates for each part of the task.

Only retained candidates are then read as complete SKILL.md files. Execution uses only the Skills written into the plan. Token usage therefore depends mainly on how many candidates the task retains, how long those documents are, and how complex the task is. It is not the same as reading the full local Skill library into the model context.

This overhead is not zero. More candidates, longer Skill documents, or a more finely divided task will use more context. The current design bounds that cost with a local index, candidate shortlisting, and on-demand reading.

Alongside the shared Skills directory, more local folders can be listed in ~/.vibeskills/skill-roots.json or <workspace>/.vibeskills/skill-roots.json.

A Skill needs a readable SKILL.md, a name that does not conflict with another Skill, and a clear fit for the current work before it can be selected. Adding a local folder makes those Skills available to later tasks without waiting for the VibeSkills repository to include them.

During planning, agent_skill_organization stores which Skills are intended for each part of the task. During execution, module_assignments stores the actual assignment. Finding a Skill means it can be considered; it does not mean the Skill has already taken part in the work.

A public example lets readers follow the requirement, plan, actual result, and final check.

VibeSkills keeps the approved requirement, plan, execution progress, and final check in the same task record. A later session can continue from the saved progress, and a review can compare the original plan with the actual result. Installation state is recorded separately so it is not confused with task completion.

File or directory	What it is for
install-receipt.json	Records the files written by the installer so check can find missing or changed files
session_root	Stores the input, progress, important decisions, and summary for one task
module-work-plan.json	Stores the approved work plan, including responsibility, expected output, and checks
module-execution.json	Stores what each part actually produced and whether it completed, failed, or was blocked
delivery-acceptance-report.json or .md	Stores the final check and shows which items passed

Maintainers can use the pre-release checks. Start with the checks in that list and run wider audits only when there is a reason.

A successful installation does not mean the task ran, and a task record does not mean the final result passed its checks.

Questions, corrections, and well-scoped contributions are welcome through GitHub Issues and pull requests.

VibeSkills discussions and community practice can also continue on LINUX DO. It is a place to exchange technical questions, AI practice, and experience. Thank you to the LINUX DO community for supporting this project.

The VibeSkills 3.1.0 community practice cases collect several examples that were shared with the community.

Community contributors include xiaozhongyaonvli and ruirui2345.

Third-party software attribution and license information are listed in NOTICE and third-party licenses.