ferret

MontFerretApache-2.0活跃维护

用声明式 DSL 描述网页数据抽取流程,由 Go 运行时执行,省去手写爬虫脚本,适合结构化抓取和浏览器自动化。

Declarative data automation language and Go runtime for structured extraction workflows.

v2.0.0-alpha.62发布于 2026-11-02 17:33 UTC6,025star324fork同步于 2026-11-10 23:35 UTC

怎么上手

由 README 提炼,执行前请确认
go get github.com/MontFerret/ferret/v2@latest
硬件要求按 README 说明

README 未说明

下载:源码 zip·GitHub Releases(安装包)本站不镜像安装包

项目说明

由 README 自动翻译提炼

声明式数据自动化语言与 Go 运行时,用于结构化提取与同步工作流

用声明式 DSL 描述网页数据抽取流程,由 Go 运行时执行,省去手写爬虫脚本,适合结构化抓取和浏览器自动化。

核心能力

  • 用 FQL 声明式查询、转换并同步多源结构化数据
  • 可嵌入 Go 运行时,支持编译计划复用与隔离会话
  • 基于能力模型向脚本暴露宿主对象与外部系统
  • 通过命名空间函数、模块、钩子扩展运行时
  • 字节码 VM 支持预编译可移植程序重复执行

容易踩的坑

当前分支为 alpha 阶段的 v2,稳定版需切到 v1 分支;v1 迁移需先装 CLI 并运行 ferret migrate,且不会自动改写业务逻辑;Run 返回 nil 错误仅表示拿到输出句柄,需继续 Collect 或 Consume 才能拿到最终执行错误

内容由模型根据 README 生成,可能不完整或不准确,以官方文档为准。看原文 README

最近发版

5 个版本v2.0.0-alpha.62
  1. v2.0.0-alpha.62pre-release最新正式版2026-11-02 · 2026-11-02 17:33 UTC
  2. v2.0.0-alpha.61pre-release2026-11-02 · 2026-11-02 15:01 UTC
    查看更新说明
  3. v2.0.0-alpha.60pre-release2026-11-02 · 2026-11-02 13:20 UTC
    查看更新说明

    What's Changed

    Full Changelog: https://github.com/MontFerret/ferret/compare/v2.0.0-alpha.59...v2.0.0-alpha.60

  4. v2.0.0-alpha.59pre-release2026-10-31 · 2026-10-31 18:19 UTC
    查看更新说明
  5. v2.0.0-alpha.58pre-release2026-10-27 · 2026-10-27 01:46 UTC
    查看更新说明

    What's Changed

    • refactor(output): Refactor output handling with collect functions and add tests by @ziflex in https://github.com/MontFerret/ferret/pull/1057
    • refactor(encoding): Add tests and refactor output handling for robustness by @ziflex in

    Full Changelog: https://github.com/MontFerret/ferret/compare/v2.0.0-alpha.57...v2.0.0-alpha.58

用户点评

登录用户的真实使用记录,可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译

    Ferret


    Explore Ferret v2

    Ferret v2 is currently in alpha. You can try the new syntax in the playground and read more about the design behind the new runtime and language capabilities:


    Notice: This branch contains the upcoming Ferret v2. For the stable v1 release, please visit Ferret v1.


    What is it?

    Ferret is a declarative-first, expression-oriented embedded language and runtime for data automation.

    FQL combines querying, transformation, synchronization, and structured results with host-defined values and capabilities. Applications can embed the runtime and decide exactly which functions, modules, data, and external operations a program can use.

    The language keeps a declarative core and adds domain-oriented orchestration plus constrained mutable state for automation that cannot be expressed as a pure data transformation. It is deliberately focused rather than a general-purpose scripting language.

    Features

    • Purpose-built declarative language for querying, transforming, synchronizing, and automating structured data
    • Embeddable Go runtime with reusable compiled plans and isolated execution sessions
    • Capability-based host values for exposing application objects, resources, and external systems directly to FQL
    • Unified query model for browsers, APIs, databases, documents, and custom data sources
    • Extensible runtime through namespaced functions, modules, hooks, and custom value types
    • Event-driven synchronization and dispatch for interacting with asynchronous and stateful resources
    • Managed resource lifecycle for files, connections, cursors, streams, and other host resources
    • Bytecode VM and portable programs for efficient repeated execution and precompiled artifacts

    Getting started

    go get github.com/MontFerret/ferret/v2@latest
    

    There are currently two ways to start with Ferret v2:

    • Native v2 API - recommended for new projects
    • compat module - recommended as a first migration step for existing v1 integrations

    Use github.com/MontFerret/ferret/v2 and ferret.New(opts...) for Native embedding. For integrations using github.com/MontFerret/api, the official adapter is the separate github.com/MontFerret/ferret/v2/uapi package. uapi.New(version, opts...) creates and owns a Native engine; runtime Close closes it. uapi.Wrap(native, version) borrows an existing engine; its Close is a no-op that leaves both usable. Both retain the supplied Ferret Core version for Runtime.Version(ctx). Settle work and close sessions and plans before their owning runtime or borrowed engine. See the Universal adapter guide for the supported options and lifecycle contract.

    New projects

    Use the native v2 API built around the following flow:

    Engine -> compile query -> create session -> run -> collect or consume
    
    package main
    
    import (
    	"context"
    	"fmt"
    	"log"
    
    	"github.com/MontFerret/ferret/v2"
    )
    
    func main() {
    	ctx := context.Background()
    
    	eng, err := ferret.New()
    	if err != nil {
    		log.Fatal(err)
    	}
    	defer eng.Close()
    
    	plan, err := eng.Compile(ctx, ferret.NewAnonymousSource(`return 1 + 1`))
    	if err != nil {
    		log.Fatal(err)
    	}
    	defer plan.Close()
    
    	session, err := plan.NewSession(ctx)
    	if err != nil {
    		log.Fatal(err)
    	}
    	defer session.Close()
    
    	output, err := session.Run(ctx)
    	if err != nil {
    		log.Fatal(err)
    	}
    
    	defer output.Close()
    	content, err := output.Collect(ctx)
    	if content != nil {
    		fmt.Println(string(content.Data))
    	}
    	if err != nil {
    		log.Print(err)
    	}
    }
    

    Run executes and encodes eagerly. Its nil error means an output handle was obtained; Collect or Consume reports terminal execution and cleanup errors. Keep the invocation context alive through consumption. Available content may accompany an error. Consume outputs used only for side effects, and close outputs you abandon. See the consumable-output migration.

    Migration from v1

    Ferret v2 introduces a new architecture and public API, so existing Go applications should migrate in two stages: first to the v2 compatibility API, then incrementally to the native v2 API.

    Run the ferret migrate command from anywhere inside the application's Go module:

    ferret migrate --dry-run # List the files that would change
    ferret migrate --print   # Print a unified diff without changing files
    ferret migrate           # Apply the migration
    

    The command rewrites the documented v1 imports to their v2 compatibility packages, updates go.mod and go.sum as required by those rewrites, and formats changed Go files. Generated, vendored, and nested-module files are left untouched. Unsupported v1 imports are reported as manual follow-up; if the project vendors dependencies, run go mod vendor after applying the migration.

    This is only the mechanical compatibility stage. The command does not convert application logic to the native v2 API or migrate drivers and other unsupported v1 packages. Running it again on an already-migrated compatibility project is a no-op and does not upgrade the Ferret v2 dependency merely because the CLI is newer.

    After applying the migration, address any reported manual follow-up, build and test the application, and then migrate to the native v2 API over time. The compatibility layer is a migration aid, not the long-term preferred API; new projects should use the native v2 packages directly.

    Alpha status

    Ferret v2 is currently in active development.

    Alpha releases are intended for early adopters, experimentation, and feedback. Some APIs and language features may still change before the stable v2 release.

    Maintainers

    Support Ferret

    Ferret is supported by organizations and community members who help fund its continued development. View all supporters.