v0.1 只是骨架版本,真实 LeRobot 策略与 LIBERO 环境需等适配器实现,fake runner 不是基准结果;torch、lerobot、libero 都是可选依赖,需按需安装对应 extras;仓库由 MindNLP 转型而来,旧代码在 legacy 分支,别照旧文档用
MindAct
candle-orgApache-2.0活跃维护让 HuggingFace 上的 Transformers 和 Diffusers 模型无缝跑在昇腾 MindSpore 上,省去手动迁移适配的麻烦,并支持加速。
MindSpore + 🤗Huggingface: Run any Transformers/Diffusers model on MindSpore with seamless compatibility and acceleration.
pip install -e .
mindact config-check configs/experiments/libero-baseline.yamlREADME 未说明
面向桌面操作任务的模仿学习策略训练与评估框架,整合 LeRobot 数据集与 LIBERO 仿真基准
让 HuggingFace 上的 Transformers 和 Diffusers 模型无缝跑在昇腾 MindSpore 上,省去手动迁移适配的麻烦,并支持加速。
核心能力
- 用适配器接口统一数据集、策略与环境
- YAML 配置加 manifest 记录实验来源,可复现
- torch、lerobot、libero 按 extras 懒加载
- 约定目录结构管理 checkpoint、日志与结果
- 内置 fake runner 跑通评估流程做冒烟测试
容易踩的坑
- v0.5.1最新正式版2025-11-05 · 2025-11-05 07:34 UTC
这个版本没有写更新说明。
- v0.5.02025-11-03 · 2025-11-03 07:42 UTC
查看更新说明
What's Changed
- init mindnlp 0.5.0 with mindtorch by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2016
- fix bugs on transformers 4.51.1 by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2017
- 【开源实习】Bartpho模型微调 by @SeasonMay in https://github.com/mindspore-lab/mindnlp/pull/2030
- update qwen jit inference by @ldqLDQ in https://github.com/mindspore-lab/mindnlp/pull/2041
- 【开源实习】Barthez模型微调 by @SeasonMay in https://github.com/mindspore-lab/mindnlp/pull/2032
- use new core to adapt huggingface library by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2056
- use lazymodule to load transformers submodules by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2057
- fix some bugs for transformers ut(bert) by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2058
- use huggingface transformers official ut to test by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2059
- fix proxy error and pass more hf ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2060
- fix all bert ut and skip useless ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2061
- pass all llama ut on Ascend by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2062
- fix qwen series bug by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2063
- fix lora training by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2064
- fix albert by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2065
- fix autoformer by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2066
- fix a class models ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2067
- fix b class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2068
- fix c class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2069
- fix d class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2070
- fix e class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2071
- fix e class by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2072
- fix g class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2073
- fix h class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2074
- fix i class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2075
- fix l class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2076
- fix m class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2077
- fix n class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2078
- fix o class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2079
- fix diffuser models ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2080
- fix diffusers unet ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2081
- fix transformers p class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2082
- fix transformers r class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2083
- fix diffuers pipelines c class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2084
- fix diffusers pipelines d class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2085
- fix diffusers pipeline h class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2086
- fix diffusers pipelines k class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2087
- fix transformers u class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2088
- fix diffusers pipelines l class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2089
- fix transformers v class ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2090
- fix diffusers omni ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/2091
- fix transformers w class ut and diffusers omni by @lvyufeng in https://github.com/mindsp
- v0.4.12025-04-10 · 2025-04-10 07:55 UTC
查看更新说明
What's Changed
- 【开源实习】Speech2Text模型迁移 by @imeet000 in https://github.com/mindspore-lab/mindnlp/pull/1725
- 【开源实习】Unispeech模型迁移 by @liuyifan123123 in https://github.com/mindspore-lab/mindnlp/pull/1722
- Fix typing mistakes in documentation by @Yashbhatt786 in https://github.com/mindspore-lab/mindnlp/pull/1740
- support O2 on OrangePi, tinyllama 450ms -> 160ms by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1744
- fix no_grad state error by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1754
- avoid lack of cumsum operator on OrangePi by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1755
- fix enable_grad by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1757
- fix name for nn.Linear by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1758
- fix nll_loss casued backward error by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1759
- fix out_channels of Linear by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1760
- add toc in readme.md by @adarsh-jha-dev in https://github.com/mindspore-lab/mindnlp/pull/1746
- safe_load_file use mmap to speedup by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1763
- fix from_numpy caused error by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1765
- fix llama embedding by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1774
- upgrade wav2vec by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1776
- update nn.utils.parametrizations by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1777
- fix sew and sew_d by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1782
- fix whisper ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1786
- fix llava on Ascend by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1788
- fix mixtral ut by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1790
- fix gamma on CPU and GPU by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1791
- 【开源之夏】add dpo trainer and support dpo training for mindnlp by @Trace2333 in https://github.com/mindspore-lab/mindnlp/pull/1701
- 【开源实习】GPT-J-6B模型应用开发 by @LuMH1027 in https://github.com/mindspore-lab/mindnlp/pull/1742
- 【开源实习】ImageGPT模型应用开发 by @Helloyouth2022 in https://github.com/mindspore-lab/mindnlp/pull/1700
- 【开源实习】X_CLIP模型应用开发 by @dyedd in https://github.com/mindspore-lab/mindnlp/pull/1694
- 【开源实习】UPerNet模型应用开发 by @Maysixi in https://github.com/mindspore-lab/mindnlp/pull/1717
- update whisper stream by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1796
- 【开源实习】multilayer_perceptron_lora模型微调 by @barryyfli in https://github.com/mindspore-lab/mindnlp/pull/1749
- dino应用开发 by @LuMH1027 in https://github.com/mindspore-lab/mindnlp/pull/1797
- support depth_estimation & doc_qa by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1800
- support fill_mask/image_classification/image_feature_extraction pipelines by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1801
- init mimm & move tests folder by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1804
- fix
per_gpuargs by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1809 - add mindspore infer function patch by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1810
- fix sync parallel and support low_cpu_mem_usage by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1814
- 【开源实习】dpt模型应用开发 by @yegoling in https://github.com/mindspore-lab/mindnlp/pull/1813
- 【开源实习】LayoutLMv2模型应用开发 by @Helloyouth2022 in https://github.com/mindspore-lab/mindnlp/pull/1805
- 【开源实习】BERT模型应用开发 by @KhunLounZai in https://github.com/mindspore-lab/mindnlp/pull/1781
- 【开源实习】peft_adalora_seq2seq模型微调 by @liuyifan123123 in https://github.com/mindspore-lab/mindnlp/pull/1747
- 【开源实习】Vision Transformer模型应用开发 by @KhunLounZai in https://github.com/mindspore-lab/mindnlp/pull/1643
- 【开源实习】SegFormer模型应用开发 by @KhunLou
- v0.4.02024-10-14 · 2024-10-14 03:54 UTC
查看更新说明
时隔5个月,MindNLP 0.4来啦!!!这一次在开源社区同学们的努力下带来了数个重大特性更新!!!
- 模型大量更新,支持250+ transformers模型,覆盖90% huggingface模型,使用MindSpore进行模型复现、开发训练再无迁移之忧(社区会同步huggingface和业界新模型,随时更新)。
- 全平台支持:全面支持昇腾910A、910B、310B(香橙派)、GPU、CPU。(PS:目前香橙派唯一可用的AI开发套件)
- 分布式并行推理:支持多卡多进程并行推理(超过10B的模型可以玩啦!)
- 量化算法支持:香橙派支持Smooth Quant,GPU支持bitsandbytes int8量化
- Sentence transformer支持:可以开心的搞RAG啦
- 动态图性能优化:昇腾硬件下动态图推理速度达到Pytorch+GPU同水平(Llama实测性能85ms/token)
- 真正的动静统一:通过mindspore.jit一行切换图模式,完全使用huggingface code style,既保证易用性,又能快速提升性能。昇腾硬件下Llama实测性能为动态图2倍(45ms/token),与mindspore其他静态图based套件性能完全一致。
- 海量LLM应用更新:包括文本信息抽取、聊天机器人、语音识别、ChatPDF、音乐生成、代码生成、声音克隆等等,伴随着模型支持度的提高,更多好玩的应用等你来开发!
欢迎大家使用、提Issue和Star一下~
安装方法:
pip install mindnlp代码仓: https://github.com/mindspore-lab/mindnlp
PS:获取昇腾算力的几个途径:
- 参加MindSpore社区实习和社区活动,免费领取华为云代金券:https://gitee.com/mindspore/community/issues/IAUJE8
- 使用启智社区免费算力:https://www.openi.org.cn/
- v0.3.12024-05-17 · 2024-05-17 02:08 UTC
查看更新说明
What's Changed
- add musicgen gradio demo by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1066
- Peft lokr support by @ultranationalism in https://github.com/mindspore-lab/mindnlp/pull/1067
- use mkdocs to generate documentation by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1070
- Add tutorial quick start by @hubosynapse in https://github.com/mindspore-lab/mindnlp/pull/1069
- fix precision error of flash_attn kernel in some case by @hypertseng in https://github.com/mindspore-lab/mindnlp/pull/1072
- support prompt_tuning by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1073
- add peft integration by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1076
- implement cogvlm model and demo by @antigone660 in https://github.com/mindspore-lab/mindnlp/pull/1075
- fix pipeline base error by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1077
- peft module documentation completion by @ultranationalism in https://github.com/mindspore-lab/mindnlp/pull/1078
- Add MindNLP Starcoder vscode demo by @xing-yiren in https://github.com/mindspore-lab/mindnlp/pull/1079
- fix starcoder example by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1080
- update docs by @lvyufeng in https://github.com/mindspore-lab/mindnlp/pull/1084
New Contributors
- @ultranationalism made their first contribution in https://github.com/mindspore-lab/mindnlp/pull/1067
- @antigone660 made their first contribution in https://github.com/mindspore-lab/mindnlp/pull/1075
- @xing-yiren made their first contribution in https://github.com/mindspore-lab/mindnlp/pull/1079
Full Changelog: https://github.com/mindspore-lab/mindnlp/compare/v0.3.0...v0.3.1
用户点评
登录用户的真实使用记录,可按版本和硬件筛选还没有人点评。第一个说出真实体验的人最有价值。
MindAct
A PyTorch and Hugging Face native training and evaluation toolkit for embodied policies
MindAct is a reproducible training and evaluation framework for imitation learning policies on desktop manipulation tasks. It integrates LeRobot datasets and policies with LIBERO simulation benchmarks, providing experiment provenance tracking and standardized evaluation protocols.
Features
- Protocol-based architecture: Minimal adapter interfaces for datasets, policies, and environments
- Reproducible experiments: YAML configurations with manifest-tracked provenance
- Optional dependencies: Lazy loading of torch, lerobot, and libero via extras
- Artifact management: Conventional directory structure for checkpoints, logs, and results
- Type-safe: Frozen dataclasses and runtime protocols throughout
Quick Start
Installation
# Basic installation (configuration and CLI only)
pip install -e .
# With PyTorch
pip install -e ".[torch]"
# With LeRobot datasets and policies
pip install -e ".[lerobot]"
# With LIBERO simulation environments
pip install -e ".[libero]"
# All integrations
pip install -e ".[torch,lerobot,libero]"
# Development tools
pip install -e ".[dev]"
Run an Example
# Validate configuration
mindact config-check configs/experiments/libero-baseline.yaml
# Run the dependency-free evaluation smoke path
mindact eval configs/experiments/libero-baseline.yaml \
--runner fake \
--run-id smoke-run \
--output-dir outputs \
--episodes 2
config-check validates a configuration, and eval --runner fake exercises the full evaluation lifecycle with built-in test doubles. Real LeRobot policies and LIBERO environments arrive with their adapter implementations; the fake runner is a contract smoke test, not a benchmark result.
Configuration Format
name: libero-baseline
seed: 42
output_dir: outputs
dataset:
repo_id: lerobot/aloha_sim_insertion_human
revision: null
split: train
policy:
name: act
pretrained_model: null
environment:
name: libero
task_suite: libero_spatial
task_ids: []
training:
steps: 10000
batch_size: 8
learning_rate: 0.0001
log_every: 100
checkpoint_every: 2000
evaluation:
episodes: 20
max_steps: 500
record_video: false
Project Status
MindAct v0.1 is the initial skeleton release. Core interfaces and configuration system are stable. Implementation priorities:
Phase B (current): Unified reproducible training and evaluation
Phase A (future): Trajectory quality diagnostics
See docs/ for architecture details and contribution guidelines.
Repository History
This repository was originally MindNLP, a MindSpore-based NLP library. The legacy codebase is preserved in the legacy branch. MindAct represents a complete pivot to embodied AI with PyTorch and Hugging Face as the native stack.
License
Apache License 2.0. See LICENSE and NOTICE for details.