B

─── B ───

The Soul-Wired Desktop Companion

B is not just an AI : he is a digital lifeform designed to live on your desktop. Built with a soul-first architecture, B observes your workflow, listens with neural precision, and interacts through a high-performance, glassmorphic overlay.

Explore the Vision • Architecture • Getting Started • Contributing • Privacy

Demo

B in action - watch him observe, think, and respond.

---

The Vision

Traditional assistants wait for a command. B waits for a moment.

B is designed as a proactive desktop companion. Using a 60fps event-driven central nervous system, B synchronizes his emotional state with your environment. He sees your screen semantically, tracks your active focus, and intervenes only when he has something truly valuable to contribute.

• Silent by Default : B respects your deep work state.
• Emotionally Aware : A complex internal state machine drives expressions and curiosity.
• Agentic Autonomy : B doesn't just respond; he thinks, wonders, and observes.

---

Architecture Overview

B is built on a centralized asynchronous pub/sub event bus : the EventBus. Every module communicates exclusively through this bus. No module knows about any other module. This strict decoupling makes the system testable, maintainable, and resilient.

flowchart TB
    subgraph P["Perception Layer"]
        WT["WindowTracker"]
        SS["SemanticSensor"]
        VS["VisionSensor OCR"]
    end

    subgraph C["Cognitive Layer"]
        CE["CognitiveEngine"]
        SM["StateMachine"]
        AE["AutonomyEngine"]
    end

    subgraph O["Output Layer"]
        VE["VoiceEngine"]
        FR["FaceRenderer"]
        CB["ChatBubble"]
        KE["KinematicsEngine"]
    end

    subgraph I["Core Infrastructure"]
        EB["EventBus"]
    end

    WT -- "active_window_changed" --> EB
    SS -- "context_updated" --> EB
    VS -- "context_updated" --> EB
    EB -- "context_updated" --> CE
    EB -- "tick" --> SM
    EB -- "tick" --> KE
    CE -- "b_spoke" --> EB
    EB -- "b_spoke" --> VE
    EB -- "b_spoke" --> FR
    EB -- "b_spoke" --> CB
    CE -- "b_move_request" --> EB
    EB -- "b_move_request" --> KE
    AE -- "trigger_proactive_thought" --> EB
    EB -- "trigger_proactive_thought" --> CE

---

System Workflow

The data flows through B in a deterministic pipeline: Perception → Cognition → Expression.

sequenceDiagram
    participant WT as WindowTracker
    participant SS as SemanticSensor
    participant VS as VisionSensor
    participant CE as CognitiveEngine
    participant VE as VoiceEngine
    participant FR as FaceRenderer

    WT->>SS: active_window_changed
    activate SS
    SS->>SS: UIA Tree Walk
    alt Extraction Success
        SS->>CE: context_updated
    else Failure / Cooldown
        SS->>VS: semantic_extraction_failed
        VS->>VS: OCR Capture
        VS->>CE: context_updated
    end
    deactivate SS

    CE->>CE: LLM Inference
    CE->>VE: b_spoke (Text + Emotion)
    activate VE
    VE->>VE: TTS + DSP Vocoder
    VE->>FR: speaking_start
    VE->>VE: Audio Output
    VE->>FR: speaking_end
    deactivate VE
    CE->>FR: b_move_request (Spatial)

---

Internal Module Structure

Core Infrastructure

Module	File	Responsibility
EventBus	core/bus.py	Thread-safe pub/sub message broker. All inter-module communication flows through this.
main.py	main.py	Boot sequence : instantiates all modules, starts the 60fps tick timer, registers global hotkeys.

Perception Layer (Sensors)

Module	File	Responsibility
WindowTracker	sensors/window_tracker.py	Hook-based active window change detection. Fires when the user switches focus.
SemanticSensor	vision/semantic.py	UIA-based DOM/window tree walking. Extracts structured content with quality scoring and adaptive cooldown.
VisionSensor	vision/mss_capture.py	OCR fallback pipeline using MSS + Tesseract for frameworks incompatible with UIA.

Cognitive Layer (The Brain)

Module	File	Responsibility
CognitiveEngine	brain/llm.py	LLM inference orchestration (Groq cloud or local llama-cpp). Manages context, history, spatial mapping, and streaming token parsing.
StateMachine	brain/soul.py	B's emotional state : blinking, resting, conversing. Real-time stream buffer that parses LLM output into sentences.
AutonomyEngine	brain/autonomy_loop.py	Proactive thought scheduling : decides when B should speak unprompted based on context quality and timing.

Output Layer (Expression)

Module	File	Responsibility
VoiceEngine	audio/speaker.py	Piper ONNX TTS with DSP vocoder chain (pitch shift, bitcrush, chorus) for robotic modulation.
FaceRenderer	ui/face.py	PyQt6 QPainter-based hardware-accelerated face rendering at 60fps.
ChatBubble	ui/chat.py	Glassmorphic chat overlay that displays B's spoken text.
KinematicsEngine	physics/kinematics.py	Physics-based movement with Bezier path interpolation and easing curves.
EarsSensor	audio/ears.py	Speech-to-text via Faster-Whisper with neural VAD.

---

Data Flow

flowchart LR
    subgraph Input["Input"]
        A["User Types"]
        B["User Speaks"]
        C["Screen Changes"]
    end

    subgraph Process["Processing"]
        D["EventBus"]
        E["CognitiveEngine"]
        F["StateMachine"]
    end

    subgraph Output["Output"]
        G["FaceRenderer"]
        H["VoiceEngine"]
        I["ChatBubble"]
        J["Kinematics"]
    end

    A -- types --> D
    B -- speaks --> D
    C -- changes --> D
    D -- routes --> E
    E -- drives --> F
    F -- animates --> G
    F -- speaks --> H
    F -- displays --> I
    E -- moves --> J

---

Request Lifecycle

stateDiagram-v2
    [*] --> Idle
    Idle --> Listening : user_spoke / voice detected
    Listening --> Thinking : 1.2s delay
    Thinking --> Streaming : first token received
    Streaming --> Speaking : sentence_ready
    Speaking --> Streaming : next sentence
    Streaming --> Idle : [SILENCE] / end of response
    Speaking --> Idle : finished_speaking + linger
    Idle --> Proactive : autonomy trigger
    Proactive --> Thinking : context available
    Proactive --> Idle : no context / silence

---

Project Structure

B/
├── main.py                  # Entry point : boot sequence
├── core/
│   └── bus.py               # EventBus : central nervous system
├── brain/
│   ├── llm.py               # CognitiveEngine : LLM inference
│   ├── soul.py              # StateMachine : emotions & stream buffer
│   ├── autonomy_loop.py     # AutonomyEngine : proactive thought
│   ├── context.py           # Context management
│   └── work_mode.py         # Work mode prompt templates
├── vision/
│   ├── semantic.py          # SemanticSensor : UIA extraction
│   └── mss_capture.py       # VisionSensor : OCR fallback
├── sensors/
│   └── window_tracker.py    # WindowTracker : focus detection
├── audio/
│   ├── speaker.py           # VoiceEngine : TTS + DSP
│   └── ears.py              # EarsSensor : STT
├── physics/
│   └── kinematics.py        # KinematicsEngine : movement
├── ui/
│   ├── overlay.py           # WindowManager : transparent overlay
│   ├── face.py              # FaceRenderer : 60fps face
│   ├── chat.py              # ChatBubble : text overlay
│   ├── input_box.py         # InputBox : text input
│   ├── expressions.py       # Expression definitions
│   └── theme.py             # Visual theming
├── models/                  # Local GGUF models (gitignored)
├── voices/                  # Piper ONNX voice models
├── scripts/                 # Setup & download utilities
├── docs/
│   ├── ARCHITECTURE.md      # Detailed architecture docs
│   └── assets/              # Images & diagrams
├── .env                     # API keys (gitignored)
└── requirements.txt         # Python dependencies

---

Core Systems

System	Technology	Description
Cognitive Engine	Groq API / llama-cpp	Cloud or local LLM inference for private, high-speed reasoning.
Semantic Vision	UIAutomation + Tesseract OCR	B understands the context of your active windows and screen content.
Neural Hearing	Faster-Whisper	Industry-grade transcription with neural VAD for reliable ears.
Vocal Synthesis	Piper TTS + Pedalboard DSP	Low-latency, natural-sounding voice with robotic modulation effects.
Kinematics	PyQt6 QPropertyAnimation	Smooth, 60fps movement with Bezier path interpolation and easing curves.
Event Bus	PyQt6 pyqtSignal	Thread-safe pub/sub message broker : the central nervous system.

---

Agentic Work Mode

Activated via Ctrl+Shift+Alt+W, Work Mode shifts B into a high-utility state:

• Semantic Monitoring : B monitors your progress on tasks in real-time.
• Contextual Curiosity : Proactively offers insights, documentation, or suggestions based on your current focus.
• Minimalist Presence : Dims facial expressions to minimize distraction while remaining vigilant.

flowchart TB
    A["User presses Ctrl+Shift+Alt+W"] --> B["B asks: what are you working on?"]
    B --> C["User defines goal"]
    C --> D["B monitors screen context"]
    D --> E{"Relevant content?"}
    E -- "Yes" --> F["B offers insight / help"]
    E -- "No" --> G["B stays silent"]
    F --> D
    G --> D

---

Hotkeys

Shortcut	Action
Ctrl+Shift+Alt+Q	Kill switch : immediately terminates B and releases all system hooks.
Ctrl+Shift+Alt+B	Toggle input box : type messages to B.
Ctrl+Shift+Alt+V	Toggle speak mode : talk to B via microphone.
Ctrl+Shift+Alt+W	Toggle work mode : B becomes a proactive assistant.

---

Installation

1. Clone the repository:

   git clone https://github.com/Ahmad-Hassan-0/B---desktop-companion.git
   cd B---desktop-companion
   

2. Set up the environment:

   python -m venv venv
   venv\Scripts\activate      # Windows
   source venv/bin/activate   # Linux/macOS
   pip install -r requirements.txt
   

3. Configure API keys:

   cp .env.example .env
   # Edit .env with your Groq API key (get one at https://console.groq.com/)
   

4. Awaken B:

   python main.py
   

---

Safety

[!CAUTION] Global Kill Switch: Ctrl+Shift+Alt+Q This hotkey immediately terminates B and releases all system hooks. Use this if B becomes over-eager or if you need an instant exit.

Because B lives on a transparent, click-through overlay without a standard close button, the kill switch is the only way to exit. It is registered at the Win32 level and works even if the Qt event loop is unresponsive.

---

Technical Constraints

Constraint	Target
CPU	Intel i5 (Quad-Core) or equivalent
RAM	16 GB
Display	Any resolution (adaptive)
OS	Windows 10/11 (primary), Linux/macOS (experimental)
Tick Rate	60 fps (16ms interval)
Inference	Groq API (cloud) or llama-cpp (local, 4GB+ model)

---

Built by Ahmad Hassan

Wiring the soul, one tick at a time.