# GhostAI — prototype 0.5.6

Save your notes once in **Settings → Notes and answers**, then Save settings. The default instruction is **Answer the question given.** Both selected text and screenshot questions use those saved notes automatically.

- Highlight a question and press **Alt+Shift+Q**. This also opens the small controller if needed.
- For a screenshot question, press **Alt+Shift+C**, then drag a rectangle over just the question.
- Wait for “Answer ready,” click where the answer belongs, then Play. Type to advance; Escape pauses.
- When the pill is already open, **F8 → Q** answers selected text and **F8 → C** captures a screenshot. Change these sequences in Settings. Change the browser-wide combinations through Set Chrome's launch shortcut.

The answer is generated and loaded automatically. Opening the percentage panel is optional, for reviewing/editing an answer. The panel never opens automatically during this shortcut flow. A configured model is required for automatic answers; sample mode remains available in the optional panel.

The **108 × 30 px** controller has transparent Play/Pause and percentage backgrounds in light and dark mode. Color and small underlines indicate active/paused states. Drag the percentage to move; click to open the optional panel. Right-click for restart, hide and close. Choose Light, Dark or System in Settings.

Standard fields read their selected range and preserve it when you click GhostAI controls. Docs' canvas selection is captured through the paired Windows helper using a short native Copy command. It waits for modifiers to be released, requires the active Docs editor, reads only the newly copied selection and attempts to restore the previous clipboard formats. If selection cannot be copied, the pill reports an error and you can use screenshot capture. The new Windows Copy/clipboard path still needs a live test on your PC.

Notes remain saved locally until edited or defaults are restored. Every answer request sends the saved notes, chosen question/image and instruction to your configured AI provider. Notes are limited to 50,000 characters; the model is instructed to use relevant notes, avoid invented facts and identify insufficient notes. This is prompt-based grounding, not an independent guarantee of accuracy.

## Updating from installed 0.4.0

Extract this update, run **INSTALL-GHOSTAI.cmd**, then click **Reload** on GhostAI in `chrome://extensions` and refresh Docs. Keep the installed extension at `%LOCALAPPDATA%\GhostAI\app\extension` so your settings and notes remain under the same extension identity. The installer keeps the existing background connection.

## One-time Windows setup

Requires Windows 10/11, desktop Chrome and Node.js 22+ for the first prototype setup. You already have a suitable Node version if you used 0.3.0 successfully. Setup copies the existing Node runtime into the installed helper, so you do not need a terminal or a globally installed Node runtime afterward.

1. Close the old black backend window. Extract **all** of this ZIP into a new folder.
2. Double-click `INSTALL-GHOSTAI.cmd`, then choose Install GhostAI in the wizard. Leave **Start GhostAI when I sign into Windows** checked for automatic startup.
3. Remove the old GhostAI extension in `chrome://extensions`. Enable Developer mode and choose Load unpacked.
4. Load the installed folder: `%LOCALAPPDATA%\GhostAI\app\extension`. Setup opens this folder for you. Use this installed copy so deleting the extracted ZIP folder will not break your extension.
5. Refresh Google Docs. GhostAI connects automatically when installed or when you press Play. There is no pairing code to enter. Settings → Connect helper can retry setup without resetting your preferences.

The setup command window is used only during installation and closes after the wizard. Daily use has no command window. The tray icon offers Pause typing, Start with Windows, a helper log and Quit. Start-menu GhostAI reopens the background app if you quit it. Playing in Docs can also start the helper on demand.

The helper runs under your Windows account with no elevation. Setup creates a per-user Chrome native-host registration, private connection state, installed app files, Start-menu shortcuts and an optional Windows sign-in startup entry. Uninstall GhostAI in the Start menu removes these; remove the Chrome extension separately. The running uninstaller itself remains available to delete afterward.

This is a source-based development installer that builds the small Windows executables on your PC. Windows installation/tray/native-message behavior has not been run in this Linux workspace. A public customer release still needs Windows testing, a signed packaged installer with a supplied runtime, and Chrome Web Store distribution. Customers then install the helper and Chrome extension once.

## Browser demo / developer server

For the standard field demo, run `node backend/server.mjs` inside `ghostai`, then open `http://127.0.0.1:8787`. Click percentage → Load sample → Use draft → click writing field → Play. Sample mode needs no AI key. The console launcher is retained for development only; it is not needed after Windows setup. Close this server before starting the installed helper because both use port 8787.

## Shortcuts

| Action | Default sequence |
| --- | --- |
| Start / open assistant | F8 → S |
| Play | F8 → P |
| Pause | F8 → A |
| Hide / show | F8 → H |
| Power on | F8 → O |
| Power off | F8 → X |
| Answer selected question | F8 → Q |
| Answer screenshot question | F8 → C |
| Generate draft | F8 → G |

Click a shortcut in Settings, press up to three keystrokes, then Enter and Save. Each step may also be a combination, e.g. Ctrl+Shift+P. Press consecutive steps within one second. Duplicate/prefix conflicts are rejected. Start with a function key or Ctrl/Alt/Command to avoid capturing normal typing. Escape always stops entry.

Custom sequences work in tabs where you already opened the assistant and last until a page reload. Browser/system shortcuts take priority. Chrome's initial launch shortcut is Ctrl+Shift+Y (Command+Shift+Y on Mac); change this global launch combination in `chrome://extensions/shortcuts`. Chrome does not support arbitrary multi-step launch sequences on uninitialized tabs.

Hide pauses entry and preserves the draft. Power off stops capture, model requests, Forms helpers and entry, clears current context/draft, and disconnects Docs mode. It discards results of requests already sent; it cannot cancel their provider charges. A native insertion already received by Chrome may finish. Power shortcuts remain available in initialized tabs.

## Forms

Open GhostAI on a Google Form. Select a question and press Alt+Shift+Q, or capture it with Alt+Shift+C. Review the generated draft, then enter it in the form. No buttons are added beneath questions and nothing is submitted automatically.

Helpers read question labels, not existing typed responses. Google may change the page structure; see verification limitations.

## Google Docs · Windows background companion

After setup, select a question in a writable Google Doc → Alt+Shift+Q → wait for the prepared answer → click its destination → Play. Screenshot capture uses Alt+Shift+C. The optional panel still provides Load sample and manual draft editing. Typing letters, numbers and punctuation advances the reviewed draft. Escape or Pause stops entry. A short error above the pill explains connection/focus failures.

The extension uses Chrome native messaging to get its connection to the installed companion. Chrome's allowed-origin registration is limited to this extension's stable ID. The connection survives helper and Windows restarts; the OS keyboard session itself is always paused on startup. Version 0.4.0 fixed the previous “Unknown request” pairing failure when Settings has a tab sender.

The helper replaces physical typing keys with the next approved text chunk and ignores its own injected input. Switching tabs/windows, changing the focused window's title, hiding the pill, power off, Escape or a lost heartbeat pauses entry. Keyboard shortcuts pause native entry and pass through. The extension checks editor focus before Play and during progress polling. These guards and Windows input delivery still need a live test on your PC.

The helper retains only the approved draft and playback position. It never records physical key values or sends keystrokes to the AI. The installer preserves an opaque connection token in a user-protected data folder. No Chrome debugger connection, browser-warning flags, administrator rights or execution-policy bypass are used. Managed-device restrictions may prevent installation or keyboard entry.

Progress counts chunks accepted by Windows input delivery; it does not independently verify document contents. Test a sample in a disposable document. macOS/Linux still support the demo and standard DOM fields, but this companion is for Windows.

## Connect AI

Set `AI_API_KEY` and `AI_MODEL` in the backend environment. Choose an API-account model supporting image input and Chat Completions. API credits are separate from ChatGPT subscriptions. Never put a provider key in the extension or in this chat.

Windows PowerShell:

```powershell
$secret = Read-Host 'API key' -AsSecureString
$env:AI_API_KEY = [System.Net.NetworkCredential]::new('', $secret).Password
$env:AI_MODEL = 'YOUR_VISION_CAPABLE_MODEL_ID'
node backend/server.mjs
```

macOS/Linux bash:

```bash
read -s -p 'API key: ' AI_API_KEY
export AI_API_KEY
export AI_MODEL='YOUR_VISION_CAPABLE_MODEL_ID'
node backend/server.mjs
```

Optional `AI_API_URL` changes the default `https://api.openai.com/v1/chat/completions` to another compatible endpoint. Keep this backend on loopback. Public deployment needs authentication and per-user quotas.

Context/drafts stay in memory and clear on close/power off. Settings and the paired helper token persist; the installed companion token survives restarts. The development server still uses temporary pairing. Generate sends the instruction and chosen context/image. Page context is limited to 20,000 characters.

## Compatibility and checks

DOM entry supports standard textareas, text/search/url/tel inputs and simple contenteditable fields. Password, email, payment and number fields are excluded. Framework-controlled rich editors need separate tests. Restart resets the buffer and leaves inserted text intact. Automatic key repeat/composition do not advance.

See `VERIFICATION.md` for checks run and remaining manual checks. `node tests/smoke.mjs` tests the backend; `node tests/shortcuts.mjs` tests sequences; `node tests/native.mjs` and `node tests/worker.mjs` test automatic connection and playback transport with mocks; `node tests/install-contract.mjs` checks the stable extension identity; `node tests/selection.mjs` checks range extraction. The smoke test verifies notes reach the model request and invalid notes are rejected. To run browser fixture checks, stop any existing backend, `npm install --no-save playwright`, `npx playwright install chromium`, then `node tests/ui.mjs`. This test starts its own backend.

Official reference APIs: https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendinput , https://developer.chrome.com/docs/extensions/reference/api/commands , https://developer.chrome.com/docs/extensions/reference/api/permissions .

Chrome native-host reference: https://developer.chrome.com/docs/extensions/develop/concepts/native-messaging .

0.5.1: Screenshot selection has no page tint or interior fill. Only a faint 1 px border remains, switching between dark and light neutral colors based on the captured region. The controller uses local page-background contrast at low idle opacity and becomes clearer on hover/focus. Image/gradient backgrounds may require moving it to a simpler area.

0.5.2: Settings has a dedicated Generate draft binding (F8 → G), a browser-wide Alt+Shift+G command, and a Generate draft item in the small controller menu. It regenerates from the most recently captured question/image and current saved notes. With no captured context, it captures the current selection first. No large panel is required.

Integrated controls are scoped to Google Docs and Google Forms: Docs uses its formatting toolbar when available; a Forms responder page uses a compact row at the top of its form. Other websites keep the standalone floating controller. If the target toolbar is absent, placement falls back to the movable controller. Dragging integrated controls detaches them for the current page. Live Google layout verification is still pending.

0.5.3: Settings gears in the popup and optional drafting panel use 26 px icons with 42 × 42 px click targets. Header spacing accommodates the larger button.

0.5.4: Apply settings and Restore defaults stay in a fixed action bar while scrolling; settings feedback appears above that bar. Extra bottom spacing keeps the last fields reachable.

0.5.5: Checks appear below the percentage without enlarging the controller: ✓ question captured (text or screenshot), ✓✓ draft ready, ✓✓✓ entry complete. New capture resets the checks; restarting a completed buffer returns to two checks; power off/close clears them. Completion reflects accepted typing chunks, not independent document verification.

## Account, Premium and embedded helper setup (0.5.6)

Settings has a Run this first card with an embedded downloadable Windows helper ZIP. Extract all files and run INSTALL-GHOSTAI.cmd; Node.js 22+ is still required for the first beta installation. Connect helper reports the actual connection result. The Chrome Web Store does not install or run Windows software. The source installer is a beta; a public release needs a signed, packaged runtime and testing on Windows.

Account and Premium cards are linked from the popup. Premium prices are USD $9.99/month, $34.99/4 months, and $89.99/year. A promo code field is present. Sign-in, promo validation and checkout are explicitly unavailable until hosted services are connected. Existing beta features are not newly gated. Plan selections and typed promo codes do not confer access or collect payment. Final recurring terms, taxes, cancellation/refunds and usage limits must be shown at live checkout.

See launch/RELEASE-CHECKLIST.md for draft Chrome Web Store upload and launch/BILLING-BACKEND.md for remaining account, AI and payment integration work.


## 0.6.0 · hosted AI and Mac developer preview
AI generation now uses the hosted GhostAI endpoint and the account signed into extension Settings. An active Premium subscription is checked server-side on every request. Manual drafts and controls remain free. Website sign-in and extension sign-in are separate sessions; sign in once in each. Your provider key belongs only in Supabase Edge Function Secrets, not in the helper or extension.

Mac preview: extract the package and run INSTALL-GHOSTAI.command. Requires macOS 12+, Node.js 22+, and Apple Command Line Tools for compiling this source preview. Setup creates a menu-bar app, Chrome native-messaging connection, and a per-user login agent. Grant Accessibility and Input Monitoring to GhostAI, then restart its helper from the G menu. Use Chrome (other Mac browsers are not supported by this preview). The preview has not been compiled or tested on macOS here. A signed, notarized installer without build-tool requirements is still needed for a customer release. Never disable Gatekeeper to install it.

To uninstall the Mac helper, run macos/UNINSTALL-GHOSTAI.command. This removes the helper and its installed extension folder, so remove that unpacked extension from Chrome too.