mirror of
https://gitea.jeffemmett.com/jeffemmett/rcom.git
synced 2026-06-25 16:35:24 +02:00
Star Trek combadge voice client for the JMJMJ mesh (TASK-051)
- JavaScript 75%
- CSS 11.2%
- HTML 8.1%
- C++ 5.7%
5 hunt voice intents over the holon substrate + tests (12/12). Read path verified live on demo; write path pending rhunt:* builders in rspace-online. |
||
|---|---|---|
| app | ||
| backend | ||
| bots | ||
| firmware | ||
| .gitignore | ||
| docker-compose.prod.yml | ||
| docker-compose.yml | ||
| README.md | ||
rCom — Star Trek combadge voice client for the JMJMJ mesh
Double-tap a 3D-printed pin → speak → the request hits JMJMJ group info (send a message, ask where someone is, query the group, raise a prompt) → spoken answer in your earpiece.
Backlog epic: TASK-051 (phases P1–P5).
Architecture (Tier A — phone-tethered, locked 2026-06-10)
double-tap (cap-touch) P2 firmware
│ BLE wake
▼
phone app ── captures audio ──► backend "brain" P5 app + P1/P3 backend
│
├─ STT whisper @ spark GX10
├─ intent LiteLLM (qwen/llama) → structured op P3
├─ action rspace-online JMJMJ API P3
└─ TTS piper @ spark GX10
▼
phone app ◄── response audio ── earpiece (BT-paired to PHONE)
The pin is a dumb BLE peripheral; the phone holds JMJMJ identity and runs the pipeline. Earpiece pairs to the phone, not the pin — this sidesteps the ESP32 classic-BT-audio blocker. Tier B (standalone) is deferred.
Layout
| Dir | Phase | What |
|---|---|---|
backend/ |
P1 + P3 | The brain. STT → intent → JMJMJ action → TTS. Runs now in mock mode with zero hardware/cloud. |
firmware/ |
P1 + P2 | PlatformIO ESP32-S3: I2S mic capture, cap-touch double-tap, chirp/haptic, BLE/WiFi uplink. |
app/ |
P5 | PWA: Web Bluetooth pairing + Web Audio capture/playback + backend session. |
Quick start (no hardware, no cloud)
cd backend
npm install # zero runtime deps; installs nothing but sets up scripts
npm test # text-in → intent → JMJMJ action, fully mocked
npm start # http://localhost:8787 (MOCK mode until you set the env URLs)
# drive it by text (simulates a transcript):
curl -s localhost:8787/interact -H 'content-type: application/json' \
-d '{"text":"where is Riker","sessionId":"demo"}' | jq
curl -s localhost:8787/interact -H 'content-type: application/json' \
-d '{"text":"send a message to Picard saying meet at ten","sessionId":"demo"}' | jq
# the send is mutating → you get a pendingId; confirm it:
curl -s localhost:8787/confirm -H 'content-type: application/json' \
-d '{"sessionId":"demo","confirm":true}' | jq
Going live
Set these in backend/.env (see .env.example). Any unset → that stage stays mocked.
STT_URL— whisper (OpenAI-compatible/v1/audio/transcriptions) on spark GX10TTS_URL— piper HTTP endpoint on spark GX10LITELLM_URL/LITELLM_KEY/LITELLM_MODEL— intent parsingRSPACE_API_BASE/RSPACE_TOKEN— JMJMJ ops on rspace-online
The JMJMJ endpoint paths in backend/src/jmjmj/client.mjs are marked TODO:
— wire them to the real rspace-online routes (rmesh send, location holon,
group-info, rChoices) when those are confirmed.