Star Trek combadge voice client for the JMJMJ mesh (TASK-051)
  • JavaScript 75%
  • CSS 11.2%
  • HTML 8.1%
  • C++ 5.7%
Find a file
Jeff Emmett 076377a8e5 Merge rHunt combadge voice intents into dev (TASK-053)
5 hunt voice intents over the holon substrate + tests (12/12). Read path
verified live on demo; write path pending rhunt:* builders in rspace-online.
2026-06-13 22:55:40 +00:00
app crew v2: alice-bot/bob-bot/charlie-bot with queryable JMJMJ footprints 2026-06-10 17:44:26 +02:00
backend rcom: rHunt voice intents — treasure-hunt ops over the holon substrate 2026-06-13 18:36:03 +00:00
bots crew: singleflight token mint + safe WS error close 2026-06-10 18:17:28 +02:00
firmware rCom scaffold: backend brain (P1+P3), firmware (P1/P2), PWA (P5) 2026-06-10 11:56:34 +02:00
.gitignore rCom scaffold: backend brain (P1+P3), firmware (P1/P2), PWA (P5) 2026-06-10 11:56:34 +02:00
docker-compose.prod.yml prod intent: spark GPU ollama (containers cannot reach netcup's own tailscale IP) 2026-06-10 17:31:37 +02:00
docker-compose.yml rCom scaffold: backend brain (P1+P3), firmware (P1/P2), PWA (P5) 2026-06-10 11:56:34 +02:00
README.md rCom scaffold: backend brain (P1+P3), firmware (P1/P2), PWA (P5) 2026-06-10 11:56:34 +02:00

rCom — Star Trek combadge voice client for the JMJMJ mesh

Double-tap a 3D-printed pin → speak → the request hits JMJMJ group info (send a message, ask where someone is, query the group, raise a prompt) → spoken answer in your earpiece.

Backlog epic: TASK-051 (phases P1P5).

Architecture (Tier A — phone-tethered, locked 2026-06-10)

double-tap (cap-touch)            P2 firmware
   │  BLE wake
   ▼
phone app  ── captures audio ──►  backend "brain"            P5 app + P1/P3 backend
                                    │
                                    ├─ STT      whisper @ spark GX10
                                    ├─ intent   LiteLLM (qwen/llama) → structured op   P3
                                    ├─ action   rspace-online JMJMJ API                P3
                                    └─ TTS      piper @ spark GX10
                                    ▼
phone app  ◄── response audio ──  earpiece (BT-paired to PHONE)

The pin is a dumb BLE peripheral; the phone holds JMJMJ identity and runs the pipeline. Earpiece pairs to the phone, not the pin — this sidesteps the ESP32 classic-BT-audio blocker. Tier B (standalone) is deferred.

Layout

Dir Phase What
backend/ P1 + P3 The brain. STT → intent → JMJMJ action → TTS. Runs now in mock mode with zero hardware/cloud.
firmware/ P1 + P2 PlatformIO ESP32-S3: I2S mic capture, cap-touch double-tap, chirp/haptic, BLE/WiFi uplink.
app/ P5 PWA: Web Bluetooth pairing + Web Audio capture/playback + backend session.

Quick start (no hardware, no cloud)

cd backend
npm install          # zero runtime deps; installs nothing but sets up scripts
npm test             # text-in → intent → JMJMJ action, fully mocked
npm start            # http://localhost:8787  (MOCK mode until you set the env URLs)

# drive it by text (simulates a transcript):
curl -s localhost:8787/interact -H 'content-type: application/json' \
  -d '{"text":"where is Riker","sessionId":"demo"}' | jq
curl -s localhost:8787/interact -H 'content-type: application/json' \
  -d '{"text":"send a message to Picard saying meet at ten","sessionId":"demo"}' | jq
# the send is mutating → you get a pendingId; confirm it:
curl -s localhost:8787/confirm -H 'content-type: application/json' \
  -d '{"sessionId":"demo","confirm":true}' | jq

Going live

Set these in backend/.env (see .env.example). Any unset → that stage stays mocked.

  • STT_URL — whisper (OpenAI-compatible /v1/audio/transcriptions) on spark GX10
  • TTS_URL — piper HTTP endpoint on spark GX10
  • LITELLM_URL / LITELLM_KEY / LITELLM_MODEL — intent parsing
  • RSPACE_API_BASE / RSPACE_TOKEN — JMJMJ ops on rspace-online

The JMJMJ endpoint paths in backend/src/jmjmj/client.mjs are marked TODO: — wire them to the real rspace-online routes (rmesh send, location holon, group-info, rChoices) when those are confirmed.