The Open Standard for Physical AI
Artificial intelligence has mastered text boxes and code IDEs. FieldKit is the open architectural paradigm that brings intelligence into the physical, chaotic, hands-busy real world.
FieldKit Real-World Operating Architecture
Deterministic data flow between edge multimodal ingest, spatial grounding, on-device biometric security, and proactive voice dispatch.
Raw camera frames, binaural audio streams, and 9-DoF IMU kinematics captured from smart glasses (Ray-Ban Meta, Android wearables), chest cams, or drones with real-time motion-stabilized ROI cropping.
21-point skeletal hand landmark raycasting against 3D volumetric targets. Evaluates pinch gestures, contact duration, and tool alignment against the deterministic mission DAG.
Hardware-backed encryption enclave. Spectral near-field voice gating isolates wearer audio against ambient machinery noise. Zero unconfirmed cloud streaming of raw video or voiceprints.
Synthesizes step completion from spatial grounding and speaks the next physical directive through bone-conduction audio before the operator has to ask. Hands-free operational autonomy.
The Shift: Desktop AI vs. Physical AI
Software engineers build for desktop assumptions. Real frontline operators work in environments where typing is impossible.
| Dimension | Desktop AI (Status Quo) | FieldKit Physical AI |
|---|---|---|
| User Posture | Seated at desk, staring at monitor | Standing, walking, repairing, greasy hands |
| Interaction Loop | Reactive: User must type or tap first | Proactive: AI detects progress & speaks next step |
| Sensory Input | Static files, screenshots, clipboard | Dynamic first-person video, binaural audio, IMU |
| Network Reality | Gigabit unmetered fiber / stable Wi-Fi | Jittery cellular, intermittent edge, local-first fallback |
| Privacy Boundary | Everything uploaded to cloud server | Ephemeral edge processing, confirm-gated streaming |
The Four Pillars of FieldKit
Every compliant FieldKit runtime adheres to four core invariants designed for frontline safety and operator autonomy.
1. First-Person Sensory Ingest
Ingests real-time video and audio from smart glasses (Ray-Ban Meta, Android wearables), chest cams, and inspection drones with motion-stabilized ROI cropping.
2. Proactive Anticipation
Instead of waiting for a wake-word or query, FieldKit tracks state changes against a mission checklist and speaks the next instruction before you have to ask.
3. Sovereign Privacy Vault
Zero passive recording. Video feeds remain local to edge memory. Audio streams are isolated to the wearer's voiceprint, and external transmissions require strict confirmation.
4. Resilient Local-First Edge Execution
Designed to function when cell towers drop. High-frequency safety checks and step validations execute on-device or via local relay, syncing state seamlessly when uplink restores.
Mission-Driven State Machine
Breaks complex tasks down into deterministic atomic checkpoints. If a step is skipped or performed out of order, FieldKit alerts the operator immediately.
Open-Access Mission Templates
Production-ready YAML task schemas ready to deploy into wearables, inspection drones, and field laptops.
Industrial Pump Overhaul
Lockout/Tagout (LOTO) 480V electrical verification, manifold zero-pressure gauge checks, star-pattern casing disassembly, and silicon-carbide mechanical seal seat inspection.
TCCC Care Under Fire
Rapid extremity arterial hemorrhage control: threat suppression verification, rapid wound identification, high-and-tight CAT tourniquet placement, and windlass torque lock.
Datacenter Tamper Sweep
Physical layer security audits: biometric cage door latch verification, cold-aisle server tray tamper seal checks, inline hardware tap detection, and CRAC ambient temperature logging.
Interactive Mission State Machine
Simulate the temporal state transitions and proactive voice guidance experienced by frontline operators wearing smart glasses or chest rigs.
{
"framework": "FieldKit v1.0",
"status": "INITIALIZING..."
}Author custom frontline task DAGs, validate tolerances against the FieldKit RFC, and preview in real time.
// FieldKit RFC Validation Engine v1.0.0
// Ready. Click "Validate against RFC" to inspect mission syntax.Open-Source Python SDK & CLI Runtime
Build custom wearable agents, parse state machines, and simulate raycasting on Linux, Android, and macOS.
Run deterministic missions directly from your terminal or integrate the state machine, raycasting evaluator, and voice gate into your own edge application.
# Install SDK & CLI directly
pip install -e sdk/python/
# Run an interactive physical mission
fieldkit run templates/industrial-repair/mission.yaml
# Test 3D skeletal hand raycast geometry
fieldkit test-raycast
# Verify on-device hardware enclave privacy vault
fieldkit vault-auditOfficial FieldKit MCP Server for AI Agents
Bridge Claude, Cursor, Cline, OpenCode, and Hermes directly into physical operations via Model Context Protocol.
Exposes 6 standard tools allowing LLMs to orchestrate physical missions, evaluate camera bounding targets, track hands-busy states, and verify biometric vault isolation.
# Claude Desktop Integration (claude_desktop_config.json)
{
"mcpServers": {
"fieldkit": {
"command": "python3",
"args": ["/path/to/fieldkit/mcp/fieldkit_mcp_server.py"]
}
}
}{
"jsonrpc": "2.0",
"result": {
"tool": "fieldkit_evaluate_fov_frame",
"status": "READY. Click 'Run MCP Tool Test' to query live engine."
}
}Shape the Next Era of AI Development
We are moving beyond text prompts. Help build the open standard that connects neural models to physical tools, vehicles, medical kits, and field gear.