|
1 | 1 | # SenseKit |
2 | 2 |
|
3 | | -SenseKit is an open-source context runtime for AI agents. |
| 3 | +[](https://github.com/Sense-Kit/sense-kit/actions/workflows/ci.yml) |
4 | 4 |
|
5 | | -It runs on iPhone, listens to passive sensor signals, turns them into deterministic context events, and delivers those events to OpenClaw over signed HTTPS. |
| 5 | +SenseKit is an open-source iPhone context runtime for AI agents. |
6 | 6 |
|
7 | | -The goal is simple: an AI agent should understand important real-world transitions without asking the user to build Shortcuts automations first. |
| 7 | +It listens to passive on-device signals like motion, location, workouts, power state, and lightweight calendar context, turns those signals into deterministic events, and delivers signed HTTPS payloads to OpenClaw. |
8 | 8 |
|
9 | | -## Product thesis |
| 9 | +The project exists to solve one practical problem: an agent cannot act intelligently around the real world if the user has to build a pile of Shortcuts automations before anything works. SenseKit aims to be useful right after install and permission approval. |
10 | 10 |
|
11 | | -SenseKit is: |
| 11 | +## Why SenseKit |
12 | 12 |
|
13 | | -- passive-first |
14 | | -- event-first |
15 | | -- deterministic |
16 | | -- local-first |
17 | | -- OpenClaw-first |
| 13 | +- `Passive-first`: useful events should be possible without manual Shortcuts setup. |
| 14 | +- `Deterministic`: the runtime uses explicit rules, weights, and thresholds instead of opaque ML guesses. |
| 15 | +- `Local-first`: raw sensor data stays on the phone by default. |
| 16 | +- `Policy-aware`: each outbound event includes the safety context OpenClaw needs to react responsibly. |
| 17 | +- `OpenClaw-first`: the primary integration path is signed outbound delivery, not hosting a server on the phone. |
18 | 18 |
|
19 | | -That means: |
| 19 | +## Current status |
20 | 20 |
|
21 | | -- the iPhone app should do useful work right after install and permissions |
22 | | -- event detection uses fixed weights and thresholds, not ML guesses |
23 | | -- raw sensor data stays local by default |
24 | | -- the main integration is an outbound webhook, not a phone-hosted server |
| 21 | +SenseKit is early, opinionated, and real. |
25 | 22 |
|
26 | | -This repository is a monorepo. It keeps the iOS runtime, shared wire contracts, OpenClaw integration helpers, and project docs together because they change together. |
| 23 | +What already exists in this repository: |
27 | 24 |
|
28 | | -## What data gets sent to OpenClaw |
29 | | - |
30 | | -SenseKit does not send a raw dump of everything your phone sees. |
| 25 | +- a deterministic corroboration engine for passive events |
| 26 | +- a Swift runtime package with Motion, Location, HealthKit, power, and calendar collectors |
| 27 | +- SQLite-backed runtime state, queueing, and audit logging |
| 28 | +- signed webhook delivery for OpenClaw |
| 29 | +- a SwiftUI app shell with onboarding, settings, audit log, and debug timeline scaffolding |
| 30 | +- a separate bench harness target for field testing |
| 31 | +- shared JSON schemas and TypeScript validation helpers |
31 | 32 |
|
32 | | -What currently goes out: |
| 33 | +What is still being validated on real devices: |
33 | 34 |
|
34 | | -- one event at a time, for example `motion_activity_observed` or `arrived_home` |
35 | | -- the event time |
36 | | -- a confidence score |
37 | | -- short reasons like `motion.primary.walking` or `location.region_enter_home` |
38 | | -- a small snapshot with coarse state like: |
39 | | - - place type: `home`, `work`, or `other` |
40 | | - - routine flags like `awake` or `workout` |
41 | | - - minimal calendar booleans |
42 | | - - battery bucket and charging state |
43 | | -- a policy block that tells OpenClaw which response modes are safe |
| 35 | +- wake detection precision |
| 36 | +- driving detection precision |
| 37 | +- battery impact over normal daily use |
| 38 | +- background wake and delivery reliability |
| 39 | +- final onboarding polish |
44 | 40 |
|
45 | | -What stays local by default: |
| 41 | +If you are evaluating the project, start with: |
46 | 42 |
|
47 | | -- exact GPS coordinates |
48 | | -- raw motion history |
49 | | -- raw HealthKit values |
50 | | -- calendar titles and attendees |
51 | | -- local debug traces |
52 | | -- tokens and secrets |
| 43 | +- [SPEC.md](SPEC.md) |
| 44 | +- [docs/README.md](docs/README.md) |
| 45 | +- [docs/bench/phase-1a-field-test.md](docs/bench/phase-1a-field-test.md) |
| 46 | +- [docs/runbooks/runtime-bootstrap.md](docs/runbooks/runtime-bootstrap.md) |
53 | 47 |
|
54 | | -In the current bench app: |
| 48 | +## Architecture at a glance |
55 | 49 |
|
56 | | -- Motion is currently forwarded as coarse activity events like `walking`, `running`, `stationary`, or `automotive` |
57 | | -- Home / Work is currently forwarded as place events like `arrived_home` |
| 50 | +```mermaid |
| 51 | +flowchart LR |
| 52 | + A["On-device collectors<br/>Motion, location, HealthKit,<br/>power, calendar"] --> B["Deterministic corroboration engine"] |
| 53 | + B --> C["Context event<br/>confidence, reasons, policy"] |
| 54 | + C --> D["Signed outbound queue"] |
| 55 | + D --> E["OpenClaw webhook"] |
| 56 | +``` |
58 | 57 |
|
59 | | -So the phone sends structured context events, not a full sensor stream. |
| 58 | +SenseKit is a runtime, not a data exhaust pipeline. The phone keeps the raw sensor layer local and only sends small, structured events once the runtime is confident something meaningful happened. |
60 | 59 |
|
61 | | -## What works today |
| 60 | +## What leaves the phone |
62 | 61 |
|
63 | | -This repo already includes the first serious product scaffolding: |
| 62 | +SenseKit does not send a raw dump of everything the device sees. |
64 | 63 |
|
65 | | -- a deterministic corroboration engine for passive events |
66 | | -- a Swift runtime package with Motion, Location, HealthKit, power, and calendar collectors |
67 | | -- SQLite-backed runtime state, queue, and audit log |
68 | | -- signed webhook delivery for OpenClaw |
69 | | -- a SwiftUI app shell with onboarding, settings, audit log, and debug timeline scaffolding |
70 | | -- a separate bench harness target for field testing |
71 | | -- shared JSON schemas and TypeScript validation helpers |
| 64 | +| Sent to OpenClaw | Stays local by default | |
| 65 | +| --- | --- | |
| 66 | +| One event at a time, such as `motion_activity_observed` or `arrived_home` | Exact GPS coordinates | |
| 67 | +| Event timestamp | Raw motion history | |
| 68 | +| Confidence score | Raw HealthKit values | |
| 69 | +| Short reason codes like `motion.primary.walking` | Calendar titles and attendees | |
| 70 | +| A small snapshot with coarse state | Local debug traces | |
| 71 | +| A policy block describing safe response modes | Tokens and secrets | |
72 | 72 |
|
73 | | -What is not proven yet: |
| 73 | +In the current bench app, motion is forwarded as coarse activity events such as `walking`, `running`, `stationary`, or `automotive`, and place transitions are forwarded as events such as `arrived_home`. |
74 | 74 |
|
75 | | -- real-device wake precision |
76 | | -- real-device driving precision |
77 | | -- battery impact over normal daily use |
78 | | -- final onboarding polish |
| 75 | +## Repository guide |
79 | 76 |
|
80 | | -In other words: the architecture and build system are in place, but the passive claims still need bench validation. |
| 77 | +- [`apps/ios`](apps/ios): the iPhone app, bench harness, runtime package, and SwiftUI package |
| 78 | +- [`packages/contracts`](packages/contracts): JSON schemas, fixtures, and TypeScript validation helpers |
| 79 | +- [`packages/openclaw-skill`](packages/openclaw-skill): example skill packaging for OpenClaw |
| 80 | +- [`packages/openclaw-plugin`](packages/openclaw-plugin): plugin surface for later OpenClaw integration work |
| 81 | +- [`packages/examples`](packages/examples): QR bootstrap and hook configuration examples |
| 82 | +- [`docs`](docs): ADRs, privacy notes, release notes, runbooks, and field-test plans |
81 | 83 |
|
82 | | -## Repo layout |
| 84 | +Documentation entry points: |
83 | 85 |
|
84 | | -- `apps/ios`: iPhone app, bench harness, Swift runtime package, and SwiftUI package |
85 | | -- `packages/contracts`: JSON schemas, fixtures, and TypeScript validation helpers |
86 | | -- `packages/openclaw-skill`: example skill packaging for OpenClaw |
87 | | -- `packages/openclaw-plugin`: plugin surface for later OpenClaw integration work |
88 | | -- `packages/examples`: webhook and QR bootstrap examples |
89 | | -- `docs`: ADRs, privacy notes, bench plans, and runbooks |
| 86 | +- [docs/README.md](docs/README.md) |
| 87 | +- [CHANGELOG.md](CHANGELOG.md) |
| 88 | +- [CONTRIBUTING.md](CONTRIBUTING.md) |
| 89 | +- [SECURITY.md](SECURITY.md) |
| 90 | +- [SUPPORT.md](SUPPORT.md) |
90 | 91 |
|
91 | 92 | ## Local setup |
92 | 93 |
|
@@ -124,46 +125,30 @@ xcodebuild -workspace apps/ios/SenseKit.xcworkspace -scheme SenseKitApp -sdk iph |
124 | 125 | xcodebuild -workspace apps/ios/SenseKit.xcworkspace -scheme SenseKitBenchApp -sdk iphonesimulator -destination 'generic/platform=iOS Simulator' CODE_SIGNING_ALLOWED=NO build |
125 | 126 | ``` |
126 | 127 |
|
127 | | -## OpenClaw connectivity and security |
| 128 | +## OpenClaw connectivity |
128 | 129 |
|
129 | 130 | For personal testing, keep OpenClaw private and use Tailscale instead of exposing the raw Gateway to the public internet. |
130 | 131 |
|
131 | | -- Recommended first setup: OpenClaw stays on `gateway.bind: "loopback"` and is exposed privately with Tailscale Serve. |
132 | | -- Do not expose the raw OpenClaw Gateway port publicly just to receive SenseKit hooks. The Gateway is a larger surface than one webhook path. |
| 132 | +- Keep OpenClaw on `gateway.bind: "loopback"` and expose it privately with Tailscale Serve. |
133 | 133 | - Use a separate `hooks.token` for SenseKit hooks. Do not reuse `gateway.auth.token`. |
134 | | -- Treat hook payloads as untrusted content even when they come from systems you control. Keep the receiving agent narrow and low-privilege. |
135 | | -- A public hook-only reverse proxy or a small verifier relay can come later, but they are not the safest first deployment. |
136 | | -- If OpenClaw returns `hook mapping failed`, remove any custom `transform` block first and test with the example mapping unchanged. SenseKit sends snake_case fields like `event.event_id`, `event.event_type`, and `snapshot.place.type`. |
| 134 | +- Treat hook payloads as untrusted content even when they come from systems you control. |
| 135 | +- If OpenClaw returns `hook mapping failed`, remove any custom `transform` block first and test with the example mapping unchanged. |
137 | 136 |
|
138 | | -## Debug Timeline vs Audit Log |
| 137 | +## Contribution focus |
139 | 138 |
|
140 | | -- `Debug Timeline` is the local notebook. It shows what the phone sensed and what the local runtime did with it. |
141 | | -- `Audit Log` is the delivery ledger. It shows whether a finished outbound event was queued, delivered, failed, or expired on the way to OpenClaw. |
142 | | - |
143 | | -That means: |
144 | | - |
145 | | -- use `Debug Timeline` when you want to debug Motion, Location, Settings, or rule evaluation |
146 | | -- use `Audit Log` when you want to know whether OpenClaw actually received the event |
147 | | - |
148 | | -Location events are included in Audit once they become real outbound events. For example, `arrived_home` should show up there after it is queued and delivered. |
149 | | - |
150 | | -## First development focus |
151 | | - |
152 | | -If you want to help on the MVP, the highest-value work is: |
| 139 | +The highest-value work for the current milestone is: |
153 | 140 |
|
154 | 141 | 1. passive wake validation on real devices |
155 | 142 | 2. driving detection validation on real commutes |
156 | 143 | 3. background wake and delivery reliability |
157 | 144 | 4. onboarding and debug timeline polish |
158 | 145 |
|
159 | | -The best starting docs are: |
160 | | - |
161 | | -- `docs/adr` |
162 | | -- `docs/bench/phase-1a-field-test.md` |
163 | | -- `docs/runbooks/runtime-bootstrap.md` |
| 146 | +Contributions are welcome. If your change affects architecture, add or update an ADR in [`docs/adr`](docs/adr). |
164 | 147 |
|
165 | | -## Open source notes |
| 148 | +## Project standards |
166 | 149 |
|
167 | 150 | - License: Apache-2.0 |
168 | | -- Contributions are welcome. See `CONTRIBUTING.md` |
169 | | -- Architecture decisions live in `docs/adr` |
| 151 | +- Contribution guide: [CONTRIBUTING.md](CONTRIBUTING.md) |
| 152 | +- Code of conduct: [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) |
| 153 | +- Security policy: [SECURITY.md](SECURITY.md) |
| 154 | +- Support guide: [SUPPORT.md](SUPPORT.md) |
0 commit comments