Toward Personal Intelligence Through Cooperative Observation
06:00 · August 19, 2026 · arXiv cs.AI RSS
A personal AI system needs a model of the user's goals, constraints, and ongoing commitments to plan and act on their behalf, and the quality of that model is bounded by what the system can observe. Broader observation does not by itself improve assistance because a bounded system must select and compress information for the task at hand. We argue that this observation bottleneck has a cooperative structure: the system builds a partial model of the user's changing life, the user evaluates its actions, and the user's consent and control shape what it can observe next. Useful and inspectable behavior can give users a reason to maintain or expand the observation channel, while failures can lead them to correct, narrow, revoke, or abandon it. We use the term cooperative observation for this feedback loop among usefulness, trust, and future access, and propose it as a framework for personal intelligence. We report a preliminary single-subject account from Organizm, a prototype used over six months, and outline evaluation directions for measuring how observation quality shapes personal AI.
Summary
A personal AI system must maintain an internal model of a user’s goals, constraints, and commitments to act effectively on their behalf. The fidelity of that model is limited by the quality and scope of the observation channel available to it. Because any computational system must select and compress incoming signals according to the task at hand, simply increasing the volume of data does not automatically improve assistance. The authors therefore frame the observation bottleneck as an inherently cooperative process in which the system constructs a partial model of the user’s evolving circumstances, the user assesses the resulting actions, and the user’s consent and control decisions determine what the system may observe next.
This feedback loop, termed cooperative observation, links the usefulness of the assistant’s behavior to the breadth of future access. Successful, inspectable actions can encourage a user to sustain or widen the channel, whereas errors or opacity may prompt correction, restriction, or revocation. The framework integrates concepts from partial-observability models and information-theoretic limits on representation, while treating the user as the sole source of both the evaluation signal and the governance of data sources. Observation channels are described along a spectrum that begins with deliberate manual reports and can progress through device APIs, continuous sensor streams, and, eventually, neural interfaces, each step widening the information available yet increasing the stakes for user oversight.
The paper illustrates the approach with Organizm, a prototype that stores user-owned files and explicit memory structures under local control and adapts its planning according to explicit feedback. A six-month single-subject deployment provides an initial account of how observation quality and channel adjustments interact in practice. The authors contrast this user-governed setting with platform-scale personalization systems whose objectives serve engagement or revenue rather than the modeled individual, underscoring that privacy, consent, and alignment with personally defined success criteria are structural requirements rather than optional features of personal intelligence.
Why it matters
Strong alignment with Dutch/EU priorities on ethical, transparent, and privacy-preserving AI; offers actionable concepts for researchers building user-owned personal agents compliant with GDPR and trustworthy AI guidelines.
