Source note on MemGPT’s memory architecture, reported results and limitations. The application idea for Hjarni is interpretation, not a result from the paper.
Citation
Charles Packer et al. (2023). MemGPT: Towards LLMs as Operating Systems. arXiv:2310.08560; v2, 12 February 2024. Read the paper.
Abstract
An operating-system-inspired design moves information between a model’s context and external storage to support longer conversations and document analysis.
Method
Function calls manage memory. Evaluations cover dialogue recall, document questions and chained lookups; dialogue baselines receive compressed histories.
Key findings
- GPT-4 recall accuracy: 92.5% with MemGPT vs 32.1% with summarised history (Table 2).
- Chained retrieval improves nested lookups (§3.2.2).
Limitations
- The agent sometimes ends its search before finding relevant documents.
- Results depend on the underlying model’s function-calling ability.
Your notes
Interpretation: saved information helps only if it is retrieved. For Hjarni, test whether the assistant checks the source before claiming it remembers. This paper does not benchmark Hjarni.
Related work
Next reading: Lost in the Middle. Not reviewed in this example.
Linked notes: Virtual context management · MemGPT improves dialogue recall in the reported test · Retrieval before synthesis · When does an assistant skip a relevant source?.