MemoryData
OpenDataBox benchmark suite for comparing memory-augmented agents across multiple benchmark families, method presets, and a unified runtime interface.
MemoryData is the code artifact for Are We Ready For An Agent-Native Memory System?. The repository describes itself as "One pipeline. Four benchmark families. Twenty-two method presets. One consistent execution interface." It is a Python benchmark suite for evaluating memory-augmented agents under a common runtime. Source: GitHub OpenDataBox/MemoryData README, 2026-06-25
What It Contains
Benchmark families:
- MemoryAgentBench
- LoCoMo
- LongBench
- MemBench
Method presets include reference baselines, sequential context approaches, structural/topological methods, and multi-paradigm hybrids. The README names methods such as Long Context, Embedding RAG, BM25 RAG, MemAgent, Mem0, MemoChat, Cognee, Zep Local, MemTree, GraphRAG, HippoRAG, RAPTOR, Zep, Letta, LightMem, SimpleMem, MemOS, and MemoryOS. Source: GitHub OpenDataBox/MemoryData README, 2026-06-25
Invocation Shape
The unified launcher shape is:
python main.py --agent_config <agent_yaml> --dataset_config <dataset_yaml>
Use MemoryData when building or comparing memory-heavy agents and you need a structured evaluation surface instead of a single anecdotal demo. It is reference-only for Kevin's wiki until a project actually needs benchmark integration.
Timeline
- 2026-06-21 | Repository created. Source: GitHub API, 2026-06-25
- 2026-06-23 | Linked as the public code artifact for "Are We Ready For An Agent-Native Memory System?" Source: arXiv 2606.24775v1
- 2026-06-25 | Captured into the wiki from Kevin's DAIR.AI X bookmark. Source: X/@dair_ai, 2026-06-24