Storage Architecture

OpenViking uses a dual-layer storage architecture that separates content storage from index storage.

Overview

┌─────────────────────────────────────────┐
│          VikingFS (URI Abstraction)      │
│    URI Mapping · Hierarchical Access     │
│           · Relation Management          │
└────────────────┬────────────────────────┘
        ┌────────┴────────┐
        │                 │
┌───────▼────────┐  ┌─────▼───────────┐
│  Vector Index  │  │      AGFS       │
│ (Semantic      │  │ (Content        │
│  Search)       │  │  Storage)       │
└────────────────┘  └─────────────────┘

Dual-Layer Storage

Layer	Responsibility	Content
AGFS	Content storage	L0/L1/L2 full content, multimedia files, relations
Vector Index	Index storage	URIs, vectors, metadata (no file content)

Design Benefits

1. Clear responsibilities: Vector index handles retrieval, AGFS handles storage
2. Memory optimization: Vector index doesn't store file content, saving memory
3. Single data source: All content read from AGFS; vector index only stores references
4. Independent scaling: Vector index and AGFS can scale separately Note: AGFS has been rewritten as a Rust implementation (RAGFS)

VikingFS Virtual Filesystem

VikingFS is the unified URI abstraction layer that hides underlying storage details.

URI Mapping

viking://resources/docs/auth  →  /local/resources/docs/auth
viking://user/memories        →  /local/user/memories
viking://agent/skills         →  /local/agent/skills

Core API

Method	Description
read(uri)	Read file content
write(uri, data)	Write file
mkdir(uri)	Create directory
rm(uri)	Delete file/directory (syncs vector deletion)
mv(old, new)	Move/rename (syncs vector URI update)
abstract(uri)	Read L0 abstract
overview(uri)	Read L1 overview
relations(uri)	Get relation list
find(query, uri)	Semantic search

Relation Management

VikingFS manages resource relations through .relations.json:

# Create relation
viking_fs.link(
    from_uri="viking://resources/docs/auth",
    uris=["viking://resources/docs/security"],
    reason="Related security docs"
)

# Get relations
relations = viking_fs.relations("viking://resources/docs/auth")

AGFS Backend Storage

AGFS provides POSIX-style file operations with multiple backend support.

Backend Types

Backend	Description	Config
localfs	Local filesystem	path
s3fs	S3-compatible storage	bucket, endpoint
memory	Memory storage (for testing)	-

Directory Structure

Each context directory follows a unified structure:

viking://resources/docs/auth/
├── .abstract.md          # L0 abstract
├── .overview.md          # L1 overview
├── .relations.json       # Relations table
└── *.md                  # L2 detailed content

Vector Index

The vector index stores semantic indices, supporting vector search and scalar filtering.

Context Collection Schema

Field	Type	Description
id	string	Primary key
uri	string	Resource URI
parent_uri	string	Parent directory URI
context_type	string	resource/memory/skill
is_leaf	bool	Whether leaf node
vector	vector	Dense vector
sparse_vector	sparse_vector	Sparse vector
abstract	string	L0 abstract text
name	string	Name
description	string	Description
created_at	string	Creation time
active_count	int64	Usage count

Index Strategy

index_meta = {
    "IndexType": "flat_hybrid",  # Hybrid index
    "Distance": "cosine",        # Cosine distance
    "Quant": "int8",             # Quantization
}

Backend Support

Backend	Description
local	Local persistence
http	HTTP remote service
volcengine	Volcengine VikingDB

Vector Synchronization

VikingFS automatically maintains consistency between vector index and AGFS.

Delete Sync

viking_fs.rm("viking://resources/docs/auth", recursive=True)
# Automatically deletes all records with this URI prefix from vector index

Move Sync

viking_fs.mv(
    "viking://resources/docs/auth",
    "viking://resources/docs/authentication"
)
# Automatically updates uri and parent_uri fields in vector index

Related Documents

• Architecture Overview - System architecture
• Context Layers - L0/L1/L2 model
• Viking URI - URI specification
• Retrieval Mechanism - Retrieval process details