A headless App Knowledge Graph Service for mobile agents. Stores app states and navigation paths in Neo4j, accessed via MCP (Model Context Protocol).

• Graph-Based Navigation: Stores app states and transitions in Neo4j
• Vector Similarity Search: Visual and semantic embeddings for state matching
• Automated Crawling: Discovers app states using uiautomator2
• Expert Knowledge Layer: Hint system for optimal routing (Phase 4)
• MCP Interface: Exposes tools for AI agents (Claude, etc.)

GraphicStore/
├── docker-compose.yml         # Neo4j 5.x with plugins
├── requirements.txt           # Python dependencies
├── .env.example              # Configuration template
├── demo_phase1.py            # Storage layer demo
├── demo_phase2.py            # Crawler demo
├── demo_phase3.py            # MCP server demo
├── demo_phase4.py            # Hint system demo
├── src/
│   ├── store/
│   │   └── neo4j_store.py    # Neo4j storage + hints
│   ├── embeddings/
│   │   └── encoder.py        # CLIP + MiniLM encoders
│   ├── crawler/
│   │   └── crawler.py        # App crawler (Phase 2)
│   └── mcp_server/
│       └── server.py         # MCP interface + hints
├── config/
│   └── settings.py           # Configuration
└── docs/
    ├── phase1_reference.md   # Storage API docs
    ├── phase2_reference.md   # Crawler docs
    ├── phase3_reference.md   # MCP tools docs
    └── phase4_reference.md   # Hint system docs

Use the quick start script with uv (fast Python package manager):

# Shell script version
./quickstart.sh

# Or Python version
python3 quickstart.py

# Or with specific options
python3 quickstart.py --phase 2 --python 3.11

This will:

• Install uv if needed
• Create Python virtual environment
• Install all dependencies
• Start Neo4j in Docker
• Initialize the database schema
• Run the demo

docker-compose up -d

# With uv (recommended)
pip install uv
uv venv --python 3.10
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
uv pip install -r requirements.txt

# Or with pip
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Phase 1: Storage layer
uv run python3 demo_phase1.py

# Phase 2: Crawler (requires Android device)
python3 demo_phase2.py com.android.settings

# Phase 3: MCP server
python3 demo_phase3.py --interactive

# Phase 4: Expert knowledge (hint system)
python3 demo_phase4.py

# App States
(AppState {
  xml_hash: string (unique),
  package: string,
  activity: string,
  ui_vector: [float x512],    # Visual embedding
  desc_vector: [float x384],  # Semantic embedding
})
-[:LEADS_TO {
  action: {action, target, bounds}
}]->
(AppState ...)

# Expert Knowledge (Phase 4)
(:Hint {
  intent_text: string,
  intent_vector: [float x384],
  advice_type: "SHORTCUT" | "AVOID",
  package: string
})
-[:POINTS_TO | :AVOIDS]->
(:AppState)

• Vector indexes for similarity search
• State management (upsert, lookup)
• Transition tracking
• Path finding

• XML state hashing with dynamic attribute stripping
• Clickable element detection
• Automated exploration loop
• State matching (exact hash + visual similarity)

• maps: Find navigation path from objective
• find_similar_states: Semantic search
• get_state_info: State details
• get_outgoing_transitions: Available actions
• crawl_app: Trigger automated crawling

• SHORTCUT hints: Suggest optimal paths
• AVOID hints: Exclude problematic routes
• Vector-based hint matching
• Constraint-aware path finding

from src.store.neo4j_store import Neo4jStore
from src.embeddings.encoder import get_encoder

# Connect to Neo4j
store = Neo4jStore()
store.initialize_schema()

encoder = get_encoder()

# Create a state
ui_vec = encoder.encode_image("screenshot.png")
desc_vec = encoder.encode_text("Main screen with navigation")

store.upsert_state(
    xml_hash="main_abc123",
    package="com.example.app",
    activity=".MainActivity",
    screenshot_vec=ui_vec,
    desc_vec=desc_vec,
)

# Add transition
store.add_transition("main_abc123", "settings_def456", {
    "action": "click",
    "target": "settings_button",
})

# Shortcut hint for common intent
store.insert_hint(
    package="com.example.app",
    intent_text="change brightness",
    intent_vector=encoder.encode_text("change brightness"),
    target_xml_hash="quick_settings_brightness",
    advice_type="SHORTCUT",
    metadata={"reason": "Quick access from notification shade"}
)

# Avoid hint to exclude expert features
store.insert_hint(
    package="com.example.app",
    intent_text="configure app",
    intent_vector=encoder.encode_text("configure app"),
    target_xml_hash="developer_options",
    advice_type="AVOID",
    metadata={"user_level": "expert"}
)

# Enhanced path finding with hint constraints
result = store.find_shortest_path_with_hints(
    start_hash="main_menu",
    end_hash="display_settings",
    package="com.example.app",
    intent_vector=encoder.encode_text("change brightness"),
    max_depth=10
)

print(result["path"])           # Action steps
print(result["shortcuts_used"]) # Applied shortcuts
print(result["avoids_applied"]) # Excluded states

from src.crawler.crawler import crawl_app

stats = crawl_app(
    package_name="com.android.settings",
    max_states=100,
    max_time=300,
)

print(f"Discovered {stats.states_discovered} states")

Add to Claude Desktop config:

{
  "mcpServers": {
    "app-atlas": {
      "command": "python3",
      "args": ["-m", "src.mcp_server.server"],
      "cwd": "/path/to/GraphicStore"
    }
  }
}

Then use from Claude:

Help me navigate to the Bluetooth settings in the Settings app.

The maps tool automatically uses hints to provide optimal routes.

Tool	Description	Phase
maps	Find navigation path from objective	3 + 4
get_state_info	Get state details	3
find_similar_states	Semantic search	3
get_outgoing_transitions	Available actions	3
crawl_app	Trigger automated crawling	3
insert_hint	Create expert knowledge hint	4

• Phase 1: Storage API
• Phase 2: Crawler
• Phase 3: MCP Tools
• Phase 4: Hint System

Copy .env.example to .env and configure:

NEO4J_URI=bolt://localhost:7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=appatlas_password
EMBEDDING_DEVICE=cuda  # or cpu

• Python 3.10+
• Neo4j 5.x (Docker)
• Android device (for crawler)
• CUDA-capable GPU (optional, for faster embeddings)

MIT