Knowledge Graph Extraction

Documentation Index

Fetch the complete documentation index at: https://explore.airia.com/llms.txt

Use this file to discover all available pages before exploring further.

​

Two Modes of Entity Extraction

​

When to Use Graph RAG

• Simple Q&A over a single document
• Keyword-driven search (e.g., “find the latest sales report”)

​

Getting Started

​

Step 1: Create a Data Source and Enable Knowledge Graph Extraction

1. Navigate to Data Sources in your project
2. Click Create Data Source
3. Configure your connector (SharePoint, OneDrive, Google Drive, etc.)
4. In the Ingestion Settings section, toggle “Enable Knowledge Graph Extraction” to ON
5. You’ll see two options:
   • Generic Entity Extraction — the LLM freely discovers entities from your content
   • Industry-Specific Entity Extraction — you define which entity types to extract via an industry preset or custom ontology

​

Step 2a: Generic Entity Extraction (simple path)

1. No additional configuration needed — just proceed with data source creation
2. During ingestion, the system will run an LLM on each chunk to extract entities of any type (people, organizations, locations, concepts, etc.)
3. Extracted entities are stored as graph nodes with relationships automatically
4. Files will show status Processed when complete (same as standard RAG)

INGESTION SERVICE (single pipeline)
====================================
Document → Parse/OCR → Chunk → Embed → [Entity Extraction] → Store to Vector DB
                                                 |
                                                 v
                                       Knowledge Graph (Apache AGE)

• Zero configuration — just toggle on and ingest
• Broad entity coverage — catches entities you might not have anticipated
• Simpler pipeline — everything runs in one service

• Less precise — may extract irrelevant entities that add noise to the graph
• No control over entity types — you can’t tell it “only extract regulations and products”
• No descriptions to guide extraction quality

​

Step 2b: Industry-Specific Entity Extraction (production path)

1. Select an industry preset or create a custom one
2. Customize entity types (add, remove, add descriptions)
3. Complete data source setup and start ingestion

INGESTION SERVICE                         INDEXING SERVICE
================                         ================
Document → Parse/OCR → Chunk → [STOP]  →  Entity Extraction → Embed → Store to Vector DB
                                                    |
                                                    v
                                          Knowledge Graph (Apache AGE)

1. Ingestion Service parses and chunks the document, then stops — file status changes to Parsed
2. Indexing Service picks up the parsed file, runs entity extraction using your configured entity types, creates embeddings, and stores everything — file status changes to Indexing, then Indexed

​

Step 3: Customize Entity Types (Industry-Specific only)

• Add entities: Type an entity name and press Enter (up to 15 entities per ontology)
• Remove entities: Click the X on any entity tag
• Add descriptions: Click the pencil icon on an entity to add a description that guides the extraction LLM
• Edit after creation: Use the Edit Data Source flow to modify entities on an existing graph

• Use specific, domain-relevant entity types (e.g., “drug_interaction” is better than “relationship”)
• Add descriptions to guide the LLM — e.g., for “obligation”: “A contractual duty or requirement that one party must fulfill”
• Entity names are automatically normalized: lowercased, spaces replaced with underscores

​

File Status Lifecycle

​

Generic Entity Extraction — Status Flow

Pending → Downloading → Downloaded → Processing → Processed
                                         |
                                         v
                                   ProcessingFailed

​

Industry-Specific Entity Extraction — Status Flow

Pending → Downloading → Downloaded → Processing → Parsed → Indexing → Indexed
                                         |                     |
                                         v                     v
                                   ProcessingFailed      IndexingFailed

​

How to Monitor File Status

1. Open your data source
2. Navigate to the Files tab
3. Each file shows its current status with a color indicator
4. Counters at the top show aggregate counts: total files, indexing files, indexed files
5. Files in Parsed status are waiting for the indexing service — this is expected, not an error
6. Files in IndexingFailed should be investigated — common causes are LLM quota exhaustion or misconfigured entity types

​

Data Source-Level Status (Industry-Specific only)

​

AI Cost and Token Consumption

​

Cost Comparison

​

Cost Estimation

Cost per file ≈ (number of chunks) × (LLM cost per extraction call)

• 200 chunks x ~500 input tokens = ~100,000 tokens
• At GPT-4 pricing: approximately $0.30–$1.00 per file

​

How to Track Costs

​

Token Consumption Feed

• Embedding calls — from ingestion (generic) or indexing service (industry-specific)
• Entity extraction calls — the new cost, labeled with the LLM model used
• Reranker calls — at retrieval time, if reranking is enabled

​

Ingestion Time vs. Indexing Time

• Generic extraction: Total processing time = ingestion time (includes entity extraction). Visible in file details as a single duration.
• Industry-specific extraction: Total time = ingestion time + indexing time. Ingestion time covers parse/chunk. Indexing time covers entity extraction + embedding. Indexing is typically 2–5x longer than standard embedding due to LLM calls.

​

Retrieval Cost

​

Cost Optimization Tips

​

During Retrieval

1. Vector search finds semantically similar chunks (standard RAG)
2. Entity-based retrieval identifies entities in the query and traverses the knowledge graph to find related entities and their source chunks
3. Results from both paths are combined and reranked for the final response
4. The LLM receives both vector-matched chunks and graph-traversed context

​

Managing Your Knowledge Graph

​

Viewing the Graph

1. Go to your Data Sources list
2. Find your graph-enabled data source and click the three-dot menu (…)
3. Select View Graph
4. You’ll see:
   • Node count by entity type
   • Relationship count by relationship type
   • Graph query capability using Cypher syntax

⚠️ Warning: The ability to view the graph is not supported for data sources with Original Source Permissions enabled. This ensures that entities and chunks of data are not accessed by users without the necessary permissions. For more details, see Original Source Permissions.

​

Querying the Graph (Advanced)

• MATCH (n) RETURN n LIMIT 50 — Browse all nodes
• MATCH (n:person) RETURN n — Find all entities of a specific type
• MATCH ()-[r]->() RETURN r LIMIT 50 — Browse all relationships
• MATCH (n {name: 'GDPR'})-[r]->(m) RETURN m — Find everything connected to a specific entity

​

Editing Entity Configuration (Industry-Specific only)

1. Open the data source in edit mode
2. Navigate to the Graph RAG settings
3. Add, remove, or modify entity types
4. Changed entities will trigger re-indexing on the next sync (files go back to Parsed → Indexing → Indexed)

Note: Re-indexing re-runs entity extraction, which incurs additional AI cost.

​

Choosing Between Generic and Industry-Specific

​

Best Practices

​

Entity Type Design (Industry-Specific)

​

Data Source Configuration

• Enable Graph RAG at creation time — adding it later requires re-ingestion of all files
• One graph per data source — each data source gets its own knowledge graph
• Graph RAG works best with document collections (100+ files) where entity relationships span documents
• Image scanning can be combined with Graph RAG — enable both for scanned documents containing entity-rich content

​

Troubleshooting