docu

2025-06-02 22:45:35 +02:00 · 2025-06-02 22:45:35 +02:00 · 2526507b49
commit 2526507b49
parent b4c4cd49f6
2 changed files with 219 additions and 2 deletions
--- a/doc/CASSANDRA.md
+++ b/doc/CASSANDRA.md
@ -1,6 +1,22 @@
-# Cassandra RAG Database
+# Cassandra - The RAG Database
 The retrieval database in the dynamixs.ai RAG system is based on Apache Cassandra which provides a distributed database system with high availability and scalability.
 The schema for the RAG database is created automatically during startup. This includes the Cassandra keyspace as also the data base schema.
 The main table has the following definition:
 ```sql
 CREATE TABLE IF NOT EXISTS document_vectors (
    id text,
    chunk_id uuid,
    content_chunk text,
    content_vector VECTOR <FLOAT, " + DIMENSIONS + ">,
    PRIMARY KEY (id, chunk_id)
 );
 ```
 The default dimensions for float vector is based on the LLM used for indexing. 'all-MiniLM-L6-v2' s is a sentence-transformers model to map sentences & paragraphs to a 384 dimensional dense vector space. Find more details in the section [RAG](RAG.md).
 The Cassandra database used by dynamixs.ai is created automatically during startup. This includes the Cassandra keyspace as also the data base schema.
 Aside from that, the Cassandra cluster can also be managed through the Command Line Console (CLI).
 ## Using the Cassandra CLI
--- a/doc/RAG.md
+++ b/doc/RAG.md
@ -0,0 +1,201 @@
 # Dynamixs AI - Retrieval-augmented generation (RAG)
 The following section give a general overview how the Retrieval-augmented generation (RAG) is realized in Dynamixs AI.
 ## Storage
 The retrieval data is stored in a [Cassandra Database](CASSANDRA.md) with the following schema:
 ```sql
 CREATE TABLE IF NOT EXISTS document_vectors (
    id text,
    chunk_id uuid,
    content_chunk text,
    content_vector VECTOR <FLOAT, " + DIMENSIONS + ">,
    PRIMARY KEY (id, chunk_id)
 );
 ```
 The retrieval data is generated by the LLM [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) which is
 a [sentence-transformers model](https://www.sbert.net/) that maps sentences & paragraphs to a 384 dimensional dense vector space.
 ### The Chunk Content
 The RAG database stores not only the vectors but also the textual representation of the content chunks. This has several advantages:
 **Context provision for the LLM**
 The vector is only used for similarity searching - it is a numerical representation of text, but not human-readable. Once the most similar documents have been found, the original text (content_chunk) can be used directly as context for a LLM. Note: the LLM cannot work with the vectors, but needs the readable text.
 **Efficiency of Queries**
 The chunk content also avoids a immediate access to the context with no need to make a separate query to the Imixs Workflow instance after the vector search to retrieve the original content. This would mean additional latency and complexity. `content_chunk` stores exactly the text section that corresponds to the vector.
 **The typical RAG workflow**
 The typical workflow in RAG system can be separated into 3 phases:
 1. **Search:** Vector similarity search finds relevant content_vector entries.
 2. **Retrieval:** The corresponding content_chunk texts are extracted.
 3. **Generation:** These texts from the `content_chunk' is sent as context to the LLM.
 Without the stored text, the retrieval part of RAG would not be possible, as you would have no content to provide to the LLM as context.
 ### The Content Reference
 Together with the `content_chunk` the RAG system stores also the UniqueID from to the referred process instance within the workflow instance. This is a hybrid concept that provides several advantages:
 **1. Normalization and Consistency**
 - Single Source of Truth: The main Document data exist only once in the imixs-workflow system.
 - Changes to data inside a process instance which is not part of the Vector DB are automatically available without the need to update the Vector DB.
 - No full data duplication.
 **2. Better Integration**
 - Easy access not only to text chunks but also the complete business process objects with its metadata.
 - Ability to apply additional business logic (permissions, status, etc.).
 **3. Flexibility**
 - The application can decide dynamically if either the content_chunk or other parts of the referred process instance is used as context.
 - Easier updates in case of document changes.
 - The `content_chunk` still provide fast access to context during the retrieval process.
 ## Example
 The following shows the advantage of hybrid concept of content_chunks and workitem references.
 The user enters the following question:
 _"Show me all service contracts from non-EU countries with a penalty clause of over 1 million EUR."_
 **The problem with pure text chunks:**
 In this kind of a request the context can be build from the received text chunks like :
 - Chunk 1: _"...in case of breach of contract, contractual penalties of 1.5 million EUR are applicable..."_
 - Chunk 2: _"...Penalty clause: contractual penalties of 2.3 million EUR..."_
 - Chunk 3: _"...penalties of 1.8 million EUR in case of non-fulfillment..."_
 What is missing here: contract number, date, contracting party, country, contract value, status, etc.
 **The Advantage of reference IDs:**
 But using the id the RAG system can access the full business data from the workflow system. This allows to build more relevant context like :
 ```
 Found service contracts:
 1. Contract SVC-2024-001 (ABC Corp Singapore)
  - Concluded: 15.03.2024
  - Contract penalty: 1.5 million EUR
  - Status: Active
 2. Contract SVC-2024-007 (XYZ Ltd Hong Kong)
  - Concluded: 22.01.2024
  - Contract penalty: 2.3 million EUR
  - Status: Under negotiation
 ```
 Using the combination of content_chunk text and metadata form the process instance allows to build even more complex context.
 So the hybrid approach is clearly more sensible for a Business Process Management System, as it combines both the advantages of RAG (semantic search) and structured business data.
 ## A More Complex Example
 Next we can examine a more complex use case.
 The user enters the following question:
 _"Are there incoming invoices in which service fees have been charged that do not correspond to the underlying contracts?"_
 This example indeed touches the limits of RAG, but not necessarily insurmountable ones.
 The challenges:
 1. **Cross-Document Reasoning:**
 - RAG finds similar chunks, but the question requires comparisons between different document types (invoices vs. contracts).
 - An LLM would need to simultaneously receive invoice chunks AND corresponding contract chunks.
 2. **Structured Data Extraction:**
 - "Service fees" need to be extracted and compared from unstructured text.
 - Numbers, prices, and service descriptions must be semantically mapped.
 3. **Relationship Logic:**
 - Which invoice belongs to which contract?
 - The LLM needs this connection to make meaningful comparisons.
 ### Possible Solutions
 The following solutions are specific implementations in the RAG workflow:
 **Approach 1:** Enhanced RAG with a Multi-Step Retrieval
 ```
 # Pseudo-Code
 def complex_query_handler(question):
    # Step 1: Finde relevante Rechnungen
    invoice_chunks = vector_search("Servicegebühren Rechnungen")
    # Step 2: Extrahiere Vertrags-IDs aus gefundenen Rechnungen
    contract_ids = extract_contract_references(invoice_chunks)
    # Step 3: Hole zugehörige Vertrags-Chunks
    contract_chunks = get_contract_chunks(contract_ids)
    # Step 4: LLM mit beiden Kontexten
    return llm.query(question, context=invoice_chunks + contract_chunks)
 ```
 **Approach 2:** Hybrid-System with structured data
 ```
 # Pseudo-Code
 def intelligent_agent(question):
    # Agent entscheidet: "Ich brauche Rechnungs- UND Vertragsdaten"
    plan = create_execution_plan(question)
    for step in plan:
        if step.type == "find_invoices":
            invoices = rag_search(step.query)
        elif step.type == "find_contracts":
            contracts = rag_search(step.query)
        elif step.type == "compare":
            result = llm_compare(invoices, contracts)
    return synthesize_answer(result)
 ```
 **Recommendation**
 Since we already have a business process management suite, we can follow the hybrid approach:
 1.  RAG for content discovery: "Find all documents with service fees"
 2.  Structured queries: Use your business data for relationships
 3.  LLM for final analysis: With all relevant data as context.
 ```
 # Pseudo-Code
 def analyze_service_fee_discrepancies():
    # RAG: Finde relevante Dokumente
    relevant_docs = vector_search("Servicegebühren")
    # Business Logic: Hole strukturierte Daten
    invoices_with_contracts = get_invoice_contract_pairs(relevant_docs)
    # LLM: Analysiere mit vollem Kontext
    return llm.analyze_discrepancies(invoices_with_contracts)
 ```
 **Conclusion:**
 A pure RAG solution reaches its limits here, but an intelligent hybrid system consisting of RAG + structured data + multi-step reasoning can certainly answer such complex questions.
 ## Chunking Business Documents
 As the content_chunk can only have a small size below 1KB the RAG system includes a split process to split documents into smaller content_chunks.
 The split-process breaks down large business processes into logical chunks. Since the data is usually available in markup or can be prepared for the indexing process within the BPMN model, the split is made based on the structure in the markup. This guarantees a clean separation of text parts without the need to compute overlapping.