65 lines
1.9 KiB
Markdown
65 lines
1.9 KiB
Markdown
# Cassandra - The RAG Database
|
|
|
|
The retrieval database in the dynamixs.ai RAG system is based on Apache Cassandra which provides a distributed database system with high availability and scalability.
|
|
|
|
The schema for the RAG database is created automatically during startup. This includes the Cassandra keyspace as also the data base schema.
|
|
The main table has the following definition:
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS document_vectors (
|
|
id text,
|
|
chunk_id uuid,
|
|
content_chunk text,
|
|
content_vector VECTOR <FLOAT, " + DIMENSIONS + ">,
|
|
PRIMARY KEY (id, chunk_id)
|
|
);
|
|
```
|
|
|
|
The default dimensions for float vector is based on the LLM used for indexing. 'all-MiniLM-L6-v2' s is a sentence-transformers model to map sentences & paragraphs to a 384 dimensional dense vector space. Find more details in the section [RAG](RAG.md).
|
|
|
|
Aside from that, the Cassandra cluster can also be managed through the Command Line Console (CLI).
|
|
|
|
## Using the Cassandra CLI
|
|
|
|
To connect to your cluster you can use the cqlsh tool.
|
|
To run cqlsh from your started docker environment run:
|
|
|
|
```bash
|
|
$ docker exec -it cassandra cqlsh
|
|
Connected to dynamixsAiCluster at 127.0.0.1:9042
|
|
[cqlsh 6.2.0 | Cassandra 5.0.4 | CQL spec 3.4.7 | Native protocol v5]
|
|
Use HELP for help.
|
|
cqlsh>
|
|
```
|
|
|
|
## CQL Examples
|
|
|
|
The following section contains some basic cqlsh commands. For full description see the [Cassandra CQL refernce](https://docs.datastax.com/en/dse/6.0/cql/).
|
|
|
|
**show key spaces:**
|
|
|
|
Show all available keyspaces:
|
|
|
|
cqlsh> DESC KEYSPACES;
|
|
|
|
**Switch to Keysapce:**
|
|
|
|
Select a keyspace be name to interact with this keyspace:
|
|
|
|
cqlsh> use embeddings;
|
|
|
|
**Show tables in a keyspace:**
|
|
|
|
Show tables schemas in current keyspace:
|
|
|
|
cqlsh:embeddings> DESC TABLES;
|
|
|
|
**Drop Keyspace:**
|
|
|
|
Drop the keyspace:
|
|
|
|
DROP KEYSPACE IF EXISTS <keyspace_name>;
|
|
|
|
### Create a dev keyspace with cqlsh
|
|
|
|
cqlsh> CREATE KEYSPACE <keyspace_name> WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
|