|
| 1 | +--- |
| 2 | +title: Metadata |
| 3 | +--- |
| 4 | +# Metadata |
| 5 | + |
| 6 | +The `Build5Nines.SharpVector` vector database enables semantic search for `Text` that is stored in the database. Being able to semantically search text is an extremely useful way to lookup more information related to the text. For this purpose, `Metadata` is stored alongside the `Text` within the vector database. This way, when `Text` is found when performing a semantic search, then the matching `Metadata` is also retrieved. |
| 7 | + |
| 8 | +## Adding Metadata |
| 9 | + |
| 10 | +The `.AddText` and `.AddTextAsync` methods access 2 arguments: |
| 11 | + |
| 12 | +- `text`: The `Text` that is added to the vector database and has vector embeddings generated for. |
| 13 | +- `metadata`: This is additional data / information that is stored alongside the `Text`. |
| 14 | + |
| 15 | +```csharp |
| 16 | +vdb.AddText(text, metadata); |
| 17 | + |
| 18 | +await vdb.AddText(text, metadata); |
| 19 | +``` |
| 20 | + |
| 21 | +## JSON and String Metadata |
| 22 | + |
| 23 | +When using the `BasicMemoryVectorDatabase` class, the `Metadata` values will always be of type `String`. This enables you to store a variety of values here, including: |
| 24 | + |
| 25 | +- **JSON data**: You can serialize any data to a JSON string for storage in the `Metadata` associated with a text item in the database. |
| 26 | +- **`String` value**: You can store any other string value as the `Metadata` associated with a text item in the database. This could be a URL, Filename, or other information. |
| 27 | + |
| 28 | +!!! info "OpenAI and Ollama Support" |
| 29 | + When working with the [OpenAI](../../embeddings/openai/index.md) `BasicOpenAIMemoryVectorDatabase` and [Ollama](../../embeddings/ollama/index.md) `BasicOllamaMemoryVectorDatabase`, the `Metadata` data type is also `String`. |
| 30 | + |
| 31 | +Here are some examples of storing `string` metadata and retrieving it from the database: |
| 32 | + |
| 33 | +=== "JSON data" |
| 34 | + |
| 35 | + ```csharp |
| 36 | + // create vector database |
| 37 | + var vdb = new BasicMemoryVectorDatabase(); |
| 38 | + |
| 39 | + // some text to store in the vector database |
| 40 | + var text = "some text value"; |
| 41 | + // serialize an object to json to store as metadata |
| 42 | + var json = JsonSerializer.Serialize(new MyMetadata{ |
| 43 | + Url = "https://build5nines.com", |
| 44 | + Author = "Chris Pietschmann" |
| 45 | + }); |
| 46 | + |
| 47 | + // Add text with metadata to vector database |
| 48 | + vdb.AddText(text, json); |
| 49 | + |
| 50 | + // perform semantic search |
| 51 | + var results = vdb.Search("something to search", pageCount: 5); |
| 52 | + |
| 53 | + // Loop through search results |
| 54 | + foreach(var item in results.Texts) { |
| 55 | + var text = item.Text; |
| 56 | + var json = item.Metadata; |
| 57 | + var metadata = JsonSerializer.Deserialize<MyMetadata>(json); |
| 58 | + |
| 59 | + // do something with results and metadata |
| 60 | + } |
| 61 | + ``` |
| 62 | + |
| 63 | +=== "String value" |
| 64 | + |
| 65 | + ```csharp |
| 66 | + // create vector database |
| 67 | + var vdb = new BasicMemoryVectorDatabase(); |
| 68 | + |
| 69 | + // some text to store in the vector database |
| 70 | + var text = "some text value"; |
| 71 | + // some metadata to store |
| 72 | + var metadata = "https://build5nines.com"; |
| 73 | + |
| 74 | + // Add text with metadata to vector database |
| 75 | + vdb.AddText(text, metadata); |
| 76 | + |
| 77 | + // perform semantic search |
| 78 | + var results = vdb.Search("something to search", pageCount: 5); |
| 79 | + |
| 80 | + // Loop through search results |
| 81 | + foreach(var item in results.Texts) { |
| 82 | + var text = item.Text; |
| 83 | + var metadata = item.Metadata; |
| 84 | + |
| 85 | + // do something with results and metadata |
| 86 | + } |
| 87 | + ``` |
| 88 | + |
| 89 | +## Custom Metadata Type |
| 90 | + |
| 91 | +The `MemoryVectorDatabase<TMetadata>` generic class allows you to create a vector database that uses your own custom class as the metadata by defining that class using generics. This enables you to store a native .NET object as the metadata alongside the text in the vector database. |
| 92 | + |
| 93 | +Here's an example of using the `MemoryVectorDatabase<TMetadata>` with a .NET class for the `Metadata`: |
| 94 | + |
| 95 | +```csharp |
| 96 | +// create vector database |
| 97 | +var vdb = new MemoryVectorDatabase<MyMetadata>(); |
| 98 | + |
| 99 | +// some text to store in the vector database |
| 100 | +var text = "some text value"; |
| 101 | +// an object to store as metadata |
| 102 | +var metadata = new MyMetadata{ |
| 103 | + Url = "https://build5nines.com", |
| 104 | + Author = "Chris Pietschmann" |
| 105 | +}; |
| 106 | + |
| 107 | +// Add text with metadata to vector database |
| 108 | +vdb.AddText(text, metadata); |
| 109 | + |
| 110 | +// perform semantic search |
| 111 | +var results = vdb.Search("something to search", pageCount: 5); |
| 112 | + |
| 113 | +// Loop through search results |
| 114 | +foreach(var item in results.Texts) { |
| 115 | + var text = item.Text; |
| 116 | + var metadata = item.Metadata; |
| 117 | + |
| 118 | + var url = metadata.Url; |
| 119 | + var author = metadata.Author; |
| 120 | + |
| 121 | + // do something with results and metadata |
| 122 | +} |
| 123 | +``` |
| 124 | + |
| 125 | +This will offer better performance with scenarios that require more complex metadata since you no longer need to handle serialization to/from JSON. |
| 126 | + |
| 127 | +!!! info "OpenAI and Ollama Support" |
| 128 | + The `OpenAIMemoryVectorDatabase<TMetadata>` and `OllamaMemoryVectorDatabase<TMetadata>` generic classes can also be used to define your own `Metadata` type when working with [OpenAI and Ollama embeddings](../../embeddings/index.md). |
0 commit comments