Documentation
¶
Overview ¶
Package vectara implements Core indexing, semantic search, and ID deletion through Vectara's current v2 HTTP API. The host owns corpus provisioning, credentials, transport, and HTTP client lifetime. Vectara owns embedding and native ranking; no Scope model, distance setting, OAuth refresh, or retry layer competes with that policy. The native corpus must be enabled, store documents, and have no custom score dimensions. This policy is reread before every complete source operation.
The current source has one native document ID, one core document part containing the complete text, and exactly one document metadata field, metadata_json. Its value is the complete Core JSON string. Numbers, arbitrary keys, nested values, null metadata, and empty objects survive without conversion to native scalar filter types. Media, multiple parts, separate context, tables, and images are unsupported and rejected. Native IDs remain the sole stored identity.
NewStore validates the native corpus policy without enumerating documents. Unfiltered Search issues native ranking directly after checking that policy. Index and filtered Search enumerate the corpus with opaque page keys and read each full document by ID. Repeated cursors or IDs and malformed scanned records fail. Enumeration requires document read permissions and costs one listing call per page plus one retrieval per document, with retained data proportional to the selected documents.
Search ¶
Core filter.Match is the sole metadata predicate evaluator. Filtered searches restrict native ranking to selected doc.id values in bounded expressions using Vectara's built-in ID filter, requiring no custom filter attributes. All identity expressions are prepared before querying and must fit the native 8,000-character limit. Native raw relevance determines global TopK before MinScore; every hit must have a current text result shape, unique identity, complete metadata, and unchanged Core membership. Any error returns no partial response.
Requests disable lexical interpolation, reranking, query rewriting, streaming, and saved history. The generation object is omitted to disable summarization. Vectara's bounded semantic relevance [-1,1] maps linearly to Core [0,1]; non-finite or out-of-range values are errors. Native text parts are explicitly supplied by Index, each mapping directly to a search result; the adapter does not rely on server-side chunking or return snippets as documents.
Writes and deletion ¶
Index validates and prepares the complete request and batcher output before mutation. Vectara's document creation endpoint rejects existing IDs, so each replacement uses native ID deletion followed by creation with wait_for=searchable. This is not atomic: readers can observe the gap, and a failed creation can leave the previous document deleted. A confirmed creation must return HTTP 201 and the requested native ID. Failed native operations are returned without retries. DeleteIDs validates all IDs first, deduplicates them, and ignores only native not-found; every other status remains a failure. Earlier IDs can be removed if a later deletion fails. The native service owns its document-change conflicts.
Breaking contract ¶
DeleteWhere and the FilterDeleter capability, MetadataPrefix, DefaultAPIVersion, and the metadata DSL compiler have been removed. Native filter deletion is best effort and can miss matching documents through indexing lag; selecting IDs and deleting them unconditionally also loses the original metadata condition. These paths cannot implement Core's complete predicate deletion contract. Explicit ID deletion is a separate caller intent, not a predicate deletion substitute.
Rebuild or reindex the corpus through the host before construction when its records do not have the current complete JSON and single-part shape. No legacy reads, metadata projections, default-prefix aliases, or schema migrations remain.
Default tests exercise the current HTTP protocol offline. Integration tests require SCOPE_VECTARA_URL and SCOPE_VECTARA_API_KEY with corpus creation/deletion permissions and create disposable corpora. Managed-service checks require those credentials; offline protocol tests do not demonstrate native cloud execution. Current protocol references: https://docs.vectara.com/docs/rest-api/list-corpus-documents https://docs.vectara.com/docs/rest-api/get-corpus-document https://docs.vectara.com/docs/rest-api/bulk-delete-corpus-documents https://docs.vectara.com/docs/learn/metadata-search-filtering/ootb-metadata-filters
Index ¶
Examples ¶
Constants ¶
const ( Provider = "Vectara" DefaultEndpoint = "https://api.vectara.io" DefaultMaxResponseBytes = int64(16 * 1024 * 1024) )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
func NewStore ¶
func NewStore(ctx context.Context, config StoreConfig) (*Store, error)
Example ¶
package main
import (
"context"
"net/http"
"os"
"time"
"github.com/Tangerg/scope/core/document"
"github.com/Tangerg/scope/core/vectorstore"
"github.com/Tangerg/scope/vectorstores/vectara"
)
type exampleBatcher struct{}
func (e exampleBatcher) Batch(_ context.Context, docs []*document.Document) ([][]*document.Document, error) {
return [][]*document.Document{docs}, nil
}
func main() {
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()
// The host has provisioned an empty or current-shape native corpus.
store, err := vectara.NewStore(ctx, vectara.StoreConfig{APIKey: os.Getenv("VECTARA_API_KEY"), CorpusKey: "documents", DocumentBatcher: exampleBatcher{}, HTTPClient: &http.Client{Timeout: 30 * time.Second}})
if err != nil {
panic(err)
}
if err = store.Index(ctx, &vectorstore.IndexRequest{Documents: []*document.Document{{ID: "one", Text: "hello"}}}); err != nil {
panic(err)
}
if _, err = store.Search(ctx, &vectorstore.SearchRequest{Query: "hello"}); err != nil {
panic(err)
}
if err = store.DeleteIDs(ctx, []string{"one"}); err != nil {
panic(err)
}
}
Output:
func (*Store) Index ¶
func (s *Store) Index(ctx context.Context, request *vectorstore.IndexRequest) error
func (*Store) Search ¶
func (s *Store) Search(ctx context.Context, request *vectorstore.SearchRequest) (response *vectorstore.SearchResponse, err error)
type StoreConfig ¶
type StoreConfig struct {
Endpoint string
APIKey string
CorpusKey string
DocumentBatcher vectorstore.Batcher
// HTTPClient remains host-owned. Scope snapshots client settings and
// restricts requests to Endpoint's origin while sharing its transport and jar.
HTTPClient *http.Client
MaxResponseBytes int64
}
func (StoreConfig) Validate ¶
func (s StoreConfig) Validate() error
Source Files