vd.analytics

Analytics and statistics for vd collections.

Provides functions to analyze collections, find duplicates, compute statistics, and gain insights into your vector database.

Functions

collection_stats(collection)	Compute comprehensive statistics for a collection.
find_duplicates(collection, *[, threshold, ...])	Find near-duplicate documents in a collection.
find_outliers(collection, *[, n_neighbors, ...])	Find outlier documents (those dissimilar to their neighbors).
metadata_distribution(collection, field, *)	Get the distribution of values for a metadata field.
sample_collection(collection, n, *[, ...])	Sample document IDs from a collection.
validate_collection(collection)	Validate collection integrity and identify issues.

vd.analytics.collection_stats(collection)

Compute comprehensive statistics for a collection.

Parameters:

collection (Collection) – Collection to analyze

Returns:

Statistics including:

• total_documents: Number of documents

• avg_text_length: Average text length in characters

• min_text_length: Minimum text length

• max_text_length: Maximum text length

• total_chars: Total characters across all documents

• metadata_fields: Set of all metadata fields used

• metadata_field_counts: Count of documents with each metadata field

• embedding_dimension: Dimension of embeddings (if available)

• has_vectors: Number of documents with vectors

Return type:

dict[str, Any]

Examples

>>> import vd
>>> client = vd.connect('memory')
>>> docs = client.create_collection('test')
>>> docs['doc1'] = ("Hello", {'category': 'greeting'})
>>> stats = vd.collection_stats(docs)
>>> print(stats['total_documents'])
1

vd.analytics.find_duplicates(collection, *, threshold=0.95, method='cosine')

Find near-duplicate documents in a collection.

Parameters:

• collection (Collection) – Collection to analyze

• threshold (float) – Similarity threshold above which documents are considered duplicates

• method (str) – Similarity method: ‘cosine’ or ‘exact’

Returns:

List of (doc_id1, doc_id2, similarity) tuples for duplicates

Return type:

list[tuple[str, str, float]]

Examples

>>> import vd
>>> client = vd.connect('memory')
>>> docs = client.create_collection('test')
>>> docs['doc1'] = "Hello world"
>>> docs['doc2'] = "Hello world"
>>> duplicates = vd.find_duplicates(docs)
>>> len(duplicates) > 0
True

vd.analytics.find_outliers(collection, *, n_neighbors=5, threshold=0.3)

Find outlier documents (those dissimilar to their neighbors).

Parameters:

• collection (Collection) – Collection to analyze

• n_neighbors (int) – Number of neighbors to consider

• threshold (float) – Average similarity threshold below which a document is an outlier

Returns:

List of (doc_id, avg_similarity) for outliers

Return type:

list[tuple[str, float]]

Examples

>>> import vd
>>> client = vd.connect('memory')
>>> docs = client.create_collection('test')
>>> # Add some documents...
>>> outliers = vd.find_outliers(docs)

vd.analytics.metadata_distribution(collection, field, *, top_n=None)

Get the distribution of values for a metadata field.

Parameters:

• collection (Collection) – Collection to analyze

• field (str) – Metadata field name

• top_n (Optional[int]) – If specified, return only the top N most common values

Returns:

Mapping of field values to their counts

Return type:

dict[Any, int]

Examples

>>> import vd
>>> client = vd.connect('memory')
>>> docs = client.create_collection('test')
>>> docs['doc1'] = ("Hello", {'category': 'A'})
>>> docs['doc2'] = ("World", {'category': 'A'})
>>> docs['doc3'] = ("Test", {'category': 'B'})
>>> dist = vd.metadata_distribution(docs, 'category')
>>> print(dist)
{'A': 2, 'B': 1}

vd.analytics.sample_collection(collection, n, *, method='random', seed=None)

Sample document IDs from a collection.

Parameters:

• collection (Collection) – Collection to sample from

• n (int) – Number of documents to sample

• method (str) – Sampling method: ‘random’, ‘first’, ‘diverse’

• seed (Optional[int]) – Random seed for reproducibility

Returns:

Sampled document IDs

Return type:

list[str]

Examples

>>> import vd
>>> client = vd.connect('memory')
>>> docs = client.create_collection('test')
>>> # Add 100 documents...
>>> sample = vd.sample_collection(docs, 10, method='random')
>>> len(sample)
10

vd.analytics.validate_collection(collection)

Validate collection integrity and identify issues.

Parameters:

collection (Collection) – Collection to validate

Returns:

Validation report with:

• valid: Whether collection is valid

• issues: List of issue descriptions

• warnings: List of warning messages

• stats: Basic stats

Return type:

dict[str, Any]

Examples

>>> import vd
>>> client = vd.connect('memory')
>>> docs = client.create_collection('test')
>>> report = vd.validate_collection(docs)
>>> print(report['valid'])
True