Search Configuration

We can update the configuration for our search space in order to

• customize search settings
• add or remove tag tracks from indexing (or index every track automatically)
• add or remove content level fabric metadata from indexing

This document will show you how to update the config as well as provide an overview of all the configurable settings.

Setting the config

When creating a new space (for decisive folks)

POST /spaces/<qid> HTTP/1.1
Host: https://ai.contentfabric.io/vectorstore
Authorization: Bearer <token>
Content-Type: application/json

{
	"collection_id": "<collection_id>",
	"name": "<name>",
	"type": "clip-search",
    "config": {...config stuff here...}
}

Later, when you've realized you screwed it up the first time

PATCH /spaces/<qid> HTTP/1.1
Host: https://ai.contentfabric.io/vectorstore
Authorization: Bearer <token>
Content-Type: application/json

{
	"config": {...config stuff here...}
}

Configuration Settings

Schema

The config schema is documented in openapi format: "link will go here when it exists"

Explanation

The config has two top level blocks:

Block	Purpose
indexer	Controls what gets indexed and how documents are built.
search	Controls the default behavior when searching the space.

Here is a fully filled out example config.

{
    "indexer": {
        "document": {
            "aggregation": {
                "track": "shot_detection"
            }
        },
        "auto_indexing": {
            "enabled": true,
            "ignore_tracks": [
                "shot_detection",
                "black_frames"
            ]
        },
        "fields": {
            "title": {
                "source": {
                    "fabric_paths": [
                        "public.asset_metadata.display_title",
                        "public.asset_metadata.title"
                    ]
                }
            },
            "genre": {
                "source": {
                    "fabric_paths": [
                        "public.asset_metadata.mpaa_genre"
                    ]
                }
            },
            "scene_description": {
                "source": {
                    "tag_tracks": [
                        "llava_caption",
                        "scene_description"
                    ]
                },
                "options": {
                    "chunk_strategy": "sentence"
                }
            },
            "dialogue": {
                "source": {
                    "tag_tracks": [
                        "auto_captions",
                        "transcription"
                    ]
                },
                "options": {
                    "chunk_strategy": "none"
                }
            }
        }
    },
    "search": {
        "clip_search": {
            "defaults": {
                "rerank_level": "document",
                "rerank_user_query": true,
                "clips_min_duration": 15,
                "clips_max_duration": 45
            }
        }
    }
}

indexer

Document Aggregation

"document": {
    "aggregation": {
        "track": "shot_detection"
    }
}

When a track is set, the indexer automatically creates aggregated documents for every tag in this track

• For all indexed tags (fields with a tag_tracks source) which overlap with the aggregation track's tag time-ranges, we merge these into a single textual document.
• Fabric level field data (fields with a fabric_paths source) is aggregated into every document that shares a matching content id.

What's the point?

Vector search by itself is surprisingly limited. Aggregating gives us rich contextual information for a scene that can be used to answer complex queries which might span across different fields. For example a query like: "Jennifer Lawrence talks with a mechanic in No Hard Feelings" could match 3 separate fields for cast/celebrity, visual scene description, and film title.

For more information see our docs on search design (doesn't exist at the time of writing this but when it does it will go right here)

Fixed time buckets

Instead of a track name, track may be set to a fixed time interval like "10s" or "15s" (whole seconds only). The indexer then aggregates tags into contiguous, fixed-width windows of that duration rather than following any track's own segments. This is useful when there is no suitable segmentation track (like shot_detection) to align documents to.

"document": {
    "aggregation": {
        "track": "10s"
    }
}

Field	Type	Description
track	string	Either a track name used to aggregate overlapping tags into documents (shot_detection is recommended so documents align to shot boundaries), or a fixed interval in whole seconds like "10s" / "15s" to aggregate into contiguous fixed-width time buckets.

Fabric Fields

A field is sourced from the content fabric metadata when its source contains fabric_paths.

"fields": {
    "title": {
        "source": {
            "fabric_paths": [
                "public.asset_metadata.display_title",
                "public.asset_metadata.title"
            ]
        }
    }
}

For every content in the space's collection, the indexer crawls the metadata at each path and indexes the value it finds.

Field	Type	Description
source.fabric_paths	string[]	One or more dot-separated paths into the fabric metadata. The values at each of these paths will be indexed under the provided field name.
options	object	Optional per-field settings, alongside source (see Field Options).

What's the point?

Content objects in the fabric often store important metadata which is relevant to the whole content rather than just a small slice: e.g. "title", "genre", "synopsis", "release date" etc.. We want to be able to be able to filter our search on these values as well.

Triggering fabric metadata crawling

Content object metadata is not crawled automatically and must be triggered manually with a valid token via the indexer API. If you add new contents to the collection you must recrawl.

Start crawl job (gives handle id): POST /spaces/{space_qid}/crawl Check crawl status: GET /spaces/{space_qid}/crawl/{handle_id}

Indexer API docs: https://ai.contentfabric.io/indexer/openapi.json

Tag Fields

A field is sourced from tags in the tagstore when its source contains tag_tracks. Listing multiple tracks groups their tags into a single field.

"fields": {
    "scene_description": {
        "source": {
            "tag_tracks": [
                "llava_caption",
                "scene_description"
            ]
        },
        "options": {
            "chunk_strategy": "sentence"
        }
    }
}

Field	Type	Description
source.tag_tracks	string[]	The tagstore track names to pull tags from for this field.
options	object	Optional per-field settings, alongside source (see Field Options).

Auto Indexing

Rather than listing every tag track by hand, auto indexing indexes every tag track in the tagstore, with a list of exceptions. Each auto-indexed track becomes a field named after the track itself.

"auto_indexing": {
    "enabled": true,
    "ignore_tracks": [
        "shot_detection",
        "black_frames"
    ]
}

Field	Type	Description
enabled	bool	Whether to auto index every tag track. When omitted, it defaults to true if no tag fields are configured and false otherwise (see below).
ignore_tracks	string[]	Tag tracks to exclude from auto indexing. Has no effect on tracks that are also named explicitly under fields.

Default behavior. If you leave enabled unset:

• A config with no configured fields auto indexes everything — so an empty config just works.
• As soon as you configure any field, auto indexing turns off automatically, and only your configured fields are indexed.

Set enabled explicitly to override this: true to index all tracks in addition to your configured fields, or false to index nothing beyond them.

Combining with explicit fields. Auto indexing and fields compose. An explicitly configured tag track is always indexed under its field name (letting you group or rename specific tracks), and every other non-ignored track is indexed under its own name. Explicit configuration wins even if the track is also in ignore_tracks.

note

Auto indexing applies to tag tracks only — fabric metadata paths cannot be auto-discovered and must always be listed explicitly under fields. This also means a config with only fabric fields turns auto indexing off by default; set "enabled": true if you also want all tag tracks indexed. If your aggregation track is itself a text-bearing tag track you don't want indexed as content, add it to ignore_tracks.

Field Options

These options can be applied to any field, alongside its source.

Option	Type	Default	Description
chunk_strategy	string	"sentence"	How the value is chunked into vectors. Set to "none" for no chunking.

search

Clip Search Defaults

"search": {
    "clip_search": {
        "defaults": {
            "rerank_level": "document",
            "rerank_user_query": true,
            "clips_min_duration": 15,
            "clips_max_duration": 45
        }
    }
}

The search config defines defaults for searching the space. These defaults allow you to override four of the clip_search API arguments so callers don't have to specify them on every request.

note

For detailed explanations of these arguments, consult the clip search API docs.