Store Index

26 min

The Store Index introduces two new indexes, store_product and store_sku, which replace the older catalog_location_live index.

In the old setup, catalog_location_live stored one document per product per location, resulting in:

Total documents = n_products × n_locations

This approach created redundant data and slowed performance.

The new indexes:

Store all location-specific pricing inside a nested pricing field in each document.
Reduce document count to:
- n_products (for store_product)
- n_skus (for store_sku)
Improve search performance and reduce reindexing time from hours to minutes.

Why Use These New Indexes?

Feature	catalog_location_live	store_product	store_sku
Document count	n_products × n_locations	n_products	n_skus
Redundant data	High	Low	Low
Performance	Slower	Faster	Faster
Reindex time	Hours	Minutes	Minutes
Search scope	Product+Location	Product-level	SKU-level

Advantages:

Reduced redundancy (1 doc per product/SKU)
Significantly faster indexing and querying
Flexible: choose product-level or SKU-level search
Consistent API responses, with only pricing, storage/query logic changed

Quick Start – How to Use Catalog Indexes

The Catalog Index introduces two optimized indexes to replace catalog_location_live:

store_product:Best for product-level queries. Returns a single document per product with all location-specific pricing in a nested pricing array.
store_sku: Best for SKU-level queries. Returns one document per SKU, allowing you to target individual product variants (e.g., color, size).

1. Choose the Correct Index

Use store_product when:
- You want to search or display products without showing every SKU variation.
- You only need to retrieve the top-level product details with pricing for specific locations.
Use store_sku when:
- You want searches to return individual SKUs directly.
- You need to target SKU-specific attributes like color, size, or material.

2. Build Your Query

API Routes:
- /store_product/search – Search for products.
- /store_sku/search – Search for SKUs.
- /store_product/suggest and /store_sku/suggest – Get autocomplete suggestions.
- Both indexes support image search and neural search queries.

3. Filter by Location

All pricing is location-specific. Use a nested filter to limit results to a specific store or warehouse:

{
    "nested": {
        "path": "pricing",
        "query": {
            "term": { "pricing.locationIds.keyword": { "value": "100003" } }
        }
    }
}

This ensures you only return pricing and availability for the relevant locationId.

4. Exclude the Full Pricing Array

To avoid returning every location’s pricing in your main search hits, exclude the pricing field:

"_source": { "excludes": "pricing" }

This keeps responses lightweight.

5. Retrieve Relevant Pricing with inner_hits

Use inner_hits to return only the matching location’s pricing:

"inner_hits": {
    "_source": [
        "pricing.buyOnline",
        "pricing.price",
        "pricing.onSale",
        "pricing.currency",
        "pricing.listPrice"
    ]
}

This structure is consistent between store_product and store_sku.

6. Collapse SKUs for Cleaner Product Results (Optional)

When using store_sku, you may want only one SKU per product in search results:

"collapse": { "field": "itemId.keyword" }

This groups all SKUs under the main product, with the option to access all variations via inner_hits.

7. Choose the Right Query Type

Standard Query – Basic search and filter.
Suggest Query – Autocomplete or typeahead suggestions.
Image Search – Search by visual similarity using an embedding vector.
Neural Query – Semantic search powered by natural language understanding.

Index: store_product

Purpose

The store_product index stores one document per product, with all location pricing inside a nested pricing array.

Data Structure Changes

The productItem.buyOnline method has been moved to pricing.buyOnline.
pricing contains all location price entries for that product.
Queries must use a nested filter to retrieve location-specific prices.

Standard Query

curl --location 'https://api.upstartcommerce.com/v1/search/routes/store_product/search' \
--header 'Content-Type: application/json' \
--header 'X-Upstart-Tenant: TENANT' \
--header 'X-Upstart-Site: SITE' \
--header 'X-Upstart-Session-Id: SESSION_ID' \
--data '{
    "search": {
        "query": {
            "bool": {
                "must": [
                    { "query_string": { "query": "chair*" } }
                ],
                "filter": [
                    { "term": { "locationIds.keyword": { "value": "100003" } } },
                    {
                        "nested": {
                            "path": "pricing",
                            "query": {
                                "term": {
                                    "pricing.locationIds.keyword": {
                                        "value": "100003"
                                    }
                                }
                            },
                            "inner_hits": {
                                "_source": [
                                    "pricing.buyOnline",
                                    "pricing.price",
                                    "pricing.onSale",
                                    "pricing.currency",
                                    "pricing.listPrice"
                                ]
                            }
                        }
                    }
                ]
            }
        },
        "_source": { "excludes": "pricing" },
        "mostValuableQuery": { "query": "chair" }
    }
}'

Suggest Query

curl --location 'https://api.upstartcommerce.com/v1/search/routes/store_product/suggest' \
--header 'X-Upstart-Tenant: TENANT' \
--header 'Content-Type: application/json' \
--header 'X-UpStart-Session-Id: SESSION_ID' \
--header 'X-UpStart-Site: SITE' \
--data '{
    "suggest": {
        "prefix": "chair",
        "size": 10,
        "suggest_fields": ["id", "name", "dynamicProperties.description"],
        "filter": {
            "bool": {
                "must": [
                    { "query_string": { "query": "chair*" } }
                ],
                "filter": [
                    { "term": { "locationIds.keyword": { "value": "100003" } } },
                    {
                        "nested": {
                            "path": "pricing",
                            "query": {
                                "term": {
                                    "pricing.locationIds.keyword": {
                                        "value": "100003"
                                    }
                                }
                            }
                        }
                    }
                ]
            }
        },
        "_source": { "includes": ["id", "media"] },
        "highlight": { "pre_tags": ["<strong>"], "post_tags": ["</strong>"] }
    },
    "mostValuableQuery": { "query": "chair" }
}'

Image Search Query

curl --location 'https://api.upstartcommerce.com/v1/search/routes/store_sku/search' \
--header 'Content-Type: application/json' \
--header 'X-Upstart-Tenant: TENANT' \
--header 'x-upstart-Site: SITE' \
--header 'x-upstart-Session-Id: SESSION_ID' \
--data '{
    "search": {
        "query": {
            "bool": {
                "must": [
                    {
                        "nested": {
                            "path": "dynamicProperties.media_url_image_model_embedding",
                            "query": {
                                "knn": {
                                    "dynamicProperties.media_url_image_model_embedding.knn": {
                                        "vector": [VECTOR_RETURNED_FROM_IMAGE_SEARCH_SVC],
                                        "k": 10
                                    }
                                }
                            }
                        }
                    }
                ],
                "filter": [
                    { "term": { "locationIds.keyword": { "value": "100003" } } },
                    {
                        "nested": {
                            "path": "pricing",
                            "query": {
                                "term": {
                                    "pricing.locationIds.keyword": {
                                        "value": "100003"
                                    }
                                }
                            },
                            "inner_hits": {
                                "_source": [
                                    "pricing.buyOnline",
                                    "pricing.price",
                                    "pricing.onSale",
                                    "pricing.currency",
                                    "pricing.listPrice"
                                ]
                            }
                        }
                    }
                ]
            }
        },
        "_source": { "excludes": "pricing" },
        "mostValuableQuery": { "query": "chair" }
    }
}'

Neural Query

curl --location 'https://api.upstartcommerce.com/v1/search/routes/store_product/search' \
--header 'Content-Type: application/json' \
--header 'X-Upstart-Tenant: TENANT' \
--header 'X-Upstart-Session-Id: SESSION_ID' \
--header 'X-Upstart-Site: SITE' \
--data '{
    "search": {
        "query": {
            "bool": {
                "filter": [
                    { "term": { "locationIds.keyword": { "value": "100062" } } },
                    {
                        "nested": {
                            "path": "pricing",
                            "query": {
                                "term": {
                                    "pricing.locationIds.keyword": { "value": "100062" }
                                }
                            },
                            "inner_hits": {
                                "_source": [
                                    "pricing.buyOnline",
                                    "pricing.price",
                                    "pricing.assetIds",
                                    "pricing.onSale",
                                    "pricing.currency",
                                    "pricing.listPrice"
                                ]
                            }
                        }
                    }
                ],
                "must": [
                    {
                        "nested": {
                            "path": "dynamicProperties.neural_description_semantic_catalog_model_embedding",
                            "query": {
                                "neural": {
                                    "dynamicProperties.neural_description_semantic_catalog_model_embedding.knn": {
                                        "query_text": "chair",
                                        "model_id": "semantic_catalog_model",
                                        "k": 10,
                                        "filter": { "term": { "locationIds.keyword": { "value": "100003" } } }
                                    }
                                }
                            }
                        }
                    }
                ]
            },
            "_source": { "excludes": "pricing" }
        },
        "mostValuableQuery": { "query": "chair" }
    }
}'

Index: store_sku

Purpose

The store_sku index stores one document per SKU, enabling SKU-specific search results.

This is useful when products have multiple SKUs with distinct attributes (e.g., color or size), and you want to allow users to search for those variations directly.

Data Structure Changes

The productItem.buyOnline method has been moved to pricing.buyOnline.
pricing contains all location price entries for that SKU.
Queries must use a nested filter to retrieve location-specific prices.

Standard Query

curl --location 'https://api.upstartcommerce.com/v1/search/routes/store_sku/search' \
--header 'Content-Type: application/json' \
--header 'X-Upstart-Tenant: TENANT' \
--header 'X-Upstart-Site: SITE' \
--header 'X-Upstart-Session-Id: SESSION_ID' \
--data '{
    "search": {
        "query": {
            "bool": {
                "must": [
                    { "query_string": { "query": "chair*" } }
                ],
                "filter": [
                    { "term": { "locationIds.keyword": { "value": "100062" } } },
                    {
                        "nested": {
                            "path": "pricing",
                            "query": {
                                "term": {
                                    "pricing.locationIds.keyword": { "value": "100062" }
                                }
                            },
                            "inner_hits": {
                                "_source": [
                                    "pricing.buyOnline",
                                    "pricing.price",
                                    "pricing.assetIds",
                                    "pricing.onSale",
                                    "pricing.currency",
                                    "pricing.listPrice"
                                ]
                            }
                        }
                    }
                ]
            },
            "collapse": { "field": "itemId.keyword" },
            "_source": { "excludes": "pricing" }
        },
        "mostValuableQuery": { "query": "chair" }
    }
}'

Suggest Query

curl --location 'https://api.upstartcommerce.com/v1/search/routes/store_sku/suggest' \
--header 'X-Upstart-Tenant: TENANT' \
--header 'Content-Type: application/json' \
--header 'X-UpStart-Session-Id: SESSION_ID' \
--header 'X-UpStart-Site: SITE' \
--data '{
    "suggest": {
        "prefix": "chair",
        "size": 10,
        "suggest_fields": ["id", "name", "dynamicProperties.description"],
        "filter": {
            "bool": {
                "must": [
                    { "query_string": { "query": "chair*" } }
                ],
                "filter": [
                    { "term": { "locationIds.keyword": { "value": "100003" } } },
                    {
                        "nested": {
                            "path": "pricing",
                            "query": {
                                "term": {
                                    "pricing.locationIds.keyword": { "value": "100003" }
                                }
                            }
                        }
                    }
                ]
            }
        },
        "collapse": { "field": "itemId.keyword" },
        "_source": { "includes": ["id", "media"] },
        "highlight": { "pre_tags": ["<strong>"], "post_tags": ["</strong>"] }
    },
    "mostValuableQuery": { "query": "chair" }
}'

Image Search Query

curl --location 'https://api.upstartcommerce.com/v1/search/routes/store_sku/search' \
--header 'X-Upstart-Tenant: TENANT' \
--header 'Content-Type: application/json' \
--header 'X-UpStart-Session-Id: SESSION_ID' \
--header 'X-UpStart-Site: SITE' \
--data '{
    "search": {
        "query": {
            "bool": {
                "must": [
                    {
                        "nested": {
                            "path": "dynamicProperties.media_url_image_model_embedding",
                            "query": {
                                "knn": {
                                    "dynamicProperties.media_url_image_model_embedding.knn": {
                                        "vector": [VECTOR_RETURNED_FROM_IMAGE_SEARCH_SVC],
                                        "k": 10
                                    }
                                }
                            }
                        }
                    }
                ],
                "filter": [
                    { "term": { "locationIds.keyword": { "value": "100003" } } },
                    {
                        "nested": {
                            "path": "pricing",
                            "query": {
                                "term": {
                                    "pricing.locationIds.keyword": { "value": "100003" }
                                }
                            },
                            "inner_hits": {
                                "_source": [
                                    "pricing.buyOnline",
                                    "pricing.price",
                                    "pricing.onSale",
                                    "pricing.currency",
                                    "pricing.listPrice"
                                ]
                            }
                        }
                    }
                ]
            }
        },
        "collapse": { "field": "itemId.keyword" },
        "_source": { "excludes": "pricing" },
        "mostValuableQuery": { "query": "chair" }
    }
}'

Neural Query

curl --location 'https://api.upstartcommerce.com/v1/search/routes/store_sku/search' \
--header 'Content-Type: application/json' \
--header 'X-Upstart-Tenant: TENANT' \
--header 'X-Upstart-Site: SITE' \
--header 'X-Upstart-Session-Id: SESSION_ID' \
--data '{
    "search": {
        "query": {
            "bool": {
                "filter": [
                    { "term": { "locationIds.keyword": { "value": "100062" } } },
                    {
                        "nested": {
                            "path": "pricing",
                            "query": {
                                "term": {
                                    "pricing.locationIds.keyword": { "value": "100062" }
                                }
                            },
                            "inner_hits": {
                                "_source": [
                                    "pricing.buyOnline",
                                    "pricing.price",
                                    "pricing.assetIds",
                                    "pricing.onSale",
                                    "pricing.currency",
                                    "pricing.listPrice"
                                ]
                            }
                        }
                    }
                ],
                "must": [
                    {
                        "nested": {
                            "path": "dynamicProperties.neural_description_semantic_catalog_model_embedding",
                            "query": {
                                "neural": {
                                    "dynamicProperties.neural_description_semantic_catalog_model_embedding.knn": {
                                        "query_text": "chair",
                                        "model_id": "semantic_catalog_model",
                                        "k": 10,
                                        "filter": { "term": { "locationIds.keyword": { "value": "100003" } } }
                                    }
                                }
                            }
                        }
                    }
                ]
            },
            "collapse": { "field": "itemId.keyword" },
            "_source": { "excludes": "pricing" }
        },
        "mostValuableQuery": { "query": "chair" }
    }
}'

Pricing in Search Responses

When using either store_product or store_sku, location-specific pricing is not returned in the main _source by default; it’s retrieved using inner_hits on the pricing nested field.

Example Search Response Snippet

"inner_hits": {
    "pricing": {
        "hits": {
            "total": {
                "relation": "eq",
                "value": 1
            },
            "hits": [
                {
                    "_index": "sp-denvermattress-default-store_product-000021",
                    "_id": "0d134a35-08a8-4f1c-b177-1602b17013f3",
                    "_score": 1.0506582,
                    "_nested": {
                        "field": "pricing",
                        "offset": 0
                    },
                    "_source": {
                        "buyOnline": true,
                        "price": 659,
                        "onSale": false,
                        "currency": "USD",
                        "listPrice": 659
                    }
                }
            ],
            "max_score": 1.0506582
        }
    }
}

Field Definitions

Field	Description
buyOnline	Boolean. Indicates whether the product/SKU can be purchased online for the given location.
price	Active selling price for the location. This can be a sale price if available; otherwise, it will be the list or MSRP price.
onSale	Boolean. true if the price field is a sale price. Useful for filtering and faceting.
currency	Currency code (e.g., USD).
listPrice	List or MSRP price for the given location.