Process

7min
The Image Search API works in several steps:
Detect products in an uploaded image
Extract features from specific products
Use these features to search for similar products
Steps to Enable Image Search for Your Online Store
ML Model and Index Setup
Contact the UpStart Commerce team to set up the index and deploy the image search machine learning model. Note: If you are storing your images in a third-party solution, please contact us for assistance integrating them seamlessly into our system.
You will need designated fields that contain an image URL for each product. Our model will retrieve all image URLs found under the fields "media" and "productItems.media".
Reindex Catalog
Once the model and index are configured, reindex your catalog.
This process generates vector representations for each product based on the images in your catalog, enabling accurate image-based matching.
Test and Refine
Run test queries to confirm that the image search is functioning as expected. Adjust configurations if necessary to improve results.
Start Using Image Search
After configuration and reindexing, image search will be active for your store, allowing users to search your catalog effortlessly by uploading images.
Step-by-Step Guide
First, you'll need to upload an image and get the coordinates of each product detected in it.
1. Use the following API endpoint:
POST-https://nochannel-dev-1-api.nochannel-dev.upstart.team/v1/ml/image-search/detect-objects
2. Include the following headers:
accept: application/json
x-upstart-tenant: [Your Tenant Identifier]
x-upstart-site: [Your Site ID]
x-upstart-api-key: [Your API Key]
X-UpStart-WebSessionId: [Random UUID to Trace Requests]
x-upstart-customerId: [Customer ID]
Note: You can also use an anonymous token
3. In the request body, include the image file.
4. The API will return a list of detected products with their corresponding coordinates and confidence scores.
5. Convert your image to a Base64-encoded string. You can use online tools or libraries in your preferred programming language to do this.
To process a specific product in the image, start by identifying the area you want to focus on (the “coordinates”) from the response in Step 1. Then, use the feature extraction endpoint:
Endpoint: POST https://api.upstartcommerce.com/v1/ml/image-search/extract-features
Request Body:
Base64-encoded image string: The image you’re using, is converted to a Base64 string format.
Product coordinates: The exact area of the image you want to analyze, formatted as follows: 
[
X-coordinate of the left side of the selected area,
Y-coordinate of the top of the selected area,
X-coordinate of the right side of the selected area,
Y-coordinate of the bottom of the selected area
]
This setup will return an AI-generated vector that uniquely represents the specific object you’ve highlighted in the image.
To search for similar products using image search, use the following endpoint:
Endpoint: POST https://api.upstartcommerce.com/v1/search/routes/catalog_location_live/search
Request Query Example
The following curl command performs an image search query, using the embeddings from Step 3 to return a list of similar products:
Curl
curl --location 'https://api.upstartcommerce.com/v1/search/routes/catalog_location_live/search' \
--header 'Content-Type: application/json' \
--header 'X-Upstart-Tenant: denvermattress' \
--header 'x-upstart-api-key: ZmFiYjYyYWQtMDE2Mi00NzBlLWFmMWQtYTg1MjcwOTN0MTI' \
--data '{

  "search": {
    "_source": {
      "excludes": "*knn"
    },
    "size": "20",
    "query": {
      "bool": {
        "must": [
          {
            "nested": {
              "path": "dynamicProperties.url_image_model_embedding",
              "query": {
                "knn": {
                  "dynamicProperties.url_image_model_embedding.knn": {
                    "vector": [], 
                    "k": 20
                  }
                }
              }
            }
          }
        ]
      }
    }
  }
}
curl --location 'https://api.upstartcommerce.com/v1/search/routes/catalog_location_live/search' \
--header 'Content-Type: application/json' \
--header 'X-Upstart-Tenant: denvermattress' \
--header 'x-upstart-api-key: ZmFiYjYyYWQtMDE2Mi00NzBlLWFmMWQtYTg1MjcwOTN0MTI' \
--data '{

  "search": {
    "_source": {
      "excludes": "*knn"
    },
    "size": "20",
    "query": {
      "bool": {
        "must": [
          {
            "nested": {
              "path": "dynamicProperties.url_image_model_embedding",
              "query": {
                "knn": {
                  "dynamicProperties.url_image_model_embedding.knn": {
                    "vector": [], 
                    "k": 20
                  }
                }
              }
            }
          }
        ]
      }
    }
  }
}
﻿
Explanation of Key Fields
_source.excludes: This tells the system not to include technical data (like embeddings) in the search results, since this extra data isn’t needed on the storefront. Excluding it makes the search faster and the response smaller.
size: This is the number of similar products you want to show in the results. It should be the same as k (see below) to ensure the complete set of results is returned.
path: Specifies where to look for the image data in each product, so it searches only the image field you’ve chosen.
vector: This is where you include the specific vector (or “code”) from Step 3 that represents the visual features of the image you’re searching for.
k: Similar to size, this is the number of products you want to match with your image. Make sure k is the same as size so all expected results are included.
Error Handling
If you receive an error response, check that all required parameters are correctly included in your request.
Ensure your API key and other credentials are valid and up-to-date.
If issues persist, please contact our support team with the error details and a sample of the problematic request (excluding any sensitive information).
﻿
Image Search
Enable Image Search with PIM