Process

the image search api works in several steps detect products in an uploaded image extract features from specific products use these features to search for similar products steps to enable image search for your online store ml model and index setup contact the upstart commerce team to set up the index and deploy the image search machine learning model note if you are storing your images in a third party solution, please contact us for assistance integrating them seamlessly into our system you will need designated fields that contain an image url for each product our model will retrieve all image urls found under the fields "media" and "productitems media" reindex catalog once the model and index are configured, reindex your catalog this process generates vector representations for each product based on the images in your catalog, enabling accurate image based matching test and refine run test queries to confirm that the image search is functioning as expected adjust configurations if necessary to improve results start using image search after configuration and reindexing, image search will be active for your store, allowing users to search your catalog effortlessly by uploading images step by step guide first, you'll need to upload an image and get the coordinates of each product detected in it 1\ use the following api endpoint post https //nochannel dev 1 api nochannel dev upstart team/v1/ml/image search/detect objects 2\ include the following headers accept application/json x upstart tenant \[your tenant identifier] x upstart site \[your site id] x upstart api key \[your api key] x upstart websessionid \[random uuid to trace requests] x upstart customerid \[customer id] note you can also use an anonymous token 3\ in the request body, include the image file 4\ the api will return a list of detected products with their corresponding coordinates and confidence scores 5\ convert your image to a base64 encoded string you can use online tools or libraries in your preferred programming language to do this to process a specific product in the image, start by identifying the area you want to focus on (the “coordinates”) from the response in step 1 then, use the feature extraction endpoint endpoint post https //api upstartcommerce com/v1/ml/image search/extract features request body base64 encoded image string the image you’re using, is converted to a base64 string format product coordinates the exact area of the image you want to analyze, formatted as follows \[ x coordinate of the left side of the selected area, y coordinate of the top of the selected area, x coordinate of the right side of the selected area, y coordinate of the bottom of the selected area x coordinate of the left side of the selected area, y coordinate of the top of the selected area, x coordinate of the right side of the selected area, y coordinate of the bottom of the selected area ] this setup will return an ai generated vector that uniquely represents the specific object you’ve highlighted in the image to search for similar products using image search, use the following endpoint endpoint post https //api upstartcommerce com/v1/search/routes/catalog location live/search request query example the following curl command performs an image search query, using the embeddings from step 3 to return a list of similar products curl location 'https //api upstartcommerce com/v1/search/routes/catalog location live/search' \\ \ header 'content type application/json' \\ \ header 'x upstart tenant denvermattress' \\ \ header 'x upstart api key zmfiyjyyywqtmde2mi00nzbllwfmmwqtytg1mjcwotn0mti' \\ \ data '{ "search" { " source" { "excludes" " knn" }, "size" "20", "query" { "bool" { "must" \[ { "nested" { "path" "dynamicproperties url image model embedding", "query" { "knn" { "dynamicproperties url image model embedding knn" { "vector" \[], "k" 20 } } } } } ] } } } } explanation of key fields source excludes this tells the system not to include technical data (like embeddings) in the search results, since this extra data isn’t needed on the storefront excluding it makes the search faster and the response smaller size this is the number of similar products you want to show in the results it should be the same as k (see below) to ensure the complete set of results is returned path specifies where to look for the image data in each product, so it searches only the image field you’ve chosen vector this is where you include the specific vector (or “code”) from step 3 that represents the visual features of the image you’re searching for k similar to size , this is the number of products you want to match with your image make sure k is the same as size so all expected results are included error handling if you receive an error response, check that all required parameters are correctly included in your request ensure your api key and other credentials are valid and up to date if issues persist, please contact our support team with the error details and a sample of the problematic request (excluding any sensitive information)