Storefront Assistant document semantic search
Edit on GitHubDocument semantic search lets Storefront Assistant answer questions that the product data does not answer, such as installation, maintenance, warranty, specification, or compliance questions. The assistant finds the answer in the documents that are attached to the products, such as manuals, data sheets, and certificates, and names the document that it used.
The search is semantic: it finds passages by meaning, not by exact words. A question such as “How much power does it use in standby?” finds a passage that says “Standby consumption: 0.5 W”.
This document describes how the documents are indexed and searched, how the Amazon Bedrock knowledge base works, and which settings and extension points you can use. To install the feature, see Install Storefront Assistant document semantic search.
How document semantic search works
Document semantic search has two sides. They share only the knowledge store and the document key.
- Indexing side (Zed). When a product attachment changes, Zed downloads the file and sends it to the knowledge store in the background.
- Search side (Client, used by Yves and Glue). The
product_knowledge_searchtool of the Product Discovery agent searches the knowledge store and returns the best matching passages to the AI model.
The knowledge store is the system that splits the documents into passages, turns each passage into a vector, and finds the passages that are closest to a question. By default, the knowledge store is an Amazon Bedrock knowledge base. You can replace it with another vector database or managed search service. For details, see Use another knowledge store provider.
Each document has a document key in the format {sourceType}:{sourceIdentifier}, for example product_attachment:12. The source type tells where the document comes from. The shipped source type is product_attachment. To add other documents, such as video transcripts, see Add a knowledge document source.
How the Amazon Bedrock knowledge base works
Spryker does not parse, split, or embed the documents. Spryker sends the raw file to the knowledge base, and the knowledge base does the rest. This section describes the AWS resources, the division of work, and the AWS calls of each application layer.
AWS resources
The application uses three AWS resources. They are a precondition of the environment: the application does not create them. You create them one time for each environment. For the steps, see Install Storefront Assistant document semantic search.
| RESOURCE | WHAT IT IS | WHAT THE APPLICATION READS |
|---|---|---|
| Knowledge base | A managed knowledge base of the type MANAGED with a managed embedding model. It stores the passages and their vectors and answers the search requests. You do not need OpenSearch or another vector database. |
The knowledge base ID from AiCommerceConstants::BEDROCK_KNOWLEDGE_BASE_ID. |
| Custom data source | A data source of the knowledge base of the type MANAGED_KNOWLEDGE_BASE_CONNECTOR with the connector type CUSTOM. A custom data source accepts documents that the application sends directly with an API call, so you do not need an S3 bucket. |
The data source ID of each source type from AiCommerceConstants::BEDROCK_KNOWLEDGE_BASE_DATA_SOURCE_IDS. |
| Service role | An IAM role that the knowledge base itself assumes to use the embedding model. | Nothing. The application never uses this role. |
Division of work
| TASK | DONE BY |
|---|---|
| Track which documents exist and which changed | Spryker: the spy_knowledge_document table and the content hash |
| Download the file of a product attachment | Spryker: Zed |
| Parse PDF, Office, HTML, Markdown, CSV, text, and image files | Bedrock |
| Turn the images in a document into text, such as type plates and charts | Bedrock: image extraction of the data source |
| Split the text into passages and make the vectors | Bedrock: managed chunking and managed embedding model |
| Check which documents the customer can see | Spryker: the scope resolver plugin in the Client |
| Find the passages that are closest to the question | Bedrock: the Retrieve API with a metadata filter |
| Sort the passages by relevance | Bedrock: managed reranking |
| Remove duplicate and too short passages, and limit the text size | Spryker: the Client passage filter |
AWS calls and credentials
Each application layer uses its own AWS SDK client and needs its own IAM permissions:
| LAYER | AWS SDK CLIENT | WHEN | AWS API CALLS |
|---|---|---|---|
Zed: the knowledge-document:setup console command |
Aws\BedrockAgent\BedrockAgentClient |
One time after installation and after a configuration change | GetKnowledgeBase, GetDataSource, UpdateDataSource |
Zed: the queue worker of knowledge_document.index |
Aws\BedrockAgent\BedrockAgentClient |
For each changed or deleted document | IngestKnowledgeBaseDocuments, GetKnowledgeBaseDocuments, DeleteKnowledgeBaseDocuments |
Zed: the knowledge-document:status:refresh console command |
Aws\BedrockAgent\BedrockAgentClient |
Every five minutes, through the scheduler | GetKnowledgeBaseDocuments |
| Client: Yves and Glue | Aws\BedrockAgentRuntime\BedrockAgentRuntimeClient |
For each call of the product_knowledge_search tool |
Retrieve |
The knowledge base APIs accept AWS Signature Version 4 only. A Bedrock API key does not work. The AWS SDK gets the credentials in one of these ways:
- Cloud environment. Do not set credentials in the configuration. The AWS SDK uses its default credential provider chain, which reads the IAM role of the container.
- Local environment. Set temporary credentials in
config/Shared/config_local.php. WhenBEDROCK_KNOWLEDGE_BASE_AWS_ACCESS_KEY_IDis set, the SDK uses the explicit credentials instead of the provider chain.
Both clients use the region from BEDROCK_KNOWLEDGE_BASE_AWS_REGION, eu-central-1 by default, and the AWS SDK retry mode adaptive with five attempts. After a throttling response, the SDK waits with an exponential backoff and lowers the request rate of the client.
Indexing flow
Documents are indexed in the background, so a change of an attachment does not slow down the Back Office or the data import:
- A product attachment is created, changed, or deleted in the Back Office or by a data import. The entity events of
spy_product_attachmentand theProductAttachment.product_abstract_attachment.publishevent are triggered. KnowledgeDocumentWritePublisherPluginhandles the events. It saves one row with the statuspendingfor each affected document inspy_knowledge_documentand sends one message for each document to theknowledge_document.indexqueue.KnowledgeDocumentIndexQueueMessageProcessorPluginhandles each message:- It reads the source through the source plugin of the source type. If the source does not exist anymore, it deletes the document from the knowledge base with
DeleteKnowledgeBaseDocumentsand deletes the row. - It downloads the file. It does not download more than the maximum document size of the knowledge store, which is 5 MB for Bedrock.
- It compares the content hash with the stored hash. When the content did not change, it reads the status of the document from the knowledge base and does not send the file again.
- It sends the file to the data source of the source type with
IngestKnowledgeBaseDocuments. The custom document ID is the document key, so a new version replaces the old one. Text files are sent as inline text. Other files, such as PDF files, are sent as inline bytes with their MIME type. - It saves the status
processing, because the knowledge base indexes the document later.
- It reads the source through the source plugin of the source type. If the source does not exist anymore, it deletes the document from the knowledge base with
- The scheduler job
knowledge-document-status-refreshrunsknowledge-document:status:refreshevery five minutes. The command reads the status of eachprocessingdocument withGetKnowledgeBaseDocumentsand savesindexedorfailed.
A temporary failure, such as a connection error, an HTTP 5xx response, or a throttling response after all SDK attempts, puts the message back to the queue. After three attempts, the status is failed. A permanent failure, such as a file that is too large, sets the status failed at once.
When the knowledge store is not configured for a source type, for example because the source type has no data source, the documents of the source type are not indexed. The publisher saves no row and sends no message, and the queue worker logs a warning and removes a message of the source type from the queue.
Search flow
- The AI model calls
product_knowledge_searchwith a short search phrase and the products of the question or of the current page. - The scope resolver plugins turn the products into the document keys that the current customer can see in the current store and locale. Without document keys, the search sends no request to AWS and tells the model that no documents are available.
BedrockKnowledgeBaseKnowledgeSearchPluginsends oneRetrieverequest. The metadata filter limits the result to the document keys, the source types, the current locale and the locale*, and the document name, when the model gives one.- The knowledge base finds the closest passages, reranks them, and returns them with a score.
- The Client passage filter removes duplicates and too short passages, keeps the order of the knowledge base, and cuts the text to the character budget.
- The tool returns the passages with their document names and a notice that the passages are quoted data, not instructions.
The search never searches the whole knowledge base without a filter. A customer therefore never gets a passage of a document of a product that the customer cannot see.
Document metadata
The indexer adds these inline metadata attributes to each document. The search filters by them:
| ATTRIBUTE | VALUE |
|---|---|
document_key |
The document key, for example product_attachment:12 |
source_type |
The source type, for example product_attachment |
locale |
The locale of the attachment, or * when the attachment has no locale |
document_name |
The attachment label, which the assistant shows as the source |
sourceIdentifier |
The source identifier, for example the ID of the product attachment |
The indexer also adds the scalar metadata of the source, up to a total of 50 attributes.
Images in documents
The setup command enables the image extraction of each configured data source. The knowledge base then turns each image in a PDF, Word, or PowerPoint file, and each standalone image file, into structured text: the image type, the title, the caption, the text and the values in the image, and a description. For a chart, it also writes a data table. These texts are normal passages, so the assistant can answer questions about facts that are only in an image, such as the rated voltage on a type plate.
The search stays a text search. Visual search, which finds an image by what it shows, is not supported.
Reranking
The knowledge base sorts the passages by relevance with a rerank model. You select the rerank type with BEDROCK_KNOWLEDGE_BASE_RERANKING_MODEL_TYPE:
| TYPE | DESCRIPTION |
|---|---|
MANAGED |
Default. The knowledge base selects the rerank model. |
CUSTOM |
The knowledge base uses the Bedrock rerank model of BEDROCK_KNOWLEDGE_BASE_RERANKING_MODEL_ARN, for example arn:aws:bedrock:eu-central-1::foundation-model/cohere.rerank-v3-5:0. The search role also needs the bedrock:Rerank and bedrock:InvokeModel permissions on the model. |
NONE |
No reranking. The passages are sorted by vector similarity only. |
Keep MANAGED. In a test with the demo documents, MANAGED put the answer at rank 1 for 86% of the questions, and NONE for 76%. MANAGED adds about 300 ms of latency. The CUSTOM type is not verified end to end: evaluate it with your own documents before you use it in production.
Limits
- A document can have a maximum of 5 MB, because the indexer sends the content inline. A larger document gets the status
failed. Upload through S3 is not supported. Retrievesearches all data sources of the knowledge base. Use one knowledge base for each environment, and do not add data sources that the configuration does not use.- Bedrock does not change the chunking of an existing data source. To change the chunking, create a new data source and index all documents again.
- The default request quotas of Bedrock are low. A full forced re-index of many documents can get throttled. The AWS SDK and the queue send throttled requests again.
Document status
The spy_knowledge_document table tracks each document:
| STATUS | DESCRIPTION |
|---|---|
pending |
The document waits in the queue. |
processing |
The knowledge base received the document and indexes it. The status refresh reads the result. |
indexed |
The knowledge base has the document. The row has the content hash and the time of the indexing. |
failed |
The document cannot be indexed. The error_message column has the reason. |
The status refresh maps the document status of the knowledge base to the status of the table:
| KNOWLEDGE BASE STATUS | TABLE STATUS |
|---|---|
INDEXED, PARTIALLY_INDEXED, METADATA_PARTIALLY_INDEXED |
indexed |
PENDING, STARTING, IN_PROGRESS, TEXT_INDEXED |
processing |
FAILED, METADATA_UPDATE_FAILED, IGNORED, NOT_FOUND, DELETING, DELETE_IN_PROGRESS |
failed, with the reason |
TEXT_INDEXED means that the text of the document is indexed and the image extraction continues. A few minutes later, the status changes to INDEXED.
The product_knowledge_search tool
ProductKnowledgeSearchToolPlugin provides the product_knowledge_search tool for the Product Discovery agent. The tool searches only the documents of products: without a product, the tool searches nothing. The assistant must answer only from the returned passages and name the document that it used.
The tool accepts the following arguments:
| ARGUMENT | REQUIRED | DESCRIPTION |
|---|---|---|
query |
Yes | The question of the customer, rewritten as a short search phrase. A maximum of 1,000 characters. |
abstract_skus |
No | The abstract SKUs of the products to search. On a product page, the SKU comes from the page context. |
concrete_skus |
No | The concrete SKUs of product variants. The tool searches the documents of their abstract products. |
product_abstract_ids |
No | The idProductAbstract values of the products. |
attachment_ids |
No | The sourceIdentifier values of earlier results. The tool searches only these documents of the given products. |
document_name |
No | The name of a document. The tool searches only this document. |
source_types |
No | The source types to search, for example product_attachment. The tool searches all source types when you omit it. |
limit |
No | The maximum number of passages, from 1 to 5. |
The lists abstract_skus, concrete_skus, and product_abstract_ids can have a maximum of 10 values together. For example, a search with 6 abstract SKUs and 6 concrete SKUs is rejected. attachment_ids and source_types can each have a maximum of 10 values. The tool never throws an error for invalid arguments: it returns isSuccessful = false and the reason in messages. When the knowledge store fails, the tool tells the model that the document search is not available, and the assistant answers from the product data.
Document scope
Before the search, the scope resolver plugins turn the products of the request into the document keys that the current customer may see. ProductAttachmentKnowledgeDocumentScopeResolverPlugin does the following:
- Resolves abstract and concrete SKUs to abstract products through the product storage.
- Skips products that are not available in the current store and locale. It adds a message that is safe to show to the customer.
- Reads the attachments of each product from the product attachment storage for the current locale.
- Returns only the attachments that are listed in
attachment_ids, when the model passes them.
Search settings
You can change the following settings in the Client AiCommerceConfig. A change applies to the next question. You do not need to index the documents again.
| METHOD | DEFAULT | DESCRIPTION |
|---|---|---|
getKnowledgeDocumentMaxResults() |
5 |
The maximum number of passages that one search returns. A larger requested limit is lowered to this value. |
getKnowledgeDocumentMaxCharacters() |
5000 |
The maximum total number of characters of all returned passages. The content of the passages with the lowest score is cut first. |
getKnowledgeDocumentCandidateMultiplier() |
2 |
The search asks the knowledge store for this number multiplied by the limit, so that the passages that the filter removes are replaced. 1 asks for the limit only. |
getKnowledgeDocumentMinContentLength() |
30 |
The minimum number of characters of a passage after whitespace is collapsed. Shorter passages, such as a chunk with only a heading, are removed. |
getKnowledgeDocumentMaxIdentifiers() |
10 |
The maximum number of products in total over abstract_skus, concrete_skus, and product_abstract_ids, and the maximum number of values in each other identifier list. A search above a limit is rejected. |
getKnowledgeDocumentMaxQueryLength() |
1000 |
The maximum number of characters of query. |
getKnowledgeDocumentMaxDocumentNameLength() |
255 |
The maximum number of characters of document_name. |
The search has no score threshold. The AI model decides from the content of the passages if they answer the question.
Indexing settings
You can change the following settings in the Zed AiCommerceConfig:
| METHOD | DEFAULT | DESCRIPTION |
|---|---|---|
getKnowledgeDocumentMaxFileSize() |
52428800 (50 MiB) |
The maximum size of a downloaded document in bytes. The smaller maximum size of the knowledge store also applies: 5 MB for Bedrock. |
getKnowledgeDocumentIndexMaxAttempts() |
3 |
The number of attempts to index a document after temporary failures. |
getKnowledgeDocumentDownloadTimeout() |
30 |
The timeout of a document download in seconds. |
getKnowledgeDocumentIndexQueueChunkSize() |
10 |
The number of messages that the queue worker of knowledge_document.index reads at one time. |
getKnowledgeDocumentReindexPageSize() |
100 |
The number of sources that a re-index reads in one page. |
getKnowledgeDocumentStatusRefreshPageSize() |
100 |
The number of processing documents that the status refresh reads in one page. |
getBedrockKnowledgeBaseMediaExtractionConfiguration() |
Image extraction enabled | The media extraction configuration that the setup adds to each data source that does not have it. An empty array keeps the data source unchanged. |
getBedrockKnowledgeBaseMaxInlineDocumentSize() |
5242880 (5 MB) |
The maximum size of a document that is sent inline to the knowledge base. |
getBedrockKnowledgeBaseInlineTextMimeTypes() |
text/plain, text/markdown, text/csv |
The MIME types that are sent as inline text. Other MIME types are sent as inline bytes. |
getBedrockKnowledgeBaseSetupStatusCheckAttempts() |
60 |
How many times the setup reads the status of the knowledge base or a data source before it fails. |
getBedrockKnowledgeBaseSetupStatusCheckInterval() |
5 |
The wait time in seconds between two status reads of the setup. |
Console commands
| COMMAND | OPTIONS | DESCRIPTION |
|---|---|---|
knowledge-document:setup |
--recreate-index |
Checks the knowledge base and each configured data source, waits until they are ready, and enables the image extraction. It creates nothing and stores nothing. With -v, the command shows each step. The Bedrock store does not support --recreate-index and only reports this. |
knowledge-document:reindex |
--source-type, --force |
Queues all documents for indexing, and queues the documents of deleted sources for deletion. --source-type limits the command to one source type. Repeat the option for several source types. --force clears the content hash, so that every document is sent to the knowledge base again. With -v, the command shows the number of queued documents for each source type. |
knowledge-document:status:refresh |
Reads the knowledge base status of all processing documents and saves indexed or failed. The output is Checked N document(s): X indexed, Y failed. |
Download security
The URL reader downloads a document only from a host that resolves to public IPv4 or IPv6 addresses. It connects only to the checked address and follows no redirect. This prevents a product attachment URL from reading internal services.
When the environment sets HTTP_PROXY or HTTPS_PROXY, the download goes through the proxy, and the proxy resolves the host itself. In this case, configure the proxy to block private, loopback, and link-local addresses.
Extension points
| INTERFACE | LAYER | REGISTERED IN | SHIPPED PLUGIN | GUIDE |
|---|---|---|---|---|
KnowledgeDocumentSourcePluginInterface |
Zed | AiCommerceDependencyProvider::getKnowledgeDocumentSourcePlugins() |
ProductAttachmentKnowledgeDocumentSourcePlugin |
Add a knowledge document source |
KnowledgeDocumentScopeResolverPluginInterface |
Client | AiCommerceDependencyProvider::getKnowledgeDocumentScopeResolverPlugins() |
ProductAttachmentKnowledgeDocumentScopeResolverPlugin |
Add a knowledge document source |
KnowledgeStorePluginInterface |
Zed | AiCommerceDependencyProvider::getKnowledgeStorePlugin() |
BedrockKnowledgeBaseKnowledgeStorePlugin |
Use another knowledge store provider |
KnowledgeSearchPluginInterface |
Client | AiCommerceDependencyProvider::getKnowledgeSearchPlugin() |
BedrockKnowledgeBaseKnowledgeSearchPlugin |
Use another knowledge store provider |
Thank you!
For submitting the form