IBM Watson™ Ideas

Welcome to the IBM Watson™ Ideas Portal

We welcome and appreciate your feedback on IBM Watson™ Products to help make them even better than they are today!

If you are looking for troubleshooting help or wondering how to use our products and services, please check the IBM Watson™ documentation. Please do not use the Ideas Portal for reporting bugs - we ask that you report bugs or issues with the product by contacting IBM support.

Before you submit an idea, please perform a search first as a similar idea may have already been reported in the portal.

If a related idea is not yet listed, please create a new idea and include with it a description which includes expected behavior as well as why having this feature would improve the service and how it would address your use case.

Make ingestion independent from document order

Currently, the data type of the indexed fields is determined when the first document is uploaded (or when the indexed field appears for the first time). If the data type is not unique in the first document, the full document is not indexed and thus not ingested.

As an example, consider a document with metadata stored in a key-value dictionary like

"dict" : [ 
{ "key" : "author", "value" : "John" },
{ "key" : "year", "value" : 2018 },
{ "key" : "confidential", "value" : true}

The elements of the dict array all contain a field "key" with value of type string and a field "value" with variable type string, numeric and boolean.

  • Ingesting a document with the above metadata as first document will fail, as the type of the "value" field is not unique.

  • But ingesting a document with just the "author" field as first document, and than one with the above three fields will work.
    The field metadata.dict.value will be set as STRING, so in the UI it will not be possible to enter a query like "metadata.dict.value > 2018", although via API it will work.

  • However, ingesting a document with just the "year" field as first document, and than one with all the three fields will fail.

We ask to make the ingestion independent from the order the documents are ingested, either by allowing multiple types on a field, or by giving the possibility to specify the schemas of the expected documents.

  • Sandro Corsi
  • Jun 20 2018
  • Needs review
Why is it useful?
Who would benefit from this IDEA? As a customer, I want WDS to behave in a way that does not depend on the order the documents are ingested.
How should it work?
Idea Priority
Priority Justification
Customer Name
Submitting Organization
Submitter Tags
  • Attach files