IBM Watson™ Ideas

Welcome to the IBM Watson™ Ideas Portal

We welcome and appreciate your feedback on IBM Watson™ Products to help make them even better than they are today!

If you are looking for troubleshooting help or wondering how to use our products and services, please check the IBM Watson™ documentation. Please do not use the Ideas Portal for reporting bugs - we ask that you report bugs or issues with the product by contacting IBM support.

Before you submit an idea, please perform a search first as a similar idea may have already been reported in the portal.

If a related idea is not yet listed, please create a new idea and include with it a description which includes expected behavior as well as why having this feature would improve the service and how it would address your use case.

Need testing support and framework for running validation and blind set experiments and provide relevant metrics

Today, various teams both internal and external have developed their own harnesses, tools to measure accuracy of their models as part of their testing efforts.  This is not only redundant but probably not the best use of their development cycles.  

Example : The TWC - Ads (Creative Labs) team spends about 2-3 weeks as part of their development cycle to measure, evaluate and deploy new version of Conversation workspaces to their customers. 


a) Leverage the experiment, testing, model management framework from Watson Studio (part of Modeler) for all Watson services starting with Watson Assistant and Watson Discovery.

b) Create a set of Notebooks that include pre-built code to run cross-fold validation, blind set and accuracy analysis.

What do we have now:

a) Research Offering called FARCAST (From Yorktown) that's shelved.

b) With Watson Team Developed Notebooks in GH for each of the services :

  • Laksh Krishnamurthy
  • Apr 2 2018
Why is it useful?

Today, every IBM product team is developing their own framework to train, test and evaluate ML models trained using Watson.  This time is well spent understanding the client business and training needs than writing test harnesses. 

Who would benefit from this IDEA? As a Data Scientist, I want to run tests against Watson services to determine accuracy metrics for my deployment needs
How should it work?
Idea Priority High
Priority Justification
Customer Name
Submitting Organization
Submitter Tags With Watson
  • Attach files
    July 27, 2018 07:20

    I agree with this idea.

    I think that NLC is good starting point because Watson Studio started to manage NLC model. 

  • Stephen Choquette commented
    July 27, 2018 10:23

    Visual Recognition is there today.  NLC is on the roadmap for 2018.