Posted by: Carlos Serrano-Morales | October 25, 2011

RulesFest 2011 – Davide Sottara: Managing Imperfect Information with Imperfect Rules

Davide is a research scientist at the University of Bologna (which has the distinction of being the oldest continuously operating university in the world and a wonderful place).

Davide Sottara presenting at RulesFest 2011

Davide Sottara presenting at RulesFest 2011

Davide started his presentation with well known story by J. C. Bezdek to illustrate the challenges of reasoning with fuzzy – or imperfect – information (a well known story that unfortunately nobody in the audience seemed to know ;)). The key challenges are well known – we are faced with:

  • lack of information
  • ill defined information
  • erroneous information

Davide explains the choice of “imperfection” rather than “uncertainty” as the term to use to denote these cases. Imperfection is in essence the opposite of perfect precision (of course), but dealing with it brings us immediately out of the scope of what traditional rules engines do. Furthermore, removing imperfection from the picture by ignoring it, simplifying it, etc, essentially leads to ignoring essential information. The characteristics of the imperfection are relevant data for writing rules.

Davide’s work focuses on creating more robust systems that may be applicable to imperfect information and still leverage rule based systems. Using a modus ponens model, he illustrates the impact of introducing imperfection to the way the rules engines work. In essence, we need to think about the connectors (combination of premises, logical implication, etc…) differently, solve conflicts created by the imperfection and handle missing value. Davide would like to preserve the structure of rules, and hide those complexities from the rules expression itself.

Davide constrasted the frequentist approach, to the Bayesian approach, to finally the Fuzzy logic approach, to deal with imperfection. Each one of these approaches has its applicability domain, as well as its philosophical foundations.  Davide then went into a few details on the approaches – I will refer you to the slides for the information (lack of scientific notation capabilities in this blog writer is part of it…).

The key lesson Davide communicates is that you should not forget imperfection when creating and managing rules.

Charles Young asked what I think is a very good question: why is it that while in the early days rules solutions included Bayesian capabilities or equivalent analytics capability, they no longer include them?

 

Advertisements

Responses

  1. I could be wrong, but historically it was the job of the knowledge engineer to identify cases where knowledge is imperfect and describe those use cases concisely. Once those gaps have been identified, the developers can write the rules appropriately. The example I’ve seen first hand is from pre/post trade compliance. Often a portfolio will have funds and those funds usually don’t provide real-time data feeds. When compliance validation is executed, the engine must “estimate” the actual value. A simple rule like “the portfolio cannot exceed 10% in any sector” becomes challenging due to “imperfect” data. If my portfolio has 4 positions that are funds, the compliance engine can look at the previous report for the fund and use that as a basis for estimating the “real” weight of each sector. In reality, the weight of each sector in the fund isn’t static, so the estimate really is a “best guess”.
    Many rule developers I’ve met forget these types of “imperfect” knowledge issues. In fact, I would say most don’t think of it and do not know how to go about discovering those use cases. There’s no magic either, it’s just blood, sweat and pain. Developers have to take the time to acquire those skills and apply them diligently.

    • Peter, you are right. This is one area of imperfection. When data is unreliable or, per your example, not precise, you have to work with experts to understand how to extrapolate or predict the attribute you want to discriminate against.

      There are also other aspects to consider like uncertainty in knowledge per say. In risk management, you often do not know for sure that a customer is actually a fraudster, whether he or she is able and willing to repay, whether he or she is likely to develop an “expensive” disease, etc. But you should also consider the scenario where you do not know for sure the logic to apply. For illustration, consider a call center where agents are recommending products. There may be a 50/50% chance that the customer will accept for a free smartphone with a 2-year subscription; but when the iPhone was released, I am guessing that this probability went down the tube.

      In terms of solution, there are lots of different ways to reduce the unertainty, some more theoretical and some empirical. Blood, sweat and pain is the way to go for some of those problems. For the most extreme problems, it may not be enough as the *knowledge* does not exist. If data is available, some analytics magic can help. In the call center example, it is *best guess* at best as neither data nor expertise is available; adaptive solutions (based on Bayesian networks for example) can help though. Carlos and I actually developed a patent on rule-driven learning strategy for adaptive decision systems several years ago.

      Managing uncertainty is pretty exciting for us, decision management afficionados ;-)


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Categories

%d bloggers like this: