Visual Logic: a Picture is worth 1,000 Words!

It all started with Spaghetti…  in the late 90’s, Business Rules technology started to become popular based on the premise it would remove the need for spaghetti code.  Those independent statements, which all contribute to checking that an applicant is eligible or that a transaction is possibly fraudulent, could be externalized for easier maintenance.  It quickly turned into the vision that business users would be empowered to maintain that decisioning logic of course.

SpaghettiOver a decade later, we are back to Spaghetti!  Certainly BRMS technology helped significantly reduce the complexity in the software code, leading to much greater agility but……..  we moved the problem from spaghetti code to spaghetti rules!

When I talked to BRMS practitioners, the maturity curve looks something like:

  1. Learning Curve: “But where do I start???”
  2. Bliss: “Wow, this is powerful!”
  3. One step too far: “Oooooh … I have too many rules!”

Business Rules Engines have evolved, as well as infrastructure, allowing for execution of very large decision services in very little time.  Runtime performance is not much of an issue.  The issue with Spaghetti is really the understandability of decisioning logic.

As a subject matter expert, the temptation is great to keep adding business rules into the pot.  After all, it is what business rules are for: they allow you to manage those rules independently and let the engine worry about execution.  The problem comes when the number of business rule grows beyond your ability to cope intellectually.

A practical example

You might start with a rule for credit score requirement on a mortgage product, e.g. “Credit score must be at least 650”.

Then comes segmentation based on the loan amount.  A jumbo loan only applies if the loan amount is greater than $417k but above this threshold, we could have increasing strict requirements for credit worthiness.  For example, 650 might be required for a Jumbo loan but you would want at least 700 for any Jumbo loan above $1M.  I am making those up of course.

Then, with still high rates of foreclosure, you may further discriminate the requirements based on neighborhoods that have been particularly affected, based on actual appraised value of the property, based on prediction of job stability…

Before you know it, your decision service for Jumbo loan origination is turning into thousands of rules.  The only way to wrap your head around the volume of rules is to stop thinking about them and have the computer think instead…  This is where automated verification and validation can take over and pinpoint anomalies such as gaps and overlaps.

The issue with this paradigm is that you really want to leverage the expertise of the skilled knowledge worker.  It is critical to provide him/her the opportunity to understand so they can contribute and improve the quality of the decisioning.

Addressing the complexity of exceptions

Carlos described in his own words the challenge of exceptions that pollute clean decision services.  I also covered the importance of User Experience in the past.  Those are critical aspects that help cope with complexity.  If you can identify the best graphical representation and you can structure your knowledge, break it down into chunks, then you can push the limit of what volume you can understand.

The issue is that a single representation may not be appropriate for all people at any point in time in the decision management life-cycle.

Let me illustrate.

The first point we beat to death in the past…  You start with a decision table then get exceptions after exceptions and your table ends up being too large for visualization and full of holes.  It can’t work any longer.  The same could be said for any representation actually, there is a point at which any will break.

An Epiphany for Carole-Ann!

The point we have not yet discussed much is life-cycle.  I am not referring here to the traditional Governance Process but rather the fact that business rules have a life-cycle of their own and the activities involving them are quite diverse.  It only occurred to me in the past year or so that my needs for looking at business rules keep changing based on my role as a user in addition to the nature of those business rules.

  1. Before I start, I need to understand the whole decision service and what piece of decisioning can be improved and how
  2. When I author the rule, I only care about that one rule and its test cases to make sure it does what I want it to do
  3. After it is unit-tested, the rule needs to be tested in aggregation in the whole decision service; this is where I want to understand interdependencies between the rules
  4. When it is ready for approval, I want to ensure it is compliant with my business objectives and corporate guidelines / industry regulations

As such, looking at a Decision Table makes a lot of sense when you have regular rules for which you need to change range limits.  It is also very good when you want the visual validation that all rules are consistent: the higher the age, the lower the risk for example.  But if I want to add an exception that refers to a brand-new attribute, I would likely prefer to add nodes in a Decision Tree.  If I am focusing on a specific use case, I do not care about the whole table or tree and I only want to see the rules that apply to that one use case.  If I am the Compliance guy, I may only care about negative outcomes and ensure that we did not apply logic that conflicts with regulations.

The fact is that the nature of the user interface is NOT ONLY applicable to the type of rules that you have…  but it is ALSO a function of the task at hand.

As a result, Tom and Mary may need to interact with different perspectives of the same piece of logic at the same time because they are involved in different tasks with different objectives.

Visual Logic

We have called this concept Visual Logic as it refers to the ability to visualize the logic for the best understanding, the best interaction at the time.  One person we talked to said, “It is like Business Intelligence on rules”.  I think this is very true.  There is a real need for better intelligence on the knowledge before, during and after its automation.

I’d love to hear from you whether you had your epiphany much sooner than me and how you have been dealing with those challenges.

10 responses to “Visual Logic: a Picture is worth 1,000 Words!”

  1. Carole-Ann,
    I agree with you that we do not want to end up with spaghetti code, spaghetti rules or spaghetti tables. Exceptions and life-cycle are crucial issues. There is no such thing as a one table fits all (set of) decision table(s). To me a decision table is a view on rules (and exceptions) and there are multiple views for multiple purposes. The purpose may be different across the life-cycle (acquisition, modeling, verification, implementation) and is different for different audiences (domain specialist, validator, tester). It will not be a surprise that this requires much better tooling than most simple editors or models today. How else could you keep track of a general rule and its exceptions, see the complete picture, and still be able to recognize or change any of the original building elements. It also requires a good definition of good decision tables and tabular decision models. I like the epiphany (maybe I had it a little earlier, because of age…), but I would like to add to your title: A picture is just a snapshot, a table gives you 1000 pictures.

    • Thanks Jan for your very appropriate comment. You phrased it very well! Let me ask a follow up question… Do you envision that Decision Tables can address all of those use cases? My experience has been very positive with regular decisioning logic, but less intuitive with large rulesets that have lots of exceptions. I am looking forward to your perspective on that.

  2. Another problem of having too many rules – based on my experience with fico and ibm – is that changimg one litttle rule might drive the whole system crazy, and resulting in manual verification of each case/application, which contradicts the whole reason of having BRMS.

    • You are ahead of my posts ;-)
      I was actually going to discuss this very topic on my blog. This is a point I think traditional BRMS tend to overlook as you pointed out. Visibility into impact analysis is not something you want to postpone until QA testing or later though.
      I do believe in the Agility (with a capital A) that Decision Management brings to the table. The addition of critical aspects like those (that I have called here Visual Logic and a couple more actually) will fulfill on its promise.

      • Well, visual logic seems like a great idea. I actually started working on a tool that would do just thatt. However, since I was working on it on my spare time, it never got past the research phase. It’d be interesting to get online together to discuss it inmore detail.

  3. Nice thought-provoking post. Externalizing rules enables transparency and independent evolution, but definitely adds complexity if you can’t readily manage dependencies/constraints in context. The trade of with decomposition is comprehension.

    I think you are right on track with visual-logic that allows a person to navigate a rule from one or more perspectives.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: