To Read or Not to Read? Hopefully the Former!

It has been far too long since I posted to this blog.  It’s time to let you know what I’ve been working on over the past few months.  I’ve been writing a book on a topic that is near and dear to my heart.

The book is titled Multi-objective Decision Analysis: Managing Trade-offs and Uncertainty.  It is an applied, concise book that explains how to conduct multi-objective decision analyses using spreadsheets.

The book is scheduled to be published by Business Expert Press in 2013.  For a little more information about my forthcoming book, please read the abstract shown below:

“Whether managing strategy, operations, or products, making the best decision in a complex uncertain business environment is challenging.  One of the major difficulties facing decision makers is that they often have multiple, competing objectives, which means trade-offs will need to be made.  To further complicate matters, uncertainty in the business environment makes it hard to explicitly understand how different objectives will impact potential outcomes.  Fortunately, these problems can be solved with a structured framework for multi-objective decision analysis that measures trade-offs among objectives and incorporates uncertainties and risk preferences.

This book is designed to help decision makers by providing such an analysis framework implemented as a simple spreadsheet tool.  This framework helps structure the decision making process by identifying what information is needed in order to make the decision, defining how that information should be combined to make the decision and, finally, providing quantifiable evidence to clearly communicate and justify the final decision.

The process itself involves minimal overhead and is perfect for busy professionals who need a simple, structured process for making, tracking, and communicating decisions.  With this process, decision making is made more efficient by focusing only on information and factors that are well-defined, measureable, and relevant to the decision at hand.  The clear characterization of the decision required by the framework ensures that a decision can be traced and is consistent with the intended objectives and organizational values.  Using this structured decision-making framework, anyone can effectively and consistently make better decisions to gain a competitive and strategic advantage.”

Look for my forthcoming book, Multi-objective Decision Analysis, on the bookshelves in 2013!

Source: favim.com

Know Thyself: The Value of Specifying Your Values

A few days ago I was chatting with a friend about a new company initiative she’s going to be working on.  The new initiative is an important strategic maneuver for the company.  If it is designed and implemented well, the company will be well-positioned to compete in the new market.  However, if it isn’t designed or implemented well, it will significantly drain the company’s limited management and financial resources and likely weaken the company’s competitive edge.

As our conversation continued, one important problem with the new initiative became apparent.  Based on available information, it appears the overall strategy and objectives for the new initiative aren’t well thought out.  This is an important problem for several reasons – it makes it difficult for workers to know what tasks need to be done to advance the initiative, makes it difficult for managers to track and guide progress on all of the project’s phases, and impedes clear communication.

Having identified the problem with the current state of the initiative, namely, that management has not clearly defined what they want to achieve with the initiative, we began discussing potential solutions.  The first thing we wanted to do was brainstorm the factors we thought were important to the company with respect to the initiative; however, it became clear right away that first we needed to define some terminology.

The problem was that we were using different words to describe the ideas we wanted to discuss.  One minute we were talking about the company’s values, another minute we were talking about the company’s objectives, and the next minute we were talking about the company’s goals.  We needed to agree on the definitions of the terms we were using.  After spending some time discussing possible definitions, we agreed on the following definitions:

• Values are the areas of concern, considerations, or matters you think are significant enough to be taken into account when evaluating alternatives.  For example, values for the company considering alternative ways of rolling out the initiative may be ease of implementation, company image, and profit.
• Objectives augment values by specifying the preferred direction of movement.  Thus, the company considering alternative ways of rolling out the initiative would find an alternative that is easier to implement more desirable.
• Goals are thresholds of achievement with respect to values and objectives that are either achieved or not by an alternative that’s being evaluated.  For example, the company might have a goal of implementing the initiative within eight weeks.  For a given alternative, this goal may or may not be achievable.
• Finally, underlying each of these terms is the idea of a measure, a measuring scale for the degree of attainment of an objective.  For example, the company may use “annual profit in dollars” as the measure for the objective of increasing profit.

By thinking carefully about these terms, and agreeing on their definitions, we developed a common vocabulary we could use to communicate clearly with one another.  Having defined the terminology, we brainstormed the company’s values and objectives with respect to the initiative and even defined the measures we would use to gauge the degree of attainment of the objectives.

From start to finish, the whole conversation didn’t take more than an hour, but by the end we had a shared understanding of the values and objectives the company should try to achieve with the initiative, a shared understanding of how we would measure degrees of attainment of the objectives, and significant insight into how we would evaluate the alternative ways of rolling out the initiative.  Not bad for an hour-long conversation.

This structuring process and terminology is applicable to any situation in which you want to evaluate alternatives based on multiple values and objectives.  By specifying the values you want to use to evaluate your alternatives and defining the objectives and measures you’ll use to judge the degree of attainment of your objectives, you’ll reap the significant benefits of thoroughly understanding your decision situation, being able to articulate a clear rationale for your decision, and being able to identify the alternative that, according to your values and objectives, is the most valuable to you.

You Might Be a Decision Analyst If…

Over the holidays, I visited my sister and brother-in-law.  While we were chatting, he mentioned he wanted to refurbish some audio equipment and was trying to decide between fixing the electrical components himself and hiring an expert to do it for him.  He said he was concerned with the potential cost and quality of the repairs, as well as how long it would take to complete the repairs, but he was uncertain about the range of values these three factors could take under the two alternatives.  I must have been smiling from ear to ear.  Without even realizing it, he’d described a simple, yet interesting multi-objective decision under uncertainty, and I wanted to show him how I could help him with his decision situation.

I pulled out a sheet of paper and quickly drew a rough influence diagram to represent his decision, the three uncertainties, and the resulting outcome “node” of his confidence or pride in the refurbished system.  Then I asked him about the range of costs the components could take and the range of time and quality the repairs could take.  After drafting probability distributions for the three factors, we chatted about his value functions and weights for the three factors and assigned scores and probabilities to the two alternatives across all three factors.  I transferred all of this information, which only took a few minutes to collect, into a spreadsheet and quickly churned out preliminary certainty equivalents for the two alternatives.   After performing a quick sensitivity analysis on some of the model’s inputs, including the weights and multi-attribute risk tolerance, we determined we had robust certainty equivalents for the alternatives.

My brother-in-law was pleased with the whole exercise.  It helped him frame and understand the decision situation.  By breaking the problem down into objectives, uncertainties, scores, and values, he could clearly see how changes in those factors would affect which alternative was preferred.  Best of all, in this case, the analysis was quick and free…unless you count the cost of the sandwiches we ate while we worked on the problem.  I enjoyed conducting this decision analysis with my brother-in-law and was very pleased that he thought it had been a valuable use of his lunchtime.

Do you have a similar story?  Have you ever helped a friend or family member with an interesting decision problem?  Are you working on an interesting problem at work?  When you get a chance, please take a few minutes to write about and share your experience.  You can think of it as your chance to tell the next “You might be a decision analyst if…” story.

I Think, Therefore My Brain Hurts

The last time you chose a health plan for yourself and your family, how much effort did you expend learning about and comparing the options before choosing a plan?  The last time you implemented a new strategy for your organization, how much effort did you expend coming up with and considering the alternatives before implementing the strategy?  The last time you updated your investment portfolio, how much effort did you expend researching and evaluating your opportunities before updating your portfolio?

Thinking Is Hard, So I’m Going to Stop Now

I’d be willing to bet that a common, honest response to these and similarly important questions is, “Not as much effort as I could or should have spent”.  There are lots of reasons why we might not spend enough time and energy thinking before making major decisions – we might not know what we want to achieve, we might not be able to collect or interpret much of the relevant information, we might not know how to deal with uncertainties, or we might be pressed for time.  In these situations, the costs of additional thought may loom larger in our minds than the potential benefits, so we halt our thinking process and simply make a decision.

Image Source: livornocorp.com.pk

As an example, in the context of group decision making, think about the last time you were in a group meeting, the group had one important item left to discuss and vote on (e.g. whether to fund a $200,000 project), and the meeting was already running over the time allocated for the meeting.  How much time and energy did the group spend discussing the important item before voting?  Probably not as much as they could or should have spent.

Thinking Hard about the Benefits of Thinking Hard

One reason why we might not value the potential benefits of additional thinking as highly in our “think more/stop thinking” calculus, and so cut short our thinking process in order to make a decision, is that we might not know or have an appreciation for the many benefits of thinking.  However, the benefits of thinking (that is, spending the time and energy necessary to think hard about issues) are abundant and consequential.  Even just four benefits are enough to give you a feel for the significance of further thinking:

1. Thinking helps you clarify your goals and preferences

2. Thinking helps you reframe your problem and identify alternatives

3. Thinking helps you discover what additional information you need and, possibly, where to get it

4. Thinking helps you have control over your actions

A noteworthy fifth benefit of in-depth thinking is that it helps you develop a habit of thinking methodically and thoroughly.  The more time you spend thinking hard about issues, even simple ones, the more likely you are to become comfortable with effective techniques for thinking about and dealing with complicated issues.  In this way, the costs of thinking hard in the short-term are dwarfed and overshadowed by the significant long-term benefits.

Image Source: 203.175.165.17

Think of Thinking Hard as a Healthy Habit

That (at times) we don’t expend enough effort thinking before making major decisions is understandable – thinking is time consuming, it’s costly, it’s hard work.  One reason why we might not think enough, even when we know additional thinking could be helpful, is that we don’t fully appreciate the long-term benefits of putting in the effort to think hard in the short-term.

By keeping in mind the wide-ranging benefits of thinking deeply, you’ll be able to properly evaluate whether, in a specific case, additional thinking would be worthwhile.  You’ll be able to learn and practice using techniques for analyzing and solving problems.  And, ultimately, you’ll be able to use the tools and strategies you’ve learned to make better decisions.

How Many Ways Can I Arrange My Words And Still Get My Point Across? May I Repeat Myself?

A couple of years ago, my friends and I went to an indoor shooting range for a friend’s bachelor party.  One of the firearms we chose was a .44 Magnum, a revolver made famous by Clint Eastwood’s “Dirty Harry” films.  Holding the revolver in my hand, I thought, “Why is this weapon called a revolver?  Doesn’t the cylinder rotate around its axis?  Perhaps it would be more accurate to call this weapon a rotator.”  I’ll admit, that’s probably not the first thing most people think about when they hold a revolver in their hand (besides, maybe the weapon is called a revolver because the bullets revolve about the cylinder’s axis).

Image Source: redensign.wordpress.com

In any case, the experience reminded me of another product that has a confusing (and arguably inaccurate) name, the combination lock.  A combination lock is a type of lock in which you have to enter a specific sequence of numbers to open the lock.  The name is confusing because, in mathematics, the correct term for a set of numbers that must be in a specific sequence is a permutation.  While it probably wouldn’t be very productive to try to change the naming convention for the combination lock, it is important to understand the difference between combinations and permutations.

Image Source: en.wikipedia.org/wiki/Combination_lock

Deos Oedrr Mettar?  Does Order Matter?

To know whether you’re dealing with a combination or a permutation, ask yourself one question – does order matter?  If it does, you’re dealing with a permutation.  If it doesn’t, you’re dealing with a combination.  After determining whether order matters, you need to find out whether repetition is allowed in the selection process.  For both combinations and permutations, the formulas are different depending on whether repetition is allowed.

In an earlier article, I mentioned I’m on a local policy advisory board and soon we’re going to elect three members of our 15-person board into leadership roles – Chair, Vice-chair, and 2^nd Vice-chair.  These are distinct roles, which are in decreasing order of responsibility.  It’s fairly obvious that one outcome (e.g. Tess being elected Chair, Filbert being elected Vice-chair, and Laura being elected 2^nd Vice-chair) is very different from another outcome (e.g. Laura being elected Chair, Tess being elected Vice-chair, and Filbert being elected 2^nd Vice-chair).

Image Source: gotoworkonenw.com

Permutation – When Order Matters

Since order matters in this example, we know we’re dealing with a permutation, and since each person can only hold one position, we also know repetition is not allowed.  Given this information, the number of ways we could select three members of our 15-person board for the ordered leadership positions is 2,730.¹

Combination – When Order Doesn’t Matter

A similar example in which order does not matter is when we’re selecting a subset of members to be on one of the board’s sub-committees.  In this case, a sub-committee composed of Tess, Filbert, and Laura is identical to a sub-committee composed of Laura, Tess, and Filbert.  That is, the order of the members doesn’t matter.  And since we can’t select a person twice for the same sub-committee, once again we know repetition is not allowed.  Given this information, the number of ways we could select three members of our 15-person board for the unordered sub-committee positions is 455.²

Does Order Matter?

Yes No Is Repetition Allowed? Permutation Combination Yes nr (n+r-1)! / r!(n-1)! No n! / (n-r)! n! / r!(n-r)!

Reinforcing with Repetition

As you can see, it is important to determine whether order matters.  In the first example, where it does, there are 2,730 permutations.  In the second example, where it doesn’t, there are only 455 combinations.  While we sometimes use the term combination loosely to describe sets of objects regardless of whether order matters, it’s important to remember that there is a difference between combinations and permutations and you can identify which one you’re dealing with by asking yourself whether the order of objects matters.  By understanding the difference between the two concepts and being able to use the correct formulas, you’ll be able to calculate the right quantities and make informed decisions based on accurate information.

¹ Permutation without repetition: n! / (n – r)! = 15! / (15 – 12)! = 15 * 14 * 13 = 2,730

² Combinations without repetition: n! / r!(n – r)! = 15! / 3!(15 – 12)! = (15 * 14 * 13) / (3 * 2 * 1) = 455

Can Someone Hand Me The Directions? A Soliloquy About Choosing Among Divergent Roads

When was the last time you had to consider your professional alternatives?  A few months ago?  A year ago?  A few years ago?  For me, it was yesterday.

I’ve known for some time that I would need to sit down and really think about my alternatives and what I want to do.  It’s been on my mind ever since I finished school, but I’ve kept myself busy since then and sitting down to analyze my situation just never seemed to make it onto my to-do list…until yesterday.

A Personal Decision Analysis about Professional Opportunities

As you’ve probably already gleaned from some of my earlier posts, I enjoy thinking about, discussing, and using decision analysis techniques, so when I sat down to model my situation I already had an idea of the information I would need – I would need to know my alternatives, the criteria I would use to evaluate the alternatives, how I would weight the evaluation criteria, and how I would score each alternative on each of the criteria.  With this information, I would be able to make a reasonable decision that’s consistent with my values and preferences.  Let me briefly describe how I formulated and analyzed my decision situation.

Image Source: pulpitpoetry.blogspot.com

What Are My Alternatives?

First, I thought about my alternatives.  I’ve been thinking about them and discussing them with family and friends for a while, so they’re fairly well developed.  In my mind, I have three alternatives.  I can: 1) Work with my wife’s father, 2) Work for myself, or 3) Continue doing what I’m doing and apply for jobs until I’m hired.  I’m sure there are additional alternatives, like a combination of these alternatives or ones I haven’t even thought of, but to keep this decision analysis simple let’s assume these are my alternatives.

What Are My Criteria?

Next, I thought about the criteria I wanted to use to evaluate my alternatives.  After some thinking, I decided on five criteria: 1) Doing what I love, 2) Staying involved with policy advisory committees, 3) Money, 4) Family dynamics, and 5) Maintaining and developing new skills.  Once again, I’m sure there are additional criteria, but these are the ones that are important to me and that can help me distinguish among my alternatives.

In order to use the decision analysis technique, I had to weight the criteria based on their relative importance to me.  Two important requirements in this step are: 1) the five weights must sum to one, like probabilities, and 2) the difference between any two weights should represent my preference for one relative to the other.  This means that if I care about the criterion ‘Doing what I love’ twice as much as the criterion ‘Money’, then the weight for the former should be twice the weight for the latter.  Weights of 0.30 for ‘Doing what I love’ and 0.15 for ‘Money’ would meet this requirement because 0.30 is twice 0.15.  Let’s say the weights I settled on, at least initially, were 0.25, 0.10, 0.25, 0.25, and 0.15, respectively.  These weights would meet both requirements because they sum to one and the differences among them represent my relative preferences for the criteria.

How Do I Score My Alternatives?

With my alternatives, evaluation criteria, and criteria weights written down (i.e. recorded in a spreadsheet), the next step was to score each of my alternatives on each of the criteria.  Since there were only three alternatives, I decided to assess the three alternatives one criterion (i.e. column) at a time rather than assessing the first alternative on all five criteria (i.e. row) before moving on to the second and third alternatives.  By using this method to record the scores, I was able to make comparisons among the scores for the three alternatives to ensure that, for each criterion, the differences among the scores reflected how I would rate one alternative compared to the other two.

Initially, the scores cannot be combined because the units for each criterion are different (e.g. dollars for ‘Money’ and “happiness” for ‘Doing what I love’), so they need to be rescaled.  One common way to score alternatives in commensurable units is to use a subjective value between 0 and 100.  0 represents the worst plausible outcome for the given criterion, and 100 represents the best.  The score I give each alternative represents my belief about how well the alternative will satisfy me on that criterion.  For example, if I believe I’m likely (loosely defined) to earn the same amount of money whether I work with my wife’s father or work for myself (within a given time horizon), then I would give each alternative an equivalent score on that criterion, let’s say 55 and 55.  And if I think I would make a slightly greater amount of money once I get a job (if I continue looking for one), then I would give that alternative a slightly higher score, let’s say 65.  Unlike the weights on the evaluation criteria, these scores do not need to sum to a specific value.

Image Source: home.ubalt.edu

How Do Use All Of This Information?

With the weights and scores in place, I was ready to combine all of the information (including my values, preferences, assumptions, and subjective judgments) in a formal, structured way to see which alternative is “best”.  For this decision analysis technique, the decision rule is that I should choose the alternative with the highest overall score.  To determine which alternative had the highest overall score, I took a weighted average of the scores and criteria weights for each alternative.

Don’t Forget About Sensitivity Analyses…

Of course, just like for any other analysis, you would want to perform sensitivity analyses (i.e. make incremental changes to the scores and weights) to see how sensitive the result is to changes to the scores and weights.  If you change the scores or weights significantly and the result does not change (i.e. the initially superior alternative remains superior), then you can be more confident in the result.  If, on the other hand, the result is sensitive to changes to the inputs, then you need to try multiple relevant combinations and think hard about your values and preferences.

As you can see, conducting a formal decision analysis to help you make an important decision can be highly personal.  It requires you to think hard about your values and preferences (i.e. what you care about and how badly you care about it).  At the same time, the technique is flexible, so you can (as I did) ask family and friends to supply scores and weights (from their perspectives and “yours”) to see which alternative is favored.  In the end, the result of the model can just be another piece of information (albeit a valuable, defensible piece of information) to consider before making your decision.  The result of a decision analysis isn’t meant to strip you of your decision making authority.  It’s simply meant to structure your thinking, clarify your preferences, and aid and inform your decision making process.

Tell Me, What Is the Answer?

So, what was the result of my decision analysis?  Which career path have I chosen?  Like an annoying season finale on television, the answer to those questions is…to be continued.  Let’s just say the results were surprising and informative, and I highly recommend you try this or a similar technique the next time you face a difficult or complex decision situation.

There’s Only One Way to Get Into This Club, and Wishing Won’t Make It So

When you’re walking in your local mall and someone asks you to respond to a quick survey question, do you provide an answer?  When you’re surfing the internet and a one-question survey “pops up” on the side of your screen, do you provide an answer?  When you’re watching your favorite network news program and the anchor asks you to text or tweet a response to the question on the screen, do you provide an answer?

If you do, you’re participating in what is known as convenience sampling, a data collection procedure in which survey respondents are included in the sample at the convenience of the researcher (and the respondent).  This sampling methodology is in sharp contrast to (simple) random sampling, a data collection procedure in which the researcher insures each member of the population of interest has an equal probability of being included in the sample.

Image Source: snapcomms.com

Let Me Choose My Own Fate, I Promise I Won’t Bias You (Wink Wink)

One of the problems with convenience sampling is that it allows respondents to self-select, meaning they get to choose whether to participate.  Self-selection is a problem because it means the people who choose to participate may be systematically different from those who choose not to participate.  And if the respondents are different from the non-respondents, then the information you glean from the respondents isn’t representative of the entire population of interest (i.e. it’s biased).

Think about the political opinion polls you see on network news programs.  Who do you think responds to those questions?  It’s people who are highly opinionated and passionate about the topic.  Anyone who is relatively indifferent or apathetic doesn’t take the time to respond to the poll.  Since the people who respond are systematically different from the people who don’t respond, the opinions collected by the poll aren’t representative of the U.S. population at-large.

Image Source: shardule.blogspot.com

If you’re only interested in the poll results because you find it entertaining to see which highly opinionated, passionate group spends the time to text or tweet their responses, there’s no real problem; however, if you’re interested in collecting public sentiments that are representative of the U.S. population at-large, an on-air political opinion poll is just about as worthless as it gets.

Our Program Helped the People Who Participated, So the Program Must Be Effective…Right?

Self-selection isn’t only a problem for survey research.  It’s also a problem for program evaluation, which means both government agencies and companies need to understand self-selection and how to avoid or correct for it.  Let’s say a government agency or company has developed a weight loss program and wants to be able to claim (legitimately) that the program can help anyone lose weight (i.e. that the program is effective).

One (unscientific) way for the government agency or company to evaluate the program is to advertise for participants, enroll everyone who responds to the advertisement, weigh them before they go through the program, weigh them again after they complete the program, and finally analyze the difference between the pre-program and post-program weights.

There are many problems with the design of this program evaluation (e.g. where’s the control group?), but one important problem is that participants self-select into the program.  People who choose to participate in a weight loss program are (very likely) different from people who don’t voluntarily choose to participate.  If the two groups are systematically different, then the “results of the program” (e.g. the average amount of weight lost by the participants) may not be due to the program at all.

Image Source: aboutweightloss.com.au

Without using a (scientific) random sampling methodology to reduce the probability of self-selection or using advanced statistical estimation techniques to correct for self-selection, it wouldn’t be appropriate for the government agency or company to claim that the program is effective.

Self-Select into Good Decision Making By Checking For Self-Selection

We’re often interested in acquiring information from a sample to learn about a specific population of interest.  There are many ways to select people for a sample, including convenience sampling and simple random sampling.  One problem with convenience sampling and other methodologies that allow individuals to choose whether to be included in the sample is self-selection.

Self-selection is a problem because those who choose to participate may be systematically different from those who don’t participate.  If that’s the case, the results of the survey, poll, or program evaluation won’t be representative of the specific population of interest.

Remember, always make sure you understand the sampling methodology, and whether self-selection might be an issue, before you form an opinion or make a decision based on information from a survey, poll, or program study; otherwise you could end up forming an inaccurate judgment or making an unwise decision based on flawed (i.e. unrepresentative) information.