Topic 6 – Confusion between Measures of Effectiveness and Measure of Performance

I was in a discussion with a sponsor recently in a project meeting about the meaning and usage of MOE (Measure of Effectiveness) and MOP (Measure of Performance). I just wanted to share some thoughts about this. The difficulty is that with these terms the definitions can vary widely from source to source such that what one will call an MOE another might call an MOP. And at a certain level of decomposition the MOE and MOP might look pretty much the same.  In my opinion, the keys to thinking about these definitions are the words themselves.

I will use the term “thing” below because system, system of system, complex system, etc. merely obfuscate the concepts by overlaying other disagreements about what certain “system” terms mean.  And “system” is not pertinent to the MOE/MOP discussion, though we usually apply them to systems.

Measure of Effectiveness – this implies that whatever is being measured is telling one whether or not the “thing” met some success criteria related to the problem it was to address.  Was this a “good” solution to the problem? Was this a better solution to the problem than another solution?  The point being that one remains focused in the problem space, solving the problem, measuring the problem.   This is why the literature for MOEs often refers to them as being mission-oriented.  The Problem Space is where one discusses what the “thing” will do/accomplish, not how it does it.

MOEs are not necessarily composed of a mathematical relationship to MOPs.  In fact, frequently they are not.  If you read the Sproles articles referenced below, he has an anecdote about the reduction of pollution in an Australian river.  The MOPs were ppms of different toxins in the river; the MOE was that the mayor of the town was willing to swim in the river. It might be an apocryphal story, but it illustrates a situation where the MOE was not just an accumulation of the MOPs.

In fact, this is one good reason to do so much modeling and simulation.  It is because there are no good ways of combining system MOPs to get MOEs.  If there were, all of the problems we solve would have closed form solutions.  Measures can be tiered when dealing with complex capabilities. For example, a weapon system could have a high level measure of “survivability” that decomposes into more granular measures of “threat exposure” and “threat effectiveness”. These can be good metrics, but given all of those what was the overall goal for “mission success”?  How did you know you were done?  Or the solution was good enough?  In the context of the whole simulation, what was the measure that showed one result was better than another.

Measures of Performance – these on the other hand are found in the Solution Space.  They are typically intrinsic attributes of the thing that describes how well it performs a function or set of functions.  Performing a function well does not necessarily mean that the thing was more effective at solving the problem the thing was created to address.

In the Problem Space we often discuss the purpose (the mission) of the thing.  The Measure of Effectiveness then evaluates the things ability to meet the needs of the mission in the context of the problem and the mission, and not the performance of the thing.

In the Solution Space, an instance of the “thing” (multiple different instances of the “thing” can be conceived and created) is created to address the problem and or mission.  Measures of Performance are what describes attributes of the thing but not necessarily how well it addresses the problem/mission.

The Problem Space is frequently decomposed into discrete elements that are generally called something like mission functions.  These are broad sets of activities that elaborate what must be done to accomplish the mission.  MOEs measure these activities.  In the Solution space there also functions (these are the solution, or system, functions).  These are not the same functions as described in the Problem Space.  MOPs measure these.  Example: SpaceX launchs a supply rocket to ISS.  One mission function, after launch and clearing the tower, could be something like “Ascend to Orbit”.  A complimentary system function could be something like “Produce Thrust”.  A MOE might be probability of achieving correct orbit.  An MOP might be the specific impulse of the rocket engine.

Sproles, N., Coming to Grips with Measures of Effectiveness, Systems Engineering Journal, Vol. 3, No. 1, 2000, pp. 50-58

Sproles, N., Formulating Measures of Effectiveness, Systems Engineering Journal, Vol. 5, No. 4, 2002, pp. 253-263

Sproles, N., The Difficult Problem of Establishing Measures of Effectiveness for Command and Control; A Systems Engineering Perspective, Systems Engineering Journal, Vol. 4, No. 2, 2001, pp. 145-155