Score

The initial assessment of an organization on a given World-Class Management Chart is conducted in the Score Phase by a DRIVER or Master with input from an organizational sponsor and key organizational members.  A key point is that only one person is responsible for the initial scoring of the attributes.  The certified DRIVER or Master may be an employee of the organization or a third party hire.  Within the process, participants from the organization review the scores and either accept an assigned score or discuss the rationale for changing that score.  The final output is the SCORE Report.

This section first covers the fundamentals of what a World-Class Management Step Chart is and how it is scored to develop Attribute Scores, Step Scores and Chart Scores.  After the fundamentals of step charts are covered, the section wraps up with a discussion of what actually occurs from the participant’s perspective (DRIVER) in the Score Phase.

World-Class Management Step Charts
Each World-Class Management Step Chart has four steps: Clerical, Mechanical, Proactive and World-Class.  Each step has a varying number of attributes.  Attributes are added, removed or edited each year through the input and collaboration of Industry and Academic Think-Tanks.

Chart Development
Managers gain a tremendous amount of insight from a self-assessment process using step charts to develop their roadmaps.  Most organizations do not have roadmaps. If they do, they are excited to compare their roadmap against the step charts to check for completeness and gaps. The activity can be time consuming, but the outcome is a new understanding of the possible projects and initiatives that the organization could undertake to progress to world- class. The activity also leads to the conclusion that the future state of organizations is enterprise and alliance focused, not functionally focused.  Hence the use of the name Core Competency Thread or just Core Competency has emerged to describe each World-Class Management Step Chart.  This view is illustrated in the following figure.

Core Competencies clearly do not stop at the borders of a given facility.  Core Competencies span divisions and business units within an enterprise as illustrated in the next figure.

Core Competencies also span multiple organizations, such as chains and alliances that are collaborating together. The next figure illustrates the example of how core competencies span
functions, business units, divisions, chains and alliances.

Chart Development
A common question that is posed regarding the Core Competency step charts is “How were the charts developed?”  For a full description of how the charts are developed and maintained, please refer to the SCORE Process White Paper. 

Attribute Score
Each attribute has two associated scores called the Capability Score and Priority Score.  The default for each score is zero, meaning that the attribute is considered “irrelevant”.  If a Project Priority Score of 0-5 is assigned by the DRIVER, then the Capability Score must be zero.  The converse is also true.  A Capability Score from 0-5 means that a new project is not needed to develop the capability.  (Existing projects may be needed, but they do not warrant a project priority since they already are in process.)

Both scores can not have a greater than zero value at the same time.  A value of 1-5 for a Capability Score means that the Priority Score must be 0.  Conversely, a value of 1-5 for a Priority Score means that the Capability Score must be 0.

Color coding of the attributes is used to quickly identify the attribute’s category and score range.  When the Capability Score is 1, 2, 3 or 4, the text associated with the attribute is yellow.  When the Capability Score is 5, then the text associated with the attribute is green.  When the Priority Score is 1, 2, 3 or 4, the text associated with the attribute is blue.  When the Priority Score is 5, then the text associated with the attribute is red.  Black is the default color for irrelevant.  As previously stated, the default values are 0 (zero) for each score, which means the attribute is irrelevant to the organization conducting the assessment. 

After color coding, the Likert Scale from 0-5 is associated with each relevant attribute.  For Priority Scores, red and blue attributes, the score is a priority assigned to future project(s) needed to achieve the capability or state.  The Likert Scale Priority Scores break down as follows:

• 5 = highest priority project that must start within the next two years,
• 4 = high priority project that should start three years into the future
• 3 = medium priority project that should start four years into the future
• 2 = low priority project that should start five years into the future
• 1 = lowest priority project to start sometime after five years into the future
• 0 = should not become a new project in the future

For Capability Scores, green and yellow attributes, the rating reflects the percentage of implementation. Green attributes are 100% implemented; hence all green attributes are rated as a 5 on the Likert Scale.  Conversely, yellow attributes are assigned a 1 - 4 rating based on the percentage of implementation from 1% – 99%.  Breaking the Capability Score down into a list as given above for the Priority Score, we have the following:

• 5 = 100% implementation of the capability (or 100% achievement of the state)
• 4 = 80% implementation of the capability (or 80% achievement of the state)
• 3 = 60% implementation of the capability (or 60% achievement of the state)
• 2 = 40% implementation of the capability (or 40% achievement of the state)
• 1 = 20% implementation of the capability (or 20% achievement of the state)
• 0 = 0% implementation of the capability (or 0% achievement of the state)

For example, consider the yellow attribute “Producer’s life cycle costs are estimated in product design stage.”  A person rating the attribute might use the following logic. 

“We know that we have 120 designs over the last year.  Of the 120 designs, 100 of them needed measurement of producer’s life cycle costs in order to better understand future costs to be generated by the design.  Of the 100 candidates, 40 had their life cycle costs measured before the design was released.  Hence, we should rate the attribute as a 2 (2/5 = 40/100 = 40%) on the Likert Scale.”

Calculating the Chart Score
Each World-Class Management Chart has four steps: Clerical, Mechanical, Proactive and World-Class.  A Step Score is associated with each step that ranges on a scale from 0 – 2.5.  The Step Scores combine to create the Chart Score, which is on a scale from 0 – 10.  Hence, the default for the DR²IVE™ Model is that each step contributes 25% to the total Chart Score.  For a complete discussion of how the step score is calculated, please refer to the SCORE Process White Paper.

The chart score is calculated in the Score Phase and throughout the SCORE Process as the Attribute Scores are adjusted.   The Chart Score is simply the sum of the four Step Scores.

Associated with a Chart Score is a designation of Clerical, Mechanical, Proactive and World-Class based on the range where the Chart Score fits.  The Clerical Designation ranges from 0.00 – 2.49, Mechanical is 2.5 – 4.99, Proactive is 5.00 – 7.49, World-Class is 7.50 – 10.00.

For example, if the Step Scores for a given chart are Clerical Step = 2.1, Mechanical Step = 1.8, Proactive Step = 1.2 and World-Class Step = .4, then the Chart Score is 2.1 + 1.8 + 1.2 + .4 = 5.5.  The Chart Score of 5.5 falls within the Proactive range of 5.00 – 7.49; therefore the organization’s designation regarding the chart is “Proactive.”

DR²IVE™ Score
Gap analyses using the World-Class Management Step Charts provide management with information regarding an organization’s capabilities in comparison to world-class organizations.   When the analyses for all charts selected by the organization are pooled together, they can form a single weighted metric that is called the DR²IVE™ Score.  Please refer to the SCORE Process White Paper for the steps involved in calculating the DR²IVE™ Score, as well as and example.

Multiple Organization Scores
Once a single weighted DR²IVE™ Score has been calculated for each organization, the DR²IVE™ Score can be combined with scores of organizations into an Enterprise Score, Chain Score, Alliance Score, Merger Score, Acquisition Score or whatever combination of organizations that management desires.

A variety of ways could be conceived for how to weight individual DR²IVE™ Scores for a combination of multiple organizations.  Presenting all of the options is beyond the scope of this introduction to DR²IVE™; however, a short example of one approach for a supply chain is given to demonstrate the power of finally having a metric to evaluate combinations of organizations.

The approach presented here is to use the value added spend for the family of products that are made within the chain being measured. For example, if a family of 3.5 horsepower lawnmowers has five core alliance members in its chain then the value added spend might break down as follows:

• Firm A sells Firm B fabricated parts that cost Firm B 0,000 annually
• Firm B sells Firm C subassemblies (using the parts from Firm A) that cost Firm C 0,000 annually
• Firm C sells Firm D unassembled lawnmowers (using subassemblies from Firm B) that cost 0,000
• Firm D sells Firm E completed lawnmowers that cost 0,000
• Firm E sells lawnmowers to customers for ,000,000

The value added cost at each point in the chain is calculated as follows:

• Firm A = 0,000
• Firm B = 0,000 – 0,000 = 0,000
• Firm C = 0,000 – 0,000 = 0,000
• Firm D = 0,000 - 0,000 = 0,000
• Firm E = ,000,000 - 0,000 = ,000

The summation of the right hand sides equals ,000,000.  Since the numbers given in the example were chosen, it is easy to see that Firm A accounts for 21% of the value added spend in the chain.  Similarly, Firm B accounts for 45%.  All of the calculated percentages of the total value added spend are given in the second column of the following table.  The first column contains the DR²IVE™ Score for each organization.

Example of Calculating a Multiple Organization DR²IVE™ Score

The single chain metric can now be used as a benchmark to gauge progress in the future. The following figure presents a graphic view of how the Chain Score could be plotted and compared against goals set in the future.

Using the DR²IVE™ Score for Strategic Goal Setting

Scoring Activities
Scoring means each attribute in a given chart is first assigned a status and then, if the status permits, a Likert “Score” from 0-5 is assigned.  Status assignment starts with three categories: existing, needed or irrelevant.  The default assignment is irrelevant.  The three categories are explained as follow:

1. Existing means a capability exists for the attribute.  If existing, a Likert Scale rating is applied from 1 – 5, that indicates the percentage of implementation of the capability.  For example, if the capability has been implemented on 40% of the opportunities that are available, then the DRIVER will assign a score of 2 (2/5 = 40%) to the attribute.
2. Needed means a capability does not exist in the organization for the attribute, but is “needed” by the organization.  If needed, then the attribute (or some variation of the attribute) will become a project or set of projects in the future.  A Likert Scale rating is then applied from 1 – 5, that indicates the importance based on time when the project will start.  For example, if the project or set of projects derived from the attribute are considered as very important or “burning platform” project priorities, then the DRIVER will assign a score of 5 to the attribute.  As discussed earlier, a score of 4 indicates 2-3 years to start the project(s), 3 indicates 3-4 years, 2 indicates 4-5 years and a 1 indicates 5+ years to start the project.
3. Irrelevant means the attribute does not apply to the organization and can be omitted from the assessment and subsequent roadmap development.  If irrelevant is selected, then the DRIVER can not assign a score from 1-5 in the Score Column.

Status Assignment is represented by E, N, I in the following figure.  After status assignment, the DRIVER selects the Score of each attribute from 1 – 5.  The two step selection inputs both the Attribute Capability Score and the Attribute Priority Score.  If both scores are zero, then the attribute is irrelevant.

Scoring the Attributes

In the Score Phase, Reviewers only have access to see results of the scoring process in progress.  Reviewers can not change the scores.  Although scoring of the attributes is done by the DRIVER, a group can collaborate with the DRIVER to decide on the scores that will be assigned.  Such collaborators should not be Reviewers.  If they are, then the Consensus “Rating” Phase will be biased.

Consensus

The Consensus Phase is where the Attribute Scores from the Score Phase are reviewed by managers from the organization being scored.  The managers that are involved are called Reviewers.  Reviewers rate attribute scores within specific World-Class Management Step Charts to which they are assigned.  In other words, Reviewers only see charts they have been assigned to by a Master or Champion.  Reviewers cannot change the Attribute Scores, but through their ratings and a subsequent discussion, they can influence the final score given by the DRIVER in the “Optimize” Phase (next Phase).

Consensus Rating
The ratings are based on a 0, 1, 2 numerical scale that mimics a stoplight as shown in the following figure.

Rating the Attributes

In the figure, a rating with a red light = 0 means the Reviewer rejects the attribute score given by the DRIVER, hence the “R” in the light given in the figure.  A yellow light = 1 means they are not sure about the accuracy of the score and want further inquiry – so they are Neutral or “N” as in the light.  Finally, a green light = 2 means the reviewer accepts the score given by the DRIVER --- as represented by “A” for accept.

Reviewers are able to see the attribute’s score (with the color indicating its status), but they cannot change the score or status.  Reviewers do not have to rate 100% of the attributes.  They rate only those on which they have knowledge.  The Reviewer in this case has rejected two attribute scores currently showing in the figure. Reviewers do rate the attributes that the DRIVER deemed as irrelevant.  Often attributes are thought to be irrelevant at first, but are changed later to existing or needed.

Consensus Threshold
Consensus occurs if the average rating for the attribute is 1.5 or higher.  There is no lower limit to the number of reviews per attribute.  (Note: Attributes with less than three reviewers will be automatically selected for addition to the Discrepancy Report in the AUDIT Process if the organization chooses to pursue registration.)  A second rule had been considered to account for variance, but ultimately a high threshold (1.5 or 50%) that is simple is easiest.

Example: if half of the Reviewers give a green light and half a yellow light, then the average attribute rating is 1.5 and the attribute score passes.  Alternatively if one Reviewer gives a red light, two a yellow light and three a green light, then the score is rejected since the average rating is 1.33.

Consensus Index
The Consensus Index for an attribute is simply the actual sum of the ratings divided by the highest possible sum of the ratings.  Example: Suppose we have four Reviewers and half of the reviewers give a green light (2 pts each) and half a yellow light (1pt each).  The attribute score passes with a 1.5 average and it has a Consensus Index of 75%. ((2+2+1+1)/(2+2+2+2)) = 6/8 = .75 or 75%.

The Consensus Index is best presented as a percentage.  As with scores, the Consensus Index data can be “rolled-up” from an Attribute Consensus Index, to a Step Consensus Index, Chart Consensus Index, Organization or DR²IVE™ Consensus Index, Chain Consensus Index, Alliance Consensus Index, Industry Consensus Index, Regional Consensus Index, etc.

Optimize

After the Consensus Phase is concluded, Reviewers discuss rejected attribute scores in a face-to-face meeting or online discussion board to optimize the true scores.  The only attributes included are those whose Attribute Scores were rejected by the rating system in the Consensus Phase.  The term “optimize” means that the Optimize Phase provides the best possible consensus given constraints on time and people.

Reviewers are invited to discuss the low consensus attributes for a preset period of time.  Usually, two weeks are needed to complete this phase.  During that time, Reviewers and the DRIVER should present their facts as to why the attribute score should or should not be changed.  The DRIVER serves as the moderator (more of a cheerleader to get reviewers to participate) for the discussions.

Anytime during the discussions, Reviewers can provide their suggestions regarding the score, called a Suggested Score that they believe should have been given to the attribute.  The Suggested Score is just informational.  It does not go into a database.  The idea is to be able to quickly summarize what the Reviewers are concluding.  Based on the discussions, the DRIVER enters the revised attribute score.

Once the DRIVER has set a new “optimized” score, the discussion for that attribute is considered to be closed.  The optimized score now automatically replaces the original score and the attribute consensus is reset to 100%.  Note that if the optimized score is the same as the original score, then it should be loaded so that the Consensus Index for the attribute is set to 100%.  The following figure shows one way of reporting the optimized scores to the Reviewers.  The figure represents a table from what is called the Optimization Report.

Optimizing the Attributes

If the DRIVER chooses to create updates during the discussion period, he or she should post the changes to the Reviewers to let them know that discussion on that attribute has been closed.  Also, allowing Reviewers to see the impact of the discussions through revisions by the DRIVER helps to foster positive discussion.

Attributes that do not have an optimized score given in the Optimization Report have either not been “closed out” by the DRIVER in the discussions or the attributes may not have been resolved in the discussion.  If left unresolved, then the original score stands.

After completion of the Optimization Report, the DRIVER releases the results to the Review Phase where an automatically generated Chart Report is created.

Review

In the Review Phase, the DRIVER publishes the Chart Report complete with a summary, tables and charts.  Summary tables include the following in order of high score to low:

• High Priority Projects (Priority Scores of 5.0)
• Future Projects (Priority Scores below 5.0)
• Current Capabilities (Capability Scores of 5.0)
• Emerging Capabilities (Capability Scores below 5.0)

In the Review Phase, Reviewers have one last chance to suggest changes to the results.  In general, the focus of the Reviewers should be on typographical errors and inconsistencies with the final discussions for each attribute and the chart.  The Master may also make suggestions in this discussion since it is his or her duty to review the Chart Report and then either release it or not into the greater SCORE Report.

Once released into the SCORE Report, the Chart Report is said to be “published” into the Engage Phase.  Publish is an accurate term since copies may be printed and dispersed if desired.  Once published, the Chart Report is out of the control of the DRIVER and in the control of the Master.

Engage

The SCORE Report is published in the Engage Phase.  The SCORE Report contains all Chart Reports on which the organization assessed itself. The Engage Phase represents the first view of all Chart Reports that -- up to this point -- were developed separately and simultaneously.  All Reviewers and DRIVERs have access to all Chart Reports through the greater SCORE Report.  (Note: This is in contrast to the Review Phase which was internal for the Reviewers, DRIVER and Master to complete their assigned Chart Reports.)  SCORE Report results are stored in a historical database maintained by the Roadmaps Institute™.

The DRIVER’s work is done for all phases after the Review Phase in the SCORE Process.  DRIVERs will still participate, but they see the same content as Reviewers from here forward in this Phase and all remaining Phases.  The Master now has control of Phase Status.

The Master now has a choice to “Engage” through the following choices:

• AUDIT Process: Transfer the SCORE Report to the AUDIT Process to register the results.
• ROADMAP Process: Transfer the SCORE Report to the ROADMAP Process to collaborate with internal divisions, business units, merger organizations or acquired organizations.
• ROBUST Process: Transfer the SCORE Report to the ROBUST Process to collaborate with external organizations, such as alliances.