Developing A Video Production Process and Metrics

Jason Paul Kazarian

Originally composed as graduate work for a Masters Equivalent in Systems and Process Science from the University of Texas at Austin, October 12, 1996.
Converted to HTML September 1, 2000.
Last updated November 16, 2008.

Abstract

This paper explores introducing process control into chaos. Instead of improving an existing process, we discuss the development of a new process and metrics. We also formulate a long term quality plan.

Introduction

For the past ten years (Comment 1), process control has been viewed as American business's savior. Foreign companies have used American quality methods (Reference 1) to grow; why can't American companies do the same? This has led to a (usually) healthy injection of quality management, process control, statistical methods, and process metrics into mainstream American business. And the gains have been significant: the value (price increase - inflation loss) of the American stock market has doubled in ten years.

Certainly process engineering has benefited for-profit business, but can it benefit other parties as well, considering their resource competitions are often more intense? Specifically can the same techniques be applied to non-profit organizations? What will the results be like?

These questions are the inspiration for this paper. We first examine the business model for a recently formed non-profit organization, including its objectives and goals. We next research relevant literature to a develop candidate process. We then identify process deficiencies and potential metrics. Finally, we formulate a quality plan for upgrading the process over time.

Business Model

Organization Charter. The Dallas-Fort Worth Video User's Group is a non-profit organization for video enthusiasts committed to learning by doing. The Group (Note 1) produces pro-bono video projects for community service organizations and provides a forum for networking and technical interchange. The production experience of Group members ranges from none to several years of professional work. The Group has a stated goal of producing three to four projects per year or as many as necessary so all members can participate in one project per year. Consequently the group expects the number of productions per year to increase as membership grows.

The Group was formed in February 1996 and has bylaws that govern general parameters of operation such as funds, officers, committees, and so on. The Group has the flexibility to adopt policies, procedures, and processes as necessary to meet or exceed a Sponsor's expectations: the Group need not support institutionalized legacy.

Critical Parameters. Because the Group is staffed entirely by volunteers, time is of the essence. Although people are willing to tolerate down time when paid, most are not willing to donate spare time unless it is used wisely. Also by nature the Group's sponsors (community service organizations) are likely to be overworked, understaffed, and have sporadic needs.

The Group's members, as mentioned previously, have a wide range of production experience. Some members have none; others own video production businesses. The members together bring different mind sets to bear against any Group undertaking.

Given the scarcity of labor resources and a wide range of member experience levels, we need a process that can:

• Predict the time necessary to complete production activities.

• Partition well so members can work on pieces of a production separately.

• Allow those with less experience to make significant contributions.

• Provide a framework acceptable to members with more experience.

Current Dynamics: No Process

As a baseline example of problems the Group runs across without process control, here is a report of an actual production. Although the events occurred as noted, details have been pruned to protect member identities:

The producer wanted to shoot a training video. He asked for help from two videographers and three actors. The stated schedule was one hour of set up at 10:00, followed by a two hour shoot. All parties would be finished by 1:00. The shoot was indoors with studio lighting; the producer had documented a list of desired shots.

The videographers arrived an hour before shooting as planned to set up lights with the producer, but this task took two hours. The first actor arrived on time and consequently had to wait one hour. After the lights were set up, the producer started working shots with the first actor. There was no documented direction for the videographers about what was needed from a creative standpoint, so the producer personally visited with each videographer and gave verbal direction. This slowed down the production further. Subsequent actors arrived on time but had to wait. One videographer left one hour after shooting was originally scheduled to end; the other videographer kept shooting for two more hours. In spite of this, the producer still needed to schedule another shooting day to complete the production.

Without a process in place, the producer really had no way to plan several key elements. He guessed at a schedule. He could not communicate artistic objectives to the crew. Volunteer time was abused. Worst of all, he had no way to feed forward this experience into the next production for realizing improvement. Clearly a production process is necessary.

Introducing a Process

Goals. Brainstorming techniques (Note 2) were used to identify goals for a candidate production process. At the outset, this adopted process must provide:

• A path for translating ideas into a finished production.

• Defined and understood interim work products.

• A mechanism to aid Sponsor and Group communication.

• A sense of direction that artistic personalities can follow.

• Guidance for scheduling productions.

• Measures of attained quality.

Macro Level Process. The Group adopted a macro level process (Figure 1) which defines the first level demarcation of activities and work products. This macro process states that the following activities must take place:

• Identify and contact potential project Sponsors.

• Retain a Sponsor.

• Define high level project goals with a Sponsor.

• Staff a project with group members and other volunteers.

• Plan a production.

• Execute a production.

• Edit generated components into the desired sequence.

The macro process concludes with one ultimate output: the Group delivers a video production to the Sponsor.

This process is sufficiently detailed for communicating with a project Sponsor, but closer inspection reveals some deficiencies. The process is sequential: there is no feedback loop. Entry and exit criteria for activities are not documented. And the entire process is subjective: there is no measurement to determine when to stop refining a work product.

Our macro process activities are not balanced. Some activities are substantially more complex than others. For example, "Develop Production Plan" will take longer to complete than "Sign Up Team." The stakes are also different from activity to activity: success is more difficult to achieve for some activities than others. A micro level process is needed to partition many macro level activities, especially production planning.

Micro Level Process. Production planning is the translation of ideas (goals, objectives, visualizations) into actions: shoot this kind of scene, play this type of music, say these words, execute this transition effect. The end result of a "good" production plan is a video project that meets the Sponsor's objectives (Note 3).

Documents defining a production planning process exist: step by step activity guides, production handbooks, and trade publications are available to guide process definition along the way. For this exercise we looked at traditional handbooks, some focusing on technique (Reference 2), others having some process focus (Reference 3). We also did a web search for relevant information (Reference 4).

Strangely enough we found that a children's video handbook (Reference 5) provided the best outline for a production process: a sequence of activities with inputs and outputs. Integrating all of these sources into a Software Engineering Institute Level 3 Process (Reference 6) resulted in the micro processes presented in Tables I-III. Our task now is to audit this process, instrument it with metrics, and develop a quality plan for future improvement.

Quality Planning

Auditing the Micro Process. Once a process is developed, how can one be sure it produces the desired results? The cause and effect or Ishikawa diagram (Reference 7) attempts to answer this question. First, the desired result is postulated. In our case, that effect is a satisfied participant. Next, major contributing factors are drawn as trunks off a spine extending from the desired result. Finally, minor contributing factors are drawn as branches from the trunks. In many ways, a cause and effect diagram is analogous to a tree structure, and trees are analogous to outlines. So we present the result of our Ishikawa diagram in Exhibit I as an outline (Note 4).

In our Ishikawa outline we have three top levels: Audience, Sponsor, and Group. This reflects our three different customer types, each of which has a different interpretation of what constitutes quality. The causal factors are networked: for example, the Sponsor believes the project is of high quality if the publicity is favorable. And the publicity will more likely be favorable if the Audience is not bored. We do not attempt to analyze this network, but merely state it exists.

Potential Metrics. Once the potential causes are identified, we begin to identify metrics: if a satisfied Audience is not bored, we can increase the Audience's perception of quality by minimizing boredom. The questions then become what are the parameters for quality improvement and how are they measured?

To address these questions we strip the cause and effect outline of all but the lowest level elements (equivalent to a tree's leaves) and determine if these elements can be subjectively or objectively measured. We then postulate the process and activity where the measurement should occur, and suggest a document or unit for capturing the metric. The result of this task is presented as Table IV. Note that some processes, activities, and documents do not yet exist: these are the areas for improvement.

Expected Process Modifications. Finally we assess the improvement parameters and metrics to derive some quality goals and process modifications. The goals are:

• Reduce the total involvement time required from the Sponsor to produce a video, yet still allow for involvement at the Sponsor's discretion.

• Reduce Group volunteer down time.

• Increase Audience satisfaction with our projects.

The process modifications to achieve these goals are as follows:

• Extend the macro process to include feedback from the Audience via, but not limited to, a viewer survey. Currently the macro process ends upon master tape delivery. Also include feedback from the Sponsor in form of a satisfaction survey.

• Expand the Project Research activity of the Develop Content process to include a Quality Function Deployment (QFD) of Audience expectations (Note 5). Right now we ignore the Audience's desires in the macro process, yet their satisfaction is a key to our success.

• Expand the Treatment activity of Develop Content to include a concept analysis where the expectations derived from Sponsor objectives and Audience QFD are accounted for in the project. This analysis should also include an audit to verify that the order of presentation and examples used in re-enforcing the concepts follow in a structured fashion.

• Track wasted or reducible labor, such as Sponsor hours of involvement, unplanned interactions, turnaround time on Sponsor feedback, Master Schedule estimates versus actual, and so on.

As time goes on, all macro process stages should be documented to the same standards as the micro process: well defined with entry criteria, inputs, activities, outputs, and exit criteria. We believe this is mandatory for success in a dynamic volunteer environment.

References

Reference 1. Deming, William Edwards, 1982, Quality, Productivity, and Competitive Position, Massachusetts Institute of Technology, Cambridge, MA.

Reference 2. Hedgecoe, John, 1989, John Hedgecoe's Complete Video Course, Simon & Schuster Inc., New York.

Reference 3. Millerson, Gerald, 1987, Video Production Handbook, Focal Press, London, pp 151-171.

Reference 4. International Television Productions, A Guide to the Video Production Process (domain registration expired).

Reference 5. Guthrie, Donna and Bentley, Nancy, 1995, The Young Producer's Video Book: How to Write, Direct, and Shoot Your Own Video, The Millbrook Press, Brookfield, CT.

Reference 6. Bate, Roger, et al, 1995, A Systems Engineering Capability Maturity Model, Version 1.1, Carnegie Mellon University, Pittsburgh, PA, p 4-11.

Reference 7. Ishikawa, Kaoru, 1982, Guide to Quality Control, Asian Productivity Organization, Tokyo, pp 18-29.

Reference 8. Thompson, M. and Chao, K., October 1990, Quality Function Deployment and HP IVI, HP Journal, Palo Alto, CA, p 9.

Notes

Note 1. Throughout this paper, capitalized nouns are words with defined meanings. For example, the word "Group" means "Dallas-Fort Worth Video User's Group," the term "Sponsor" means "an organization that retains the Group to produce a video," and so on.

Note 2. Brainstorming is a multiple step process. In the first step, ideas are collected from and documented by the participants without constraint or edit. Next, redundant ideas are combined. Then similar ideas are grouped into categories. Finally a list of topics based on these categories is generated. The resulting list represents the group's thought process.

Note 3. There are parallels between video production planning and software engineering: the project objectives (system requirements) start at a high level. Eventually a script (software requirements) is generated. Work continues until production plans (software design) are detailed enough to commence shooting (coding). Shots are then edited (integration testing) into a video master (acceptance testing).

Note 4. Ishikawa postulates additional notations for cause and effect diagrams, such as drawing boxes around objectively measurable factors and underlining subjectively measurable factors. We delayed this delineation to the next activity.

Note 5. Quality Function Deployment is a standard method of ranking a market's expectations to a producer's capabilities. Customer needs, sometimes called murmurs, form columns of a matrix; engineering solutions form the rows. At each row and column intersection a positive, neutral, or negative correlation is recorded. The resulting matrix shows which needs fit well with available capabilities and which do not.

Comments

Comment 1. Although we have edited this paper for content and diction, all chronological references are still relative to the original date of authorship. In this case, "for the past ten years" references something that was true from October 1986 to October 1996.

Exhibit I: Micro Process Ishikawa Diagram Rendered as an Outline

Note: Branches Presented in Initial Capitals. Leaves presented in sentence case.

1. Audience Found Presentation Worth Viewing Time
   1. Presentation Made a Psychological Connection
      1. Relevant theme.
      2. Objectives followed from Audience expectations or needs.
   2. Viewer Learned Something
      1. Strong but Few Concepts Presented
         1. Three to five major concepts per presentation.
         2. No more than five concurrent dependencies.
      2. Concepts Easy to Grasp
         1. Concepts presented in sequential order.
         2. Concepts fit in unified theme.
         3. Examples reenforce concepts.
   3. Viewer Entertained
      1. Professional Quality Production
         1. Goofs, flubs not present.
         2. All elements fit together.
      2. Viewer not Bored
         1. Length of presentation just right, not too long or short.
         2. Length dictated by concepts.
   
   
2. Sponsor's Community Reputation Increased
   1. Sponsor Addressed a Genuine Need
      1. Sponsor has strong mission.
      2. Sponsor devotes energies to mission.
      3. Sponsor not diverted from mission due to production.
   2. Production Met Objectives
      1. Production designed top-down from objectives.
      2. Alignment with Objectives Maintained Throughout Process
         1. Constant reviews with Sponsor.
         2. Immediate incorporation of feedback from Sponsor.
   3. Favorable Publicity
      1. Noteworthy topic.
      2. Audience reacted well.
   
   
3. Group Enjoyed Producing Project
   1. Project Executed According to Plan; Volunteer Time Not Wasted
      1. Plan not ambiguous.
      2. Scheduling Accurate
         1. Activity sequence fits together.
         2. Time estimates are reasonably correct.
   2. Sponsor Flexible With Group
      1. Sponsor has confidence in Group.
      2. Production well planned.
      3. Process allows flexibility.
   3. Production Went Smoothly
      1. Professional integrity maintained.
      2. Group and Sponsor supportive of each other's mission.

Process Tables

Table I, Table II, and Table III are incorporated by reference.

Table IV: Improvement Parameters and Potential Measurements

Note: Existing Process Elements in Initial Capitals. New process elements in sentence case. O = Objective measurement factor. S = Subjective measurement factor.

Copyright © 1996-2008, Jason Paul Kazarian. All rights reserved.

See Table I, Develop Content Process.

See Table II, Design Production Process.

See Table III, Plan Production Process.

You are browsing the white paper.

+1 214 234 9283