Caption Quality Initiative

The full transcript of the conference and videotapes will be available after November 1, 2002,  from Jeff Hutchins, nomadjeff1@aol.com
Caption Quality Initiative

CONFERENCE REPORT
SEPTEMBER 14, 2002
The first-ever Caption Quality Initiative Conference was held September 14, 2002, in Fairfax, Virginia.  It was an all-day event attended by nearly 80 service providers and consumers of captioning.  Co-chairs of the event were Jo Ann McCann of the U.S. Department of Education and Jeff Hutchins, an independent consultant with 29 years’ experience in the captioning industry.
The agenda was designed to help the group meet the mission set out for the Conference, namely, to consider:
·        Whether there are problems in the delivery of captioning services to consumers;
·        What is the nature of any such problems;
·        Whether there is any need to fix these problems;
·        What, if any, solutions there may be; and
·        How to identify and implement those solutions.
One problem that the captioning industry faces is that there is no clear understanding what it means – or should mean – to say that a program is “captioned.”  Consumers and television industry executives have asked many questions about the nature of captioning.  The Federal Communications Commission (FCC) has chosen to set no standards for captioning content or to define what constitutes acceptable captioning.  If all the words in a program appear in its captions, even if there are some mistakes, it is pretty to clear to everyone that the program is “captioned.”  On the other hand, if none of the words in that program appear in captions, it is equally clear the program cannot be called “captioned.”  One mission of the Conference was to examine the question of how much of a program’s dialogue must be conveyed before the program is considered to be “captioned.”  In other words, what is the standard for captioning?
The day began with the introduction of Traci Randolph with the Disability Rights Office of the FCC.  She attended as an “observer” for the FCC to gather information that might be useful if the Commission revisits its 1998 decision not to set quality standards for captioning.
Following Traci, who spoke of what she hoped to learn at the Conference, there were reports from two panels, each of which included one “neutral” party (neither a consumer nor a service provider) and two service providers who were involved in the initial February 2002 meetings that led to the idea of having the CQI Conference.  (See “History of the CQI Conference” at the end of this report.)  Each panel was to report on those problems that have already been identified and the other topics already discussed in previous meetings.  The first panel addressed issues related to captioning of live programs (e.g. news and sports), and the second panel dealt with issues faced in captioning prerecorded programs (e.g. primetime dramas and comedies, soap operas, and documentaries).  Non-panelists were encouraged to ask questions or to comment on the points raised by the panelists.
Mark Golden, Executive Director of the National Court Reporters Association (NCRA), led the panel for discussion on live programming.  He was joined by Jack Gates, President of the National Captioning Institute (NCI), and Joe Karlovits, President of VITAC.
Judy Harkins, professor at Gallaudet University and Director of Gallaudet’s Technology Access Program, led the panel on prerecorded programming.  Her panelists included Tom Apone, Director of Operations of the Media Access Group at WGBH, and Max Duckler, President of CaptionMax.
 The items identified by the panelists and the conferees allowed the creation of two lists of known problems.  Following lunch, the conferees were asked to vote by colored dots as to which quality issues they considered to be the most important to solve at this time.  Each consumer was given 4 red dots to cast votes on live-captioning issues and 4 red dots to cast votes on prerecorded-captioning issues.  Each of the 18 captioning companies in attendance was given 4 blue dots for each category.
In subsequent discussion, there was some concern raised that people were not allowed to vote on a lack of captions on certain types of programming, notably Spanish-language broadcasts.  It was pointed out that the quantity of captioning is perceived to be a perennial problem regardless of a program’s language, and it is assumed that everyone would like to see more captioning available.  Therefore, there was no need to attract votes away from quality-related problems in favor of concerns about quantity.  There were two exceptions made to this “rule.”  One was in the lack of captions for special news reports, as this was deemed to be a quality issue because it is a matter of compliance with FCC rules.  The second was in the lack of local realtime captioned news programs.  It was also pointed out that the same issues would affect the quality of Spanish-language and English-language captioning.  There was general agreement that future meetings should make a greater effort to include Hispanic and Latino consumers.  (All consumers at this conference were members of advisory panels of the attending companies, and were invited by those companies to participate.  No other consumers were allowed to attend, though it is the intention of everyone to be more inclusive in the future.  Again, please see the “History of the CQI Conference” at the end of this report.)
The following tables show the problems/issues identified and the number of votes cast by consumers and service providers.  They are ranked from most total votes to fewest total votes.  Keep in mind that if all attendees cast all their votes in each category, there would be 72 votes by service providers (18 providers x 4 votes each), and 116 votes by 29 consumers.

PRERECORDED CAPTIONING

LIVE CAPTIONING

NOTES:  Dots for voting were given to 2 non-affiliated attendees:  Judy Harkins of Gallaudet and Mark Golden of NCRA.  That may explain the different numbers of votes cast by service providers.  It would appear that only 27 of 29 consumers voted on live-captioning issues, while all 29 in attendance voted on prerecorded-captioning issues.  Voters were allowed to place more than one of their four dots in the same category if they felt an issue was of overwhelming importance.
One will have to review the conference transcript or videotape for a complete description of what was meant by each problem.  In the Live Captioning table, “standard operating procedures” was a catchall phrase referring to a number of possible standards for captioner training, captioning style, quality measurement, and other factors that affect consistency in the industry.
Perhaps the greatest surprise of the voting is that the accuracy of the captions as written is not much of a concern, at least at this time.  We do not interpret this vote to mean that captioning accuracy is not a real problem or a concern among consumers, but rather that consumers will not worry about the content of the captions until they can consistently receive and read the captions.  On the other hand, the strong vote in favor of standards and a way to complain about problems does suggest that accuracy is an underlying problem.
It is very important to note the differing views of consumers and of captioning professionals.  The numbers do not tell the whole story, as there are unequal numbers of voters.  (Remember that companies could cast just one vote, no matter how many employees they had in attendance.)  Consumers and service providers selected the same 5 issues (in differing order) as their top concerns for Live Captioning, but not in Prerecorded Captioning.
In Live Captioning, virtually all consumers (30 votes from 27 voters) felt that the need for “standard operating procedures” among live captioners and broadcasters was a major concern.  Yet less than half – only 8 out of 17 voting companies – agreed that standardization is a top priority.  On the other hand, nearly all service providers recognize a shortage of skilled realtime captioners as a concern (making it the #1 anxiety among captioning companies, even though some of the companies in attendance do no realtime captioning), while fewer than half of the consumers shared that concern.  Captioners were also far more concerned about the accuracy of the captions than consumers were.  This discrepancy is likely due to the fact that the captioners pay attention to the captions they create, and are less concerned with problems associated with the reception and display of the captions.
Concerns about caption delivery are especially prevalent in the results of voting on Prerecorded Captioning problems.  The top two issues in the minds of consumers are both related to matters of consumption.  (Gee.  Consumers are mostly concerned about consumption.  Who’d have imagined that, eh?)  Everyone agrees that there is a need for a single point of contact when problems occur, and, indeed, this issue could just as easily have applied to Live Captioning as Prerecorded.  Service providers are less concerned about on-going monitoring, perhaps because they already do a great deal of monitoring, and also, perhaps, because they know very well the difficulties and costs involved in doing more.
Finding and training off-line captioners (“off-line” refers to the process of captioning a prerecorded program) is not one of the top 5 concerns of service providers, yet about half of the consumers feel that better recruitment and training will lead to improved captioning.  This fact suggests that accuracy in Prerecorded Captioning may be a problem after all.  If you combine the votes of the two caption-timing issues, that issue becomes a slightly higher priority than captioner training.  According to consumers, captions terminating in the middle of a program is the fifth most important issue, but according to service providers, grammar errors in the captions deserve much higher consideration.
Despite these small discrepancies in the perceptions of consumers and service providers, the top five priorities in each category are remarkably consistent, and helped point the way to the second half of the Conference – deciding what steps to take, if any, to remedy the major problems.
Service providers in attendance were then invited to speak, for up to 5 minutes, on any subject related to captioning quality.  Eleven companies took advantage of this time to speak about their approach to quality control, or about the issues already raised.  Several spoke, in particular, about the need for more research into captioning to help pinpoint some of the goals the industry should be trying to achieve in terms of quality.  There was some discussion about the fact that commercials are being captioned by non-captioners working at production facilities.  Other speakers talked about the problems related to captioning companies being forced to find sponsors for the captioning and related to the recent decision by NBC to award contracts to the lowest bidder in an Internet auction.  Several speakers offered suggestions for solving some of the identified problems, especially in terms of certifications and standards for captioners to follow.
The conferees then broke into five small groups.  Each group was told to take 50 minutes to come up with a maximum of five recommendations to address the top issues voted upon earlier.  The discussions in these groups were extremely lively, and the methods used were diverse.  Each group was asked to appoint a “leader” to report that group’s recommendations.
The recommendations portion of the Conference was fascinating.  Approximately 75 people broke into small groups.  Each of these people had similar backgrounds, came into the Conference reasonably well informed about the process of captioning, and listened to all the same information shared during the first part of the day.  The question was, would the five groups reach similar conclusions about the next steps to take, or would opinions be splintered, sending the Conference in multiple directions?
Amazingly, the 5 groups – each of which included consumers and service providers, and none of which included more than 2 people from the same company – made nearly identical recommendations.  There were a total of 9 different recommendations received.  (Two others simply set a deadline for action and suggested that actions be coordinated with the NCRA.)  Again, everyone was asked to vote using dots as to the four steps they would most like to see taken next.  Again, service providers were allowed 4 dots per company, and each consumer also got 4 dots.  The following table shows the results.  Readers should note that the complete meaning of each recommendation is contained in the Conference transcripts and videotapes.

What is most significant here is that all 18 captioning companies present appear to have voted to support the creation of a trade association!  That bodes well for the future.  At least in the case of the two most-favored actions, consumers and service providers were in total agreement.  In truth, if a trade association is created and becomes active, that association is likely to tackle all of the problems identified at the Conference, and to act upon virtually all the recommendations from the conferees.
One consumer pointed out, during the final voting, that the recommendations did not include setting “standard operating procedures” for captioners and broadcasters, which was earlier identified as one of the most urgent problems.  The reply was that setting “standards for the practice of captioning” amounts to the same thing.  And, indeed, this solution was one of the top vote getters.
With the announcement of the vote, final discussion in the room turned to how the group might implement its own recommendations.  It was suggested that the Conference should not adjourn without a plan for how an association would become organized.  Several conferees asked whether Conference Co-Chair Jeff Hutchins would be willing and able to chair the Organizing Committee.  He agreed to do so.  It was strongly recommended that efforts be made to form the association in time for it to hold its first meeting at the huge National Association of Broadcasters Convention in April in Las Vegas.  To do so would require that organizational meetings commence this year.  The suggestion was made that the Organizing Committee consist of one employee and one consumer advisor from each captioning company.  There was general agreement that both consumers and service providers should be part of the Organizing Committee, and that meetings of this committee should take place in different parts of the country.  With Jeff’s consent to begin forming the Organizing Committee immediately, the first CQI Conference was adjourned at 5:55 p.m. and the conferees gathered for a communal dinner.

HISTORY OF THE CQI CONFERENCE

Some people have wondered how the Conference came into being and how it was decided who should participate.  Here is a brief history of events.
The idea for the Conference was born last February when 4 of the captioning companies got together (at NCI) to talk about common problems they faced, particularly on September 11, 2001, when the major networks stayed on the air for more than 4 days without a break to cover the terrorist attacks on America.  At the meeting, in addition to NCI, VITAC, CaptionMax, and WGBH, were Jo Ann McCann of the U.S. Department of Education, and Mark Golden, who is Executive Director of the National Court Reporters Association (NCRA).

Following the meeting, Joe Karlovits of VITAC called Jeff Hutchins and said it had been suggested that service providers and consumers should come together to discuss captioning quality, to determine if consumers agree with the problems identified by the captioners and, if so, to chart a course of action to deal with these problems.  It was noted that there is no agreed upon standard for captioning and that the FCC has declined to set any rules for what constitutes a "captioned program."  Joe said he and Jo Ann felt Jeff would be the right person to organize a meeting of consumers and the service providers.

Jeff began organizing a conference almost immediately, working independently.  Jo Ann agreed to be co-chair, and together, she and Jeff began mapping out the details for the conference, which was dubbed the Caption Quality Initiative -- CQI.  Initially, they intended to invite just those captioning companies that have DOED grants, each of whom is required to have an advisory panel of consumers.  Companies were to bring their panelists to the conference, and Jo Ann and Jeff intended also to invite representatives from all the major consumer groups.

As plans began to fall into place, many problems surfaced with the original ideas.  The original site was to be the Kellogg Conference Center at Gallaudet, but after several delays and problems were encountered there, Jeff and Jo Ann began searching for alternate sites in the DC area.  As for the invitation list, with hundreds of consumer groups in the country, things threatened to get out of hand logistically AND politically if attendance were not limited.  Furthermore, because there was no group or organization that was sponsoring the conference, all the costs of the meeting would have to be borne by the attendees.  Jeff and Jo Ann realized the cost was going to be close to $150 per person, and felt that would be a hardship on many consumer groups, who would also have to pay for travel and accommodations.  If they invited reps from a few consumer groups, such as NAD, SHHH, AGBA, and ALDA, then other groups might be insulted and angry.  The captioning companies planned to pay the expenses for their consumer advisers, many of whom also represent the major consumer groups.  That would mean that some groups would get free representation while others would have to come up with a lot of money on their own.

In the end, it was decided that the only fair and workable solution was to limit attendance for this first CQI Conference to consumers who are already members of a formal advisory panel.  To do anything else would have invited chaos and charges of inequity.  With very limited resources for doing the actual Conference planning, it was important to keep it to a manageable size.  In mid-June, a contract was signed with the Hilton Garden Inn in Fairfax, Virginia.  The hotel said they would be unable to accommodate more than 80 people.  The only prudent course of action was to limit the Conference.

However, Jo Ann and Jeff had already changed their minds about which companies to invite.  They decided to extend invitations to all the major service providers, whether or not they had a current DOED grant for captioning.  Invited companies were asked to report any other service providers who should be invited. In the end, invitations were sent to 22 companies, encouraging each of them to bring consumers with them.  By mid-August, we had nearly 80 people signed up (and paid) to attend, and on September 1, it was announced the Conference was "Sold Out."