First Draft Practice Jobs Report Explained

Overview

The First Draft Practice Job report allows administrators to see how their editors have performed on the various First Draft Practice Jobs that are available through the ITC. Each time an editor completes one of the practice jobs and clicks the “Grade” button in the ITC, the results (QA score, keystrokes, time taken) are sent up to the platform. An administrator can then evaluate and compare those attempts between editors to see who may need additional help in the transition to doing First Draft work.

Tasks

To access the report, follow these steps:

• Log in to the IMC.
• Click the tab First Draft -> Reporting.
• Click on the Practice Job Report button.
• Set the date range based on which attempts you want to look at
• Select the MT or MT pool desired, or leave “All” selected.
• Click Display Report

The left most column / top group of the report is the MT whose attempts you are looking at. Under that, you’ll see the individual practice jobs that MT has attempted. If an MT is not listed, then he or she did not attempt any practice jobs in the date range specified. The top gray line for each practice job is an estimate of what that MT’s best attempt was in the specified date range. The attempts are ordered by those having a high QA score (greater than 90%), then a high keystroke score (greater than 80%), then by the amount of time it took to work the job.

Each attempt is listed on a white line with summaries of how many keystrokes were used in each category. To see a further breakdown of the specific keystrokes, click the ‘+’ button in the Navigation column. Each keystroke column first displays how many keystrokes an editor used in that category, then in parentheses a comparison to the ideal. For instance, if the MT used 46 Navigation keystrokes, and the ideal set used 51, then the column would read “46 (-5 from ideal).” The ideal set of keystrokes was generated by a tester attempting to edit the document in as few keystrokes as possible. It’s possible to edit the document in fewer keystrokes (especially if audio synch is being used), but editing the document in more keystrokes is probably using more than needed. The keystroke grade column is based on the difference between the total keystrokes the editor used vs the total keystrokes in the ideal set. If the editor used a lot more keystrokes, then the keystroke grade will go below 100%.

The right most columns provide some additional statistics for each attempt.

• Chars typed – these are keystrokes which aren’t tracked in any of the earlier categories. This includes not only normal a-z characters, but things like the space bar and delete key. If an MT presses delete several times (instead of ctrl + delete) then this number could be much higher than the ideal.
• Mouse clicks – Since the ideal number of mouse clicks is 0, this is the only column that doesn’t have a comparison to the ideal. Each mouse click will count as 20 keystrokes when generating the final keystroke grade.
• Copied / pasted chars – these metrics are not counted in the keystroke grade since the keystrokes used (ctrl + c / ctrl + v) are tracked elsewhere. These metrics are mostly here to make sure there isn’t any suspicious activity happening. If the pasted character count is much higher than the copied character count, then it could be that the MT is copy and pasting in from an external document (and is likely cheating).