TIMSS 23 DC Verification guide - QQ and Paper Based cognitive materials

If you are new to a verification project, refer to the following induction articles.

What is "Verification"?

DOs and DONTs in comment writing

NAF: National Adaptation form - a documentation form in Excel format

NRC: National Research Center

NPM: National Project Manager

VFF: Verification Feedback Form

ISC: International Study Centre - the organisation which approves the national adaptations AFTER the verification

The countries' translators received this  Translation and Adaptation guidelines QQ (PDF) and produced the translations accordingly.

It is a rather long document. You do not have to read everything but you may want to consult it when you have a doubt about national adaptations.

This part applies to the verifiers who received more than one Teacher questionnaire files 

Some countries will submit TQ in just one file (+ online cover, or no online cover).
Other countries will submit TQ in 3 TQ files. The following is about those countries:

Grade 4: There are 3 teacher questionnaire files (TQ, TQ_M and TQ_S). The TQ file has General section (question numbers are G1, G2, etc.), Math section (M1, M2, etc.) and Science section (S1, S2, etc.). The general section of TQ is duplicated in TQ M and TQ S. The Math section of TQ is duplicated in TQ M. 

Grade 8: There are 2 teacher questionnaire files (TQ_M and TQ_S). The TQ M file has General section and Math section (M1, M2, etc.). The TQ S file has General section and Science section (S1, S2, etc.). So the general section appear in both TQ M and TQ S.

👆 You will verify the "Main" or "Mother" TQ. You will not verify again other "child" TQs (in case of G4, TQ M and TQ S, in case of G8, TQ S), as the contents are identical. You will just copy your interventions and comments into child TQs.

In G4 TQ, make interventions and comment in the TQ file, reflect your interventions and comments in other two files. 

In G8 TQ, make interventions and comment in the TQ M file,reflect your interventions and comments in other two file: TQ_S.  

NextCloud

In the dispatch mail, you received an NextCloud link. The password is VerifierDC23. You will see the following folders:

Download all files from 1_cApStAn_to_VER. You will find following files.

• NAF (Excel)

• TARGET questionnaires and covers 
  

Download all files from z_Source_and_reference_files. You will find following files:

• SOURCE questionnaires and covers and (if your country take part in FT you will also have FT to DC change list (this list will show you all the changes in the source from FT to DC- please check them carefully in target)
  If your country took part in the Field Trial, in the z_Verified_TIMSS_23_FT folder: you will find the verified FT files and NAF - these are provided for reference. As there was no final check for this project, you can refer to the NAF to see NRC feedback. Please check this NAF and if your edits in FT were not accepted, it is better not to propose the same edits.
  
  

• Before starting your verification please change your user name to "Verifier"
  

  Go to Edit (or the Acrobat menu if you're on a Mac) - Preferences - Commenting and tick off the box next to "Always use Log-In Name for Author name".

  Now right-click a comment and select Properties. Go to the General tab and change the Author name. Make sure to also tick the box that says "Make Properties Default".

  Click OK, and any new comment you create (from that type) from now on will have your name as the Author by default.

(If the links do not work, try to open them in Firefox.)

Verify

Open the source, target and the NAF. 

Please refer to the following video about how to fill in the NAF.

https://www.gotostage.com/channel/9f04bf40aa2c4570aaa24d74f14ed8ac/recording/d996a5de24dd42bba5e06cc55a61d07d/watch

Before you begin, read through the Linguistic Style Guide (separate Excel file - available on NextCloud > 1_cApStAn_to_VER folder), where the country's translation team noted the agreed stylistic conventions such as form of address, decimal separators, date/time notations and language variant.

👉Is your target PDF format?

Please refer to this video: https://www.gotostage.com/channel/9f04bf40aa2c4570aaa24d74f14ed8ac/recording/a68eca3f28b24cdf8bb5fc6f55032ff8/watch

First, make sure you have Adobe Acrobat Reader on your computer. If not, please download it here: https://get.adobe.com/uk/reader/ by clicking on "Download Acrobat Reader DC".

Highlight the parts that requires correction. Insert a comment.

Once you have highlighted a part, you can double-click on it and insert a comment. Your comment could read something like:

Not able to highlight the correct parts - i.e. you are not able to select a part but the highlight bubble gets so big? Like this?

It's because when country exported to PDF format, they did not select "reconise the content as TEXT" but selected "as IMAGE".

In that case, instead of highlight, use the Comment bubble , and try to place the bubble as close as possible to the text you are referring to.

👉Is your target RTF (=Word) format?

First set the language of the document as your target language in Review tab:

Activate Track Change mode:

Make a correction, select the part where you made a correction and insert a comment (right mouse click, or from Review tab, select "New Comment"):

💡 Recurring issue: make corrections in all occurrences in the target file but comment only once per questionnaire target file and also in the corresponding tab in NAF. Comment in the first occurrence and end your comment with "Comment applies throughout in this questionnaire. Comment not repeated." 

🛠 Errata

Currently , there is NO errata in QQ.

Covers NOT included?

No covers are included in your version?

In that case, in the target files you will not find most of the required adaptations documented in the Covers & Directions tab in NAF.

• In the tab Covers & Directions in NAF, in row 7, enter a comment:

• “Covers have not been submitted for verification. Verifier proofread the version in Col. C for any linguistic defects. All OK in that respect. Comment not repeated in this worksheet. Requires ISC approval.” without a severity code.

• In all other rows that contain country’s documentation about the covers, enter “see VER’s general comment in row 7.”

To report your code 1, code 1? interventions made to the Intro or Prologue part of each questionnaire, use also the Covers & Directions tab in NAF.

Online covers

Are "online covers" included in your version?

Some countries will submit online covers. Those are the additional questionnaire covers. The content of those online covers are almost the same as the covers that are included in the questionnaire file. Differences are the last couple of paragraphs (after "... and thank you for your cooperation and contribution"). Your comment would look like this:

"Code 1. Required adaptation not implemented as documented for <Grade 4>. Corrected by VER in both cover and online cover."

Required Adaptations

• Required Adaptations (compulsory adaptations) are shaded in yellow in the NAF.

• Check in the target files (PDF or Word) to see the adaptations are implemented as documented. If yes, enter a comment in the NAF (not in the target file): "Adaptation implemented as documented. Requires ISC approval."

Other country comments

• Whenever there are comments from the country in NAF, please read them.

• If a comment is about changes/adaptations, or about how the country translated certain texts, it requires your reaction. Check if the target looks the same as what is documented in the NAF. If yes, enter a comment in the NAF (not in the target file): "Adaptation implemented as documented. Requires ISC approval."

Does every row have to be filled in, in NAF?

No, leave the rows empty when there are no issues to report and no required adaptations.

Do all comments be copied in NAF?

No, copy only your comments with Code 1and 1? from the target questionnaire into the NAF = major issues only

Code 3 comment should not be in NAF at all.

Code 2 comment shouldn't normally be in NAF. The only case Code 2 comment should be copied in NAF is when you are reporting a minor error in Required Adaptations.

💡 Tip - use of Filter

Do NOT add rows or columns in NAF.

You can make use of a filter. For instance you can filter the column "International version" by light yellow cell color to filer Required Adaptations.

❗ ATTENTION: do NOT sort. 

You can deactivate the filter by selecting "Clear filter From...".

Last check

• Check that you have provided a comment for each row which contains a required adaptation in NAF (shaded in light yellow).

• Read your comments through once more to make sure they are understandable.

• Make sure that it is clear from the comment whether you have made he change in the target file or not.