ܫܪܝܪܘܬܐ ܒܝܢܝ ܪܡܙ̈ܢܐ: ܩܐܦܐ ܕ-Cohen ܘܐܠܦܐ ܕ-Krippendorff

ܡܚܫܒܢܐ (calculator)

ܛܥܢܐ ܕܝܗܝ̈ܒܬܐ (loading the data)

ܟܡܕܒܩܚ ܥܡܘ̈ܕܐ ܬܪܝܨܐܝܬ ܡܢ ܓܠܝܘܢܐ (worksheet)، ܥܡ ܣܕܪܐ ܕܪ̈ܫܐ (header row) – ܟܪܫܡܚ ܓܘ ܐܩܣܠ، ܟܢܣܚܚ، ܘܟܡܕܒܩܚ ܠܐܟܐ:

ܓܘܒܝܐ ܕܪܡܙ̈ܢܐ ܘܕ-metric

ܡܕܒܪܢܐ (guide)

ܐܝܡܢ ܡܦܠܚܝܚ ܒܡܚܘ̈ܝܢܐ ܕܫܠܡܘܬܐ ܒܝܢܝ ܪܡܙ̈ܢܐ؟

ܐܝܡܢ ܕܒܢܝ̈ܢܫܐ ܝܢܐ ܡܐܢܐ ܕܡܫܘܚܬܐ. ܦܘܢܝܐ ܦܬܝܚܐ ܓܘ ܒܘܨܝܐ ܕܫܘܐ̈ܠܐ (survey) ܕܦܪܝܫܐ ܝܠܗ ܠܡܘܠܐ، ܦܘܣܬ ܕܪܫܝܡܐ ܝܠܗ ܐܝܟ "ܥܕܘܝܐ" ܝܢ "ܡܨܥܝܐ"، ܡܢܬܐ ܕܡܚܙܝܢܘܬܐ ܕܫܩܠܠܗ ܪܡܙܐ ܡܢ ܟܬܒܐ ܕܪ̈ܡܙܐ، ܦܘܪܫܢܐ ܕܟܘܪܗܢܐ (diagnosis) ܡܢ ܪܫܝܡܬܐ ܕܓܢ̈ܣܐ – ܓܘ ܟܠ ܐܢܝ ܡܢܝܢܐ ܕܥܐܠ ܠܦܘܫܩܐ ܕܝܢܐ ܝܠܗ. ܫܪܝܪܘܬܐ ܒܝܢܝ ܪܡܙ̈ܢܐ (intercoder reliability) ܟܫܐܠܐ ܐܢ ܪܡܙܢܐ ܐܚܪܢܐ، ܕܫܩܠܠܗ ܗܢܝ ܦܘܩ̈ܕܢܐ، ܗܘܐ ܡܛܐ ܠܗܢܝ ܪ̈ܡܙܐ – ܘܕܠܐ ܦܘܢܝܐ ܚܝܘܒܝܐ، ܠܐ ܡܨܝܐ ܠܡܝܕܥ ܐܢ ܡܫܟܚ̈ܢܘܬܐ (findings) ܟܡܚܘܝܝ ܟܬܒܐ ܝܢ ܠܐܝܢܐ ܕܩܪܐܠܗ. Cohen (1960) ܟܬܒܠܗ ܐܗܐ ܒܨܘܪܬܐ ܕ"ܡܐܢܐ ܕܡܫܘܚܬܐ ܕܬܪܬܝܢ ܐܩ̈ܠܬܐ" (two-legged meter): ܪܡܙ̈ܢܐ ܨܘ̈ܪܬܐ ܡܩܒ̈ܠܢܝܬܐ ܝܢܐ ܕܗܘ ܡܐܢܐ، ܘܫܘܐܠܐ ܝܠܗ ܟܡܐ ܐܢܝ ܫܠܝܡ̈ܐ ܝܢܐ. Krippendorff (2019، ܨܚܚܐ 11) ܟܡܙܝܕ ܓܢܒܐ ܐܚܪܢܐ: ܫܪܝܪܘܬܐ (reliability) ܫܪܛܐ ܣܢܝܩܐ ܝܠܗ ܩܐ ܬܪܝܨܘܬܐ (validity)، ܠܐ ܥܪܒܘܬܐ ܩܐ ܕܝܠܗ – ܬܪܝܢ ܪܡܙ̈ܢܐ ܡܨܝܐ ܕܫܠܡܝ ܘܛܥܝ ܐܟܚܕܐ.

ܐܗܐ ܦܐܬܐ ܓܫܩܐ ܫܬܝܬܝܐ ܝܠܗ ܓܘ ܛܟܣܐ ܕܥܒܕܐ ܕܦܐܬܐ NLP ܒܠܫܢܐ ܥܒܪܝܐ: ܒܬܪ ܪܡܙܢܘܬܐ ܒܐܝܕܐ ܕܕܘܡܝܐ (ܒܝܕ ܬܪܝܢ ܒܢܝ̈ܢܫܐ، ܝܢ ܒܝܕ ܒܪܢܫܐ ܘܡܘܕܝܠ)، ܗܪܟܐ ܟܡܫܚܚ ܫܠܡܘܬܐ – ܘܒܣ ܐܢ ܣܦܩܐ ܝܠܗ، ܟܐܙܠܚ ܠܩܕܡ ܠܣܓܝܐܘ̈ܬܐ، ܠܟܝ ܡܪܒܥܐ ܘܠܐܣܘܪ̈ܐ ܕܚܕ̈ܕܐ.

ܬܪܝܢ ܡܢܝ̈ܢܐ ܡܫܘܬܦ̈ܐ، ܬܪܝܢ ܫܘܐ̈ܠܐ: κ ܕ-Cohen ܘ-α ܕ-Krippendorff

ܫܠܡܘܬܐ ܓܘܠܡܝܬܐ (percent agreement) – ܡܢܬܐ ܕܚܕ̈ܝܘܬܐ ܕܥܠܝܗܝܢ ܪܡܙ̈ܢܐ ܫܠܝܡ̈ܐ ܗܘܘ – ܝܠܗ ܡܢܝܢܐ ܕܟܡܬܦܗܡ ܡܢ ܓܢܒܗ، ܘܟܡܛܥܐ: ܬܪܝܢ ܪܡܙ̈ܢܐ ܕܟܡܫܕܝ ܙܘܙܐ ܒܕ ܫܠܡܝ ܓܘ 50% ܡܢ ܡܘܨ̈ܒܐ ܓܘ ܬܪܝܢ ܓܢ̈ܣܐ، ܘܓܘ 80% ܐܢ ܚܕ ܓܢܣܐ ܣܓܝ ܡܬܚܙܐ. ܟܠ ܡܢܝ̈ܢܐ ܡܫܘܬܦ̈ܐ ܪ̈ܨܝܢܐ ܟܡܬܪܨܝ ܐܗܐ ܒܗܝ ܐܘܪܚܐ: (ܫܠܡܘܬܐ ܚܙܝܬܐ − ܫܠܡܘܬܐ ܡܣܟܝܬܐ ܒܓܕܫܐ) / (1 − ܫܠܡܘܬܐ ܡܣܟܝܬܐ ܒܓܕܫܐ). ܦܘܪܫܢܐ ܒܝܢܝ ܐܢܝ ܝܠܗ ܒܡܐ ܕܡܬܚܫܒ "ܓܕܫܐ".

ܐܝܡܢ ܕܪܡܙ̈ܢܐ ܟܡܦܠܚܝ ܒܓܢ̈ܣܐ ܒܗܢܝ ܣܓܝܐܘ̈ܬܐ، ܬܪܝܢ ܡܢܝ̈ܢܐ ܡܫܘܬܦ̈ܐ ܩܪܝܒܐ ܚܕ ܝܢܐ. ܐܝܡܢ ܕܠܐ – ܐܢܝ ܟܦܪܫܝ، ܘܒܐܘܪܚܐ ܝܕܝܥܬܐ: Krippendorff (2019، ܦܣܩܐ 11.5) ܟܡܚܘܐ ܬܪܝܢ ܠܘ̈ܚܐ ܥܡ ܗܢܝ 46 ܡܢ 100 ܡܘܨ̈ܒܐ ܕܫܠܡܘܬܐ، ܕܓܘ ܚܕ ܡܢܝܗܝ ܫܠܡܘܬܐ ܕܦܝܫܬܐ ܝܠܗ ܒܕܝܪܬܐ ܝܠܗ ܒܫܘܝܘܬܐ ܘܓܘ ܐܚܪܢܐ ܟܢܝܫܬܐ ܝܠܗ ܓܘ ܚܕ ܓܢܒܐ؛ π ܕ-Scott (ܘ-α) ܟܝܗܒܝ .186 ܓܘ ܬܪܘܝܗܝ، ܐܠܐ κ ܟܣܠܩܐ ܡܢ .186 ܠ-.258 ܓܘ ܬܪܝܢܝܐ – "ܠܐ ܫܠܡܘܬܐ ܪܒܬܠܗ ܐܠܐ ܡܨܝܢܘܬܐ ܕܡܢܒܝܐ ܓܢ̈ܣܐ ܕܚܕ ܪܡܙܢܐ ܡܢ ܕܐܚܪܢܐ"، ܘܠܦܘܬ ܪܥܝܢܐ ܕܝܠܗ κ ܒܗܕܐ ܟܡܫܚܐ ܡܕܡ ܕܠܐ ܝܠܗ ܫܪܝܪܘܬܐ. Cohen (1960، ܦܐܬܐ 41) ܚܙܐܠܗ ܒܗܝ ܕܝܠܢܝܘܬܐ ܒܚܬܝܬܘܬܐ ܝܘܬܪܢܐ: ܓܘܒܝܐ ܡܬܬܩܢܐ ܕܪܡܙܢܐ ܩܐ ܓܢܣܐ ܦܪܝܫܐ ܝܠܗ ܡܒܘܥܐ ܕܠܐ ܫܠܡܘܬܐ، ܘܠܐ ܣܢܝܩ ܝܠܗ ܠܬܘܪܨܐ ܠܥܠ. ܚܪܝܢܐ ܠܐ ܝܠܗ ܦܣܝܩܐ، ܘܦܐܬܐ ܟܡܘܬܢܐ ܬܪܘܝܗܝ: κ ܡܛܠ ܕܩܪܘ̈ܝܐ، ܟܬܒ̈ܐ ܕܙܒܢܐ ܘ-SPSS ܟܣܟܝ ܠܗ؛ α ܡܛܠ ܕܐܝܠܗ ܝܚܝܕܝܐ ܕܟܦܠܚ ܥܡ ܬܠܬܐ ܪܡܙ̈ܢܐ ܘܠܥܠ ܘܥܡ ܚܣܝ̈ܪܐ ܕܠܐ ܫܕܝܐ ܕܝܗܝ̈ܒܬܐ.

ܩܪܝܢܐ ܕܦܐܪܐ (Interpreting the result)

ܡܐܛܪܝܟܣ ܕܫܠܡܘܬܐ (ܠܘܚܐ 1، ܩܐ ܬܪܝܢ ܪܡܙ̈ܢܐ) ܝܠܗ ܠܘܚܐ ܕܡܢܗ ܟܠ ܡܕܡ ܚܫܝܒܐ ܝܠܗ: ܣܕܪ̈ܐ – ܪܡܙܢܐ ܩܕܡܝܐ، ܥܡܘ̈ܕܐ – ܬܪܝܢܝܐ، ܣܪܛܐ ܡܨܠܝܐ (diagonal) – ܫܠܡܘܬܐ. ܩܕܡ ܕܚܙܚ ܡܢܝܢܐ ܡܫܘܬܦܐ ܛܒ ܝܠܗ ܠܡܚܙܐ ܠܗ: ܐܝܢܝ ܬܪܝܢ ܓܢ̈ܣܐ ܟܡܬܚܠܛܝ ܚܕ ܥܡ ܐܚܪܢܐ؟ ܐܗܐ ܝܠܗ ܐܘܠܨܢܐ ܕܣܢܝܩ ܝܠܗ ܠܬܘܪܨܐ ܓܘ ܟܬܒܐ ܕܪ̈ܡܙܐ، ܘܡܢܝܢܐ ܡܫܘܬܦܐ ܒܠܚܘܕܗ ܠܐ ܒܕ ܓܠܐ ܠܗ.

p_e – ܫܠܡܘܬܐ ܡܣܟܝܬܐ ܒܓܕܫܐ. ܟܡܐ ܕܪܡܬܐ ܝܠܗ (ܓܢܣܐ ܚܕ ܫܠܝܛܐ)، ܫܠܡܘܬܐ ܓܘܠܡܝܬܐ ܚܫܝܚܬܐ ܝܠܗ ܒܨܪ ܘ-κ ܙܥܘܪܐ ܝܠܗ ܝܬܝܪ. κ_max (Cohen, 1960, Equation 6) – ܩܐܦܐ ܪܡܬܐ ܡܢ ܟܠܗ ܕܡܨܝܐ ܠܡܛܝܐ ܥܡ ܣܦܪ̈ܐ ܝܗܝ̈ܒܐ ܕܠܘܚܐ؛ κ_max ܢܚܝܬܐ ܟܐܡܪܐ ܕܪܡܙ̈ܢܐ ܟܡܦܠܚܝ ܒܓܢ̈ܣܐ ܒܣܓܝܐܘ̈ܬܐ ܦܪ̈ܝܫܬܐ، ܘ"1 − κ_max ܝܠܗ ܡܢܬܐ ܕܫܠܡܘܬܐ ܕܠܐ ܡܨܝܐ ܠܡܛܝܐ ܠܗ ܡܛܠ ܣܦܪ̈ܐ" – ܪܘܫܡܐ ܕܬܚܘ̈ܡܐ ܕܓܢ̈ܣܐ ܕܠܐ ܦܪ̈ܝܫܐ ܕܡܠܦܢܘܬܐ ܡܨܝܐ ܠܚܪܦܐ ܠܗܘܢ. ASE ܘܬܚܘܡܐ ܕܬܘܟܠܢܐ – ܛܥܝܘܬܐ ܬܩܝܢܬܐ asymptotic ܠܦܘܬ Fleiss، Cohen ܘ-Everitt (1969)، ܗܝ ܫܘܝܘܬܐ ܕ-SPSS (Cohen ܒܓܢܒܗ ܝܗܒܠܗ ܓܘ 1960 ܩܘܪܒܐ ܦܫܝܛܐ ܝܬܝܪ، ܫܘܝܘܬܐ 7، ܕܟܝܗܒܐ ܛܝܡܐ ܩܠܝܠ ܪܡܐ ܝܬܝܪ). z ܘ-p ܟܒܚܪܝ ܐܢ κ ܦܪܝܫܐ ܝܠܗ ܡܢ ܣܦܝܩܘܬܐ – ܒܘܚܪܢܐ ܕ-Cohen ܒܓܢܒܗ ܩܪܐܠܗ "ܕܠܐ ܛܝܡܐ ܒܪܘܒܐ": ܕܩܐܦܐ ܪܒܬܐ ܝܠܗ ܡܢ ܣܦܝܩܘܬܐ ܠܐ ܝܠܗ ܫܘܐܠܐ؛ ܫܘܐܠܐ ܝܠܗ ܟܡܐ.

α – D_o ܝܠܗ ܠܐ ܫܠܡܘܬܐ ܡܨܥܝܬܐ ܚܙܝܬܐ ܓܘ ܚܕ̈ܝܘܬܐ، D_e – ܠܐ ܫܠܡܘܬܐ ܡܣܟܝܬܐ ܒܝܢܝ ܟܠ ܛܝ̈ܡܐ ܓܘ ܝܗܝ̈ܒܬܐ، ܘ-α = 1 − D_o/D_e. ܓܘ metric ܕܫܡܗ̈ܐ ܟܠ ܠܐ ܫܠܡܘܬܐ ܫܘܝܐ ܝܠܗ؛ ܓܘ metric ܕܛܟܣܐ ܠܐ ܫܠܡܘܬܐ ܒܝܢܝ "ܢܚܝܬܐ" ܘ"ܪܡܐ" ܩܫܝܬܐ ܝܠܗ ܝܬܝܪ ܡܢ ܒܝܢܝ "ܢܚܝܬܐ" ܘ"ܡܨܥܝܐ"؛ ܓܘ metric ܕܪܘ̈ܚܩܐ ܫܘ̈ܝܐ – ܠܦܘܬ ܡܪܒܥܐ ܕܦܘܪܫܢܐ. ܡܛܠ ܗܕܐ α ܕܣܒܠܬܐ 1-7 ܓܘ metric ܕܫܡܗ̈ܐ ܢܚܝܬܐ ܝܠܗ، ܘܒܥܕܢ̈ܐ ܢܚܝܬܐ ܣܓܝ، ܡܢ ܗܝ α ܓܘ metric ܕܪܘ̈ܚܩܐ ܫܘ̈ܝܐ: ܦܘܪܫܢܐ ܠܐ ܝܠܗ "ܛܥܝܘܬܐ" ܐܠܐ ܬܚܡܐ ܕܡܐ ܕܡܬܚܫܒ ܫܠܡܘܬܐ.

ܬܚܘܡܐ ܕܬܘܟܠܢܐ ܕ-α ܘ-q. ܠ-α ܠܝܬ ܫܘܝܘܬܐ ܕܛܥܝܘܬܐ ܬܩܝܢܬܐ؛ Krippendorff (2019، ܦܣܩܐ 11.4.2) ܟܡܚܝܪ ܒܕܘܟܬܗ bootstrap: ܟܚܫܒܚ α ܥܠ ܐܠܦ̈ܐ ܕܕܘ̈ܡܝܐ ܡܢ ܪܝܫ ܡܢ ܝܗܝ̈ܒܬܐ، ܘܦܘܠܓܐ ܕܛܝ̈ܡܐ ܕܟܫܩܠܚ ܟܝܗܒ ܬܚܘܡܐ ܕܬܘܟܠܢܐ – ܘܡܐ ܕܚܫܝܚ ܝܠܗ ܝܬܝܪ ܩܐ ܕܝܠܗ – q، ܡܣܬܒܪܢܘܬܐ ܕ-α ܠܐ ܥܒܪܐ ܬܚܘܡܐ ܕܦܣܝܩܐ ܝܠܗ. ܒܘܚܪܢܐ "ܐܢ α ܦܪܝܫܐ ܝܠܗ ܡܢ ܣܦܝܩܘܬܐ" ܕܠܐ ܣܘܟܠܐ ܝܠܗ، ܐܝܟ ܕܟܬܒ؛ ܫܘܐܠܐ ܝܠܗ ܟܡܐ ܚܢܢ ܬܟܝܠ̈ܐ ܝܘܚ ܕ-α ܠܥܠ ܡܢ .800 ܝܠܗ (ܝܢ .667). ܦܐܬܐ ܟܡܦܠܚܐ bootstrap ܕܚܕ̈ܝܘܬܐ (Efron & Tibshirani, 1993): 2,000 ܕܘ̈ܡܝܐ ܕ-N ܚܕ̈ܝܘܬܐ ܥܡ ܕܥܪܬܐ، α ܩܐ ܟܠ ܕܘܡܝܐ، ܕܪ̈ܓܐ ܡܢ ܡܐܐ 2.5 ܘ-97.5 ܩܐ ܬܚܘܡܐ، ܘܡܢܬܐ ܕܕܘ̈ܡܝܐ ܬܚܘܬ .667 ܘ-.800 ܐܝܟ q. ܬܪܝܢ ܦܘܪ̈ܫܢܐ ܡܢ ܡܐܩܪܘ KALPHA ܕ-Krippendorff: ܐܗܘ ܟܓܒܐ ܙܘܓ̈ܐ ܕܛܝ̈ܡܐ ܡܢ ܡܐܛܪܝܟܣ ܕܚܘ̈ܠܛܐ ܘܠܐ ܚܕ̈ܝܘܬܐ، ܘܡܛܠ ܗܕܐ ܬܚܘ̈ܡܐ ܕܝܠܗ ܡܨܝܐ ܕܗܘܝܐ ܩܠܝܠ ܦܪ̈ܝܫܐ؛ ܘܦܐܬܐ ܟܡܦܠܚܐ ܒܙܪܥܐ ܩܝܡܐ، ܗܟܢܐ ܕܗܢܝ ܝܗܝ̈ܒܬܐ ܟܠ ܐܝܡܢ ܟܝܗܒܝ ܗܘ ܬܚܘܡܐ – ܢܡܘܣܐ ܕܣܐܝܛ: ܚܘܫܒܢܐ ܦܣܝܩܐ (deterministic).

κ ܡܬܩܠܬܐ (Cohen, 1968؛ ܡܚܘܝܬܐ ܝܠܗ ܐܝܡܢ ܕܓܢ̈ܣܐ ܡܢܝܢܝ̈ܐ ܝܢܐ، ܬܠܬܐ ܘܠܥܠ، ܘ-metric ܠܐ ܝܠܗ ܕܫܡܗ̈ܐ) – ܩܐ ܣܒ̈ܠܬܐ ܕܛܟܣܐ: ܠܐ ܫܠܡܘܬܐ ܒܝܢܝ 1 ܘ-2 ܠܐ ܝܠܗ ܐܝܟ ܒܝܢܝ 1 ܘ-5. ܡܬܩܠܐ ܣܪܛܢܝܐ (linear) ܟܝܗܒ ܠܦܘܬ ܪܘܚܩܐ، ܡܪܒܥܐ – ܠܦܘܬ ܡܪܒܥܐ ܕܪܘܚܩܐ؛ ܡܪܒܥܬܐ ܫܘܝܐ ܝܠܗ ܠܐܣܘܪܐ ܕܚܕ̈ܕܐ ܓܘ ܓܘܕܐ (intraclass correlation) ܓܘ ܡܘܨ̈ܒܐ ܣܓܝ̈ܐܐ، ܘܡܛܠ ܗܕܐ ܐܝܗ ܕܡܬܚܙܝܐ ܣܓܝ. ܬܪܘܝܗܝ ܓܘ ܠܘܚܐ، ܡܛܠ ܕܓܘܒܝܐ ܒܝܢܝ ܐܢܝ ܦܘܣܩܢܐ ܝܠܗ ܕܒܘܨܝܐ، ܠܐ ܕܬܘܟܢܝܬܐ – ܘܟܡܘܬܢܚ ܐܝܢܝ ܓܒܝܬܐ ܝܠܗ. κ ܕ-Fleiss (Fleiss, 1971؛ ܡܚܘܝܬܐ ܝܠܗ ܡܢ ܬܠܬܐ ܪܡܙ̈ܢܐ) – ܡܬܚܬܐ ܡܩܒܠܬܐ ܕ-κ ܩܐ ܪܡܙ̈ܢܐ ܣܓܝ̈ܐܐ، ܐܦܢ ܕܒܫܪܪܐ ܡܬܚܬܐ ܝܠܗ ܕ-π ܕ-Scott: ܫܠܡܘܬܐ ܡܣܟܝܬܐ ܚܫܝܒܬܐ ܝܠܗ ܡܢ ܦܘܠܓܐ ܕܓܢ̈ܣܐ ܓܘ ܟܠܗܘܢ ܐܟܚܕܐ، ܠܐ ܓܘ ܟܠ ܪܡܙܢܐ ܒܦܪܝܫܘܬܐ. ܣܢܝܩܬܐ ܝܠܗ ܠܡܢܝܢܐ ܫܘܐ ܕܪܡܙ̈ܢܐ ܩܐ ܟܠ ܚܕܝܘܬܐ، ܘܡܛܠ ܗܕܐ ܚܕ̈ܝܘܬܐ ܥܡ ܛܝܡܐ ܚܣܝܪܐ ܟܦܠܛܝ – ܠܐ ܐܝܟ α. ܐܝܡܢ ܕܬܪܝܢ ܡܢܝ̈ܢܐ ܩܪܝܒ̈ܐ ܝܢܐ، ܐܗܐ ܣܗܕܘܬܐ ܝܠܗ ܕܚܣܝ̈ܪܐ ܠܐ ܟܫܚܠܦܝ ܨܘܪܬܐ. ܛܥܝܘܬܐ ܬܩܝܢܬܐ ܕ-Fleiss (1971) ܟܝܗܒ ܬܪܝܨܬܐ ܝܠܗ ܒܣ ܬܚܘܬ ܗܘܦܘܬܣܝܣ κ = 0 – ܛܒܬܐ ܩܐ ܒܘܚܪܢܐ z، ܠܐ ܩܐ ܬܚܘܡܐ ܕܬܘܟܠܢܐ – ܘܡܛܠ ܗܕܐ ܬܚܘܡܐ ܕܬܘܟܠܢܐ ܕ-κ_F ܓܘ ܠܘܚܐ ܐܬܐ ܡܢ ܗܘ bootstrap ܕܚܕ̈ܝܘܬܐ ܐܝܟ ܩܐ α (ܥܠ ܚܕ̈ܝܘܬܐ ܓܡܝܪ̈ܬܐ ܒܠܚܘܕ). ܩܐ κ ܡܬܩܠܬܐ ܡܚܘܝ̈ܐ ܝܢܐ ܐܦ z ܘ-p: ܒܘܚܪܢܐ κ_w = 0 ܒܛܥܝܘܬܐ ܬܩܝܢܬܐ ܬܚܘܬ H₀ ܕ-Fleiss، Cohen ܘ-Everitt (1969) – ܗܘ ܥܝܕܐ ܕܓܘܗ SPSS ܟܚܫܒ Approx. T ܓܘ ܠܘܚܐ Symmetric Measures، "ܒܛܥܝܘܬܐ ܬܩܝܢܬܐ asymptotic ܥܡ ܣܝܡܐ ܕܗܘܦܘܬܣܝܣ ܕܣܦܝܩܘܬܐ" (Pallant, 2020, Chapter 16). ܐܗܐ ܒܘܚܪܢܐ ܟܦܢܐ ܥܠ ܫܘܐܠܐ ܕܩܪܝܒܐ ܠܐ ܡܫܬܐܠܐ ܓܘ ܒܘܨܝܐ ܕܪܡܙܢܘܬܐ – ܐܢ ܫܠܡܘܬܐ ܪܒܬܐ ܝܠܗ ܡܢ ܣܦܝܩܘܬܐ – ܘܬܚܘܡܐ ܕܬܘܟܠܢܐ ܝܠܗ ܝܕܥܬܐ ܡܘܬܪܢܝܬܐ؛ ܐܝܬܘܗܝ ܓܘ ܠܘܚܐ ܩܐ ܕܡܦܩܢܐ ܦܐܐ ܠܗ ܠܕ-SPSS.

ܬܪܝܢ ܣܪ̈ܓܠܐ، ܘܡܘܕܝ ܝܠܗ ܦܘܪܫܢܐ. Landis ܘ-Koch (1977، ܦܐܬܐ 165) ܦܠܓܠܗܘܢ ܦܬܝܘܬܐ ܕ-κ ܠܫܬܐ ܕܪ̈ܓܐ – ܩܠܝܠܬܐ، ܟܪܝܗܬܐ، ܡܨܥܝܬܐ، ܪܒܬܐ، ܩܪܝܒܐ ܓܡܝܪܬܐ – ܘܡܙܝܕܠܗܘܢ ܒܗܝ ܢܫܡܐ ܕܦܘܠܓܐ "ܡܢ ܨܒܝܢܐ ܓܠܝܐܝܬ" (clearly arbitrary) ܘܥܒܝܕܐ ܝܠܗ ܩܐ ܕܗܘܐ "ܡܫܘ̈ܚܬܐ ܡܘܬܪܢ̈ܝܬܐ" ܩܐ ܡܬܠܐ ܕܓܘ ܡܐܡܪܐ ܕܝܠܗܘܢ. ܐܗܐ ܣܪܓܠܐ ܫܪܫܠܗ ܣܓܝ ܠܒܪ ܡܢ ܨܒܝܢܐ ܕܝܠܗܘܢ. Krippendorff (2019، ܦܣܩܐ 11.4.4) ܟܦܣܩ ܒܕܘܟܬܗ ܬܪܝܢ ܬܚܘ̈ܡܐ: ܠܡܣܡܟ ܒܣ ܥܠ ܡܫܬܚܠܦ̈ܢܐ ܥܡ α ܠܥܠ ܡܢ .800، ܘܠܦܠܛܐ ܦܘܣܩ̈ܢܐ ܙܗܝܪ̈ܐ ܒܠܚܘܕ ܒܝܢܝ .667 ܘ-.800 – ܘܟܡܘܕܐ ܕܐܦ .800 ܝܠܗ "ܬܩܢܐ ܢܚܝܬܐ ܒܝܫ" ܩܘܒܠ ܗܢܕܣܐ ܘܐܣܝܘܬܐ. ܦܘܪܫܢܐ ܪܒܐ ܝܠܗ: κ = .65 ܝܠܗ "ܫܠܡܘܬܐ ܪܒܬܐ" ܓܘ Landis ܘ-Koch ܘܠܐ ܡܛܝܬܐ ܠܬܚܘܡܐ ܓܘ Krippendorff. ܦܐܬܐ ܟܡܚܘܝܐ ܬܪܘܝܗܝ، ܘܒܕܘܟܬܐ ܕܦܣܩܐ ܟܡܚܝܪܐ ܫܘܐܠܐ ܕ-Krippendorff: ܡܘܕܝ ܝܠܗ ܛܝܡܐ ܕܦܘܣܩܢܐ ܛܥܝܐ ܓܘ ܐܗܐ ܒܘܨܝܐ؟

ܫܪ̈ܛܐ ܘܙܘܗܪ̈ܐ (Assumptions and warnings)

ܬܠܬܐ ܫܪ̈ܛܐ ܕ-Cohen (1960، ܦܐܬܐ 38): ܚܕ̈ܝܘܬܐ ܠܐ ܬܠܝ̈ܬܐ ܝܢܐ؛ ܓܢ̈ܣܐ ܠܐ ܬܠܝ̈ܐ ܝܢܐ، ܟܡܦܩܝ ܚܕ ܠܐܚܪܢܐ ܘܟܡܓܡܪܝ ܟܠ ܡܕܡ؛ ܘܪܡܙ̈ܢܐ ܟܦܠܚܝ ܒܚܐܪܘܬܐ. ܬܠܝܬܝܐ ܝܠܗ ܕܟܬܒܪ ܒܫܪܪܐ – ܬܪܝܢ ܪܡܙ̈ܢܐ ܕܝܬܒܝ ܐܟܚܕܐ، ܝܢ ܪܡܙܢܐ ܬܪܝܢܝܐ ܕܟܚܙܐ ܪ̈ܡܙܐ ܕܩܕܡܝܐ، ܟܒܪܝ "ܫܠܡܘܬܐ" ܕܠܐ ܝܠܗ ܫܪܝܪܘܬܐ.

ܫܪܝܪܘܬܐ ܕܝܠܢܝܘܬܐ ܝܠܗ ܕܡܫܬܚܠܦܢܐ، ܠܐ ܕܒܘܨܝܐ. Krippendorff (2019، ܦܣܩܐ 11.4.4) ܟܡܙܗܪ ܡܢ ܡܨܥܝܬܐ ܕ-α ܕܟܡܐ ܡܫܬܚܠܦ̈ܢܐ: ܡܫܬܚܠܦ̈ܢܐ ܕܛܟܣܐ ܕܟܬܒ̈ܐ (ܝܘܡܐ، ܐܘܪܟܐ) ܟܠ ܐܝܡܢ ܫܪܝܪ̈ܐ ܝܢܐ ܘܟܡܪܡܝ ܠܡܨܥܝܬܐ، ܐܠܐ ܡܫܬܚܠܦ̈ܢܐ ܕܚܫܝܚ̈ܐ ܝܢܐ – ܐܢܝ ܕܩܫܝܐ ܝܠܗ ܠܪܡܙܐ ܠܗܘܢ – ܟܡܛܫܝ ܬܚܘܬܗ. ܟܡܘܬܢܚ ܩܐ ܟܠ ܡܫܬܚܠܦܢܐ ܒܦܪܝܫܘܬܐ.

ܣܓܝܐܘܬܐ ܕܓܢ̈ܣܐ. ܐܝܡܢ ܕܚܕ ܓܢܣܐ ܟܡܬܚܙܐ ܓܘ 90% ܡܢ ܚܕ̈ܝܘܬܐ، ܐܦ ܫܠܡܘܬܐ ܓܘܠܡܝܬܐ ܕ-90% ܟܝܗܒܐ κ ܢܚܝܬܐ – ܫܠܡܘܬܐ ܥܠ ܓܢܣܐ ܕܣܓܝ ܡܬܚܙܐ "ܡܣܟܝܬܐ ܝܠܗ ܒܓܕܫܐ". ܐܗܐ ܕܝܠܢܝܘܬܐ ܬܪܝܨܬܐ ܝܠܗ ܕܡܚܘܝܢܐ (Krippendorff, 2019، ܦܣܩܐ 11.4.1: ܡܫܬܚܠܦܢܐ ܕܠܐ ܫܘܚܠܦܐ ܠܐ ܛܥܝܢ ܝܕܥܬܐ)، ܐܠܐ ܟܡܬܡܗܐ ܠܒܘܨ̈ܝܐ؛ ܐܢ ܡܫܬܚܠܦܢܐ ܩܪܝܒܐ ܩܝܡܐ ܝܠܗ، ܣܢܝܩܘܬܐ ܐܝܬ ܠܡܫܐܠ ܐܢ ܒܫܪܪܐ ܡܫܬܚܠܦܢܐ ܝܠܗ.

ܪܒܘܬܐ ܕܕܘܡܝܐ ܩܐ ܒܘܚܪܢܐ ܕܫܪܝܪܘܬܐ. ܬܚܘܡܐ ܕܬܘܟܠܢܐ ܕ-κ ܟܩܛܢ ܥܡ N؛ Krippendorff (2019، ܦܣܩܐ 11.4.3) ܟܡܚܝܪ ܠܡܦܣܩ ܡܢܝܢܐ ܕܚܕ̈ܝܘܬܐ ܗܟܢܐ ܕܓܢܣܐ ܕܡܬܚܙܐ ܒܨܪ ܡܢ ܟܠܗ ܡܬܚܙܐ ܣܦܩ ܙܒ̈ܢܐ ܩܐ ܕܠܐ ܫܠܡܘܬܐ ܥܠܘܗܝ ܡܨܝܐ ܕܡܬܓܠܝܐ – ܕܘܡܝܐ ܕ-30 ܚܕ̈ܝܘܬܐ ܥܡ ܓܢܣܐ ܕܡܬܚܙܐ ܬܪܝܢ ܙܒ̈ܢܐ ܠܐ ܟܒܚܪ ܐܗܐ ܓܢܣܐ.

ܚܣܝ̈ܪܐ. κ ܚܫܝܒܬܐ ܝܠܗ ܒܣ ܥܠ ܚܕ̈ܝܘܬܐ ܕܬܪܝܢ ܪܡܙ̈ܢܐ ܓܘ ܙܘܓܐ ܪܡܙܠܗܘܢ (ܫܩܠܬܐ ܙܘܓܢܝܬܐ – ܐܝܟ ܓܘ SPSS)؛ α ܟܡܦܠܚܐ ܒܟܠ ܛܝܡܐ ܕܐܝܬ ܠܗ ܠܐ ܒܨܪ ܡܢ ܚܕ ܒܪ ܙܘܓܐ ܓܘ ܚܕܝܘܬܐ ܕܝܠܗ، ܘܡܛܠ ܗܕܐ N ܕ-κ ܘ"ܚܕ̈ܝܘܬܐ ܕܡܨܝܐ ܠܦܘܚܡܐ" ܕ-α ܡܨܝܐ ܕܗܘܝܐ ܦܪ̈ܝܫܐ ܓܘ ܗܢܝ ܝܗܝ̈ܒܬܐ. ܬܪܘܝܗܝ ܡܘܬܢ̈ܐ ܝܢܐ.

ܡܘܕܝ ܠܐ ܟܚܫܒܐ ܐܗܐ ܦܐܬܐ. α ܩܐ ܦܘܠܓܐ ܠܚܕ̈ܝܘܬܐ (unitizing; Krippendorff, 2019, Section 11.6) – ܐܝܡܢ ܕܪܡܙ̈ܢܐ ܐܦ ܟܦܣܩܝ ܐܝܟܐ ܚܕܝܘܬܐ ܟܫܪܝܐ ܘܟܫܠܡܐ؛ ܛܥܝܘܬܐ ܬܩܝܢܬܐ asymptotic ܩܐ κ ܕ-Fleiss (Fleiss, 1971، ܟܝܗܒ ܠܗ ܒܣ ܬܚܘܬ H₀؛ ܬܚܘܡܐ ܕܬܘܟܠܢܐ – ܒ-bootstrap)؛ ܘܬܚܘܡܐ ܕܬܘܟܠܢܐ ܠܦܘܬ algorithm ܦܪܝܫܐ ܕ-KALPHA (ܚܙܝܡܘܢ ܠܥܠ).

ܘܕܟܝ ܟܥܒܕܚ ܐܗܐ ܒ-SPSS؟ (And how do you do it in SPSS?)

ܡܥܝܪܢܘܬܐ ܕܢܘܣ̈ܚܐ. ܦܘܠܚܢܐ ܩܝܡܐ ܝܠܗ ܡܢ ܢܘܣ̈ܚܐ ܩܕܡܝ̈ܐ ܣܓܝ؛ ܫܡܗ̈ܐ ܕܪ̈ܫܝܡܬܐ ܠܬܚܬ ܠܦܘܬ SPSS 29. Cohen (1960) ܘ-Landis ܘ-Koch (1977) ܩܕܡ SPSS ܗܘܘ؛ ܫܘܝܘܬܐ ܕܛܥܝܘܬܐ ܬܩܝܢܬܐ ܓܘ SPSS ܕ-Fleiss، Cohen ܘ-Everitt (1969) ܝܠܗ، ܘܐܗܝ ܝܠܗ ܕܦܐܬܐ ܟܚܫܒܐ.

κ ܕ-Cohen: Analyze › Descriptive Statistics › Crosstabs… – ܪܡܙܢܐ ܩܕܡܝܐ ܠ-Row(s)، ܬܪܝܢܝܐ ܠ-Column(s)، ܘܓܘ button Statistics… ܟܪܫܡܚ Kappa. ܣܝܢܬܐܟܣ (syntax):

CROSSTABS
  /TABLES=coder_A BY coder_B
  /STATISTICS=KAPPA
  /CELLS=COUNT.

ܓܘ ܠܘܚܐ Symmetric Measures: Value = κ، Asymp. Std. Error = ASE ("ܕܠܐ ܣܝܡܐ ܕܗܘܦܘܬܣܝܣ ܕܣܦܝܩܘܬܐ")، Approx. T = z ("ܒܛܥܝܘܬܐ ܬܩܝܢܬܐ asymptotic ܥܡ ܣܝܡܐ ܕܗܘܦܘܬܣܝܣ ܕܣܦܝܩܘܬܐ" – ܬܪܬܝܢ ܡܥܝܪ̈ܢܘܬܐ ܕܬܚܬ ܠܘܚܐ؛ Pallant, 2020، ܨܚܚܐ 16)، Approx. Sig. = p. ܠܘܚܐ Crosstabulation ܝܠܗ ܡܐܛܪܝܟܣ ܕܫܠܡܘܬܐ. ܚܙܝܡܘܢ: SPSS ܟܚܫܒ κ ܒܣ ܐܝܡܢ ܕܬܪܝܢ ܡܫܬܚܠܦ̈ܢܐ ܟܡܦܠܚܝ ܒܗܢܝ ܓܢ̈ܣܐ؛ ܐܢ ܚܕ ܪܡܙܢܐ ܡܢ ܡܬܘܡ ܠܐ ܡܘܦܠܚܠܗ ܒܓܢܣܐ ܦܪܝܫܐ، ܠܘܚܐ ܠܐ ܝܠܗ ܡܪܒܥܐ ܘ-SPSS ܠܐ ܒܕ ܛܒܥ ܩܐܦܐ – ܦܐܬܐ ܟܚܫܒܐ ܠܗ ܥܠ ܟܠ ܓܢ̈ܣܐ ܡܫܘܬܦ̈ܐ، ܐܝܟ ܕ-Cohen ܬܚܡܠܗ.

κ ܡܬܩܠܬܐ: ܡܢ ܢܘܣܚܐ 27 – Analyze › Scale › Weighted Kappa…، ܥܡ ܡܬܩܠܐ ܣܪܛܢܝܐ (ܓܘܒܝܐ ܡܩܕܡܐ) ܝܢ ܡܪܒܥܐ ܘܬܚܘܡܐ ܕܬܘܟܠܢܐ asymptotic؛ ܣܝܢܬܐܟܣ – ܓܘ button Paste ܕ-dialog. κ ܕ-Fleiss: ܠܝܬ ܓܘ SPSS؛ ܚܙܝܡܘܢ R ܠܬܚܬ.

α ܕ-Krippendorff: ܠܝܬ ܦܘܩܕܢܐ ܒܢܝܐ. ܡܐܩܪܘ KALPHA ܕ-Hayes ܘ-Krippendorff (2007) – ܟܡܪܗܛܚ ܚܕ ܙܒܢܐ ܦܐܝܠ ܕܡܐܩܪܘ، ܘܒܬܪ:

KALPHA judges = coder_A coder_B coder_C
  /level = 1 /detail = 0 /boot = 2000.

level: 1 ܕܫܡܗ̈ܐ، 2 ܕܛܟܣܐ، 3 ܕܪܘ̈ܚܩܐ ܫܘ̈ܝܐ، 4 ܕܦܘܚܡܐ؛ boot – ܡܢܝܢܐ ܕܕܘ̈ܡܝܐ ܕ-bootstrap ܩܐ ܬܚܘܡܐ ܕܬܘܟܠܢܐ (0 – ܕܠܐ)؛ algorithm ܕ-KALPHA ܟܓܒܐ ܙܘܓ̈ܐ ܕܛܝ̈ܡܐ ܘܠܐ ܚܕ̈ܝܘܬܐ، ܘܡܛܠ ܗܕܐ ܬܚܘ̈ܡܐ ܕܝܠܗ ܩܪܝܒ̈ܐ ܝܢܐ ܠܐܢܝ ܕܓܘ ܦܐܬܐ ܐܠܐ ܠܐ ܚܕ. Button "ܡܚܬܬܐ ܕܣܝܢܬܐܟܣ SPSS" ܓܘ ܡܦܩܢܐ ܟܒܪܝܐ ܬܪܬܝܢ ܡܢ̈ܘܬܐ ܥܠ ܝܗܝ̈ܒܬܐ ܕܛܥܝ̈ܢܬܐ ܝܢܐ، ܥܡ ܦܘܩ̈ܕܢܐ ܕܟܬܒܐ ܕܪ̈ܡܙܐ ܐܢ ܗܘܐ ܡܦܫܩܢܐ.

ܘܕܟܝ ܟܥܒܕܚ ܐܗܐ ܒ-R؟ (And how do you do it in R?)

ܚܙܡܐ (package) irr ܟܚܫܒ ܬܪܝܢ ܡܢܝ̈ܢܐ ܡܫܘܬܦ̈ܐ: kappa2 – κ ܕ-Cohen ܩܐ ܙܘܓܐ (ܚܕ̈ܝܘܬܐ ܓܡܝܪ̈ܬܐ ܒܠܚܘܕ، ܐܝܟ SPSS ܘܐܝܟ ܦܐܬܐ)؛ kripp.alpha – α ܕ-Krippendorff ܩܐ ܟܠ ܡܢܝܢܐ ܕܪܡܙ̈ܢܐ، ܥܡ NA ܐܝܟ ܚܣܝܪܐ، ܓܘ ܐܪܒܥܐ metrics. ܣܩܪܝܦܬ ܪܗܛܐ ܝܠܗ ܥܠ ܦܐܝܠ ܕܡܬܠܐ ܘܝܗܒܠܗ ܛܝ̈ܡܐ ܕܠܘܚܐ (ܓܘ ܡܥܝܪ̈ܢܘܬܐ). ܢܡܘܣܐ ܕܣܐܝܛ ܩܐ ܣܘܪܝܬ ܓܘ R: ܦܐܝܠ ܢܛܝܪܐ ܝܠܗ ܒ-UTF-8، ܟܠ ܣܕܪܐ ܕܡܥܝܪܢܘܬܐ ܟܫܪܐ ܒܡܠܬܐ ܕܣܘܪܝܬ؛ ܘܝܬܝܪ – ܫܡܗ̈ܐ ܕܓܠܝܘ̈ܢܐ ܘܕܡܫܬܚܠܦ̈ܢܐ ܒܥܒܪܝܐ ܣܢܝܩ̈ܐ ܝܢܐ ܠܐܬܪܐ UTF-8 (ܓܘ Windows: Sys.setlocale('LC_ALL', 'Hebrew') ܝܢ ܢܘܣܚܐ ܕ-R 4.2 ܘܠܥܠ، ܕܓܘܗ UTF-8 ܝܠܗ ܓܘܒܝܐ ܡܩܕܡܐ).

# ܫܪܝܪܘܬܐ ܒܝܢܝ ܪܡܙ̈ܢܐ – ܦܪܘܦܣܘܪ ܣܛܐܛ. ܐܗܐ ܦܐܝܠ ܢܛܝܪܐ ܝܠܗ ܒ-UTF-8؛ ܟܠ ܣܕܪܐ ܕܡܥܝܪܢܘܬܐ ܟܫܪܐ ܒܡܠܬܐ ܕܣܘܪܝܬ
library(readxl)   # ܩܪܝܢܐ ܕܐܩܣܠ
library(irr)      # ܚܙܡܐ kappa2, kripp.alpha

# ܛܥܢܐ: ܛܘܦܣܐ ܬܩܝܢܐ – ܣܕܪܐ = ܚܕܝܘܬܐ ܕܪܡܙܢܘܬܐ، ܥܡܘܕܐ = ܪܡܙܢܐ؛ ܓܠܝܘܢܐ "נתונים"
d <- read_excel("C:/Rdata/DATA_reliability_example_heb.xlsx", sheet = "נתונים")

# ܩܐܦܐ ܕ-Cohen ܩܐ ܙܘܓܐ ܕܪܡܙ̈ܢܐ (ܚܕ̈ܝܘܬܐ ܓܡܝܪ̈ܬܐ ܒܠܚܘܕ، ܐܝܟ ܓܘ SPSS)
k <- kappa2(d[, c("מקודד א", "מקודד ב")])
k$value        # κ = .653
k$subjects     # N = 139
k$statistic    # z = 15.14 (ܒܘܚܪܢܐ κ = 0)

# ܐܠܦܐ ܕ-Krippendorff – ܟܠ ܪܡܙ̈ܢܐ، ܚܣܝ̈ܪܐ ܦܣܝܚ̈ܐ ܝܢܐ؛ ܣܕܪ̈ܐ = ܪܡܙ̈ܢܐ، ܥܡܘ̈ܕܐ = ܚܕ̈ܝܘܬܐ
m <- t(as.matrix(d[, c("מקודד א", "מקודד ב", "מקודד ג")]))
kripp.alpha(m, method = "nominal")$value    # α = .629
kripp.alpha(m, method = "ordinal")$value    # α = .659
kripp.alpha(m, method = "interval")$value   # α = .654

# ܩܐܦܐ ܡܬܩܠܬܐ ܩܐ ܙܘܓܐ – ܡܬܩܠܐ ܣܪܛܢܝܐ ("equal") ܝܢ ܡܪܒܥܐ ("squared")؛ ܓܢ̈ܣܐ ܣܢܝܩ̈ܐ ܝܢܐ ܕܗܘܝܐ ܡܛܟ̈ܣܐ
kappa2(d[, c("מקודד א", "מקודד ב")], weight = "equal")$value     # ܩܐܦܐ κw ܣܪܛܢܝܬܐ = .662
kappa2(d[, c("מקודד א", "מקודד ב")], weight = "squared")$value   # ܩܐܦܐ κw ܡܪܒܥܬܐ = .664

# ܩܐܦܐ ܕ-Fleiss – ܬܠܬܐ ܪܡܙ̈ܢܐ ܘܠܥܠ، ܒܣ ܚܕ̈ܝܘܬܐ ܕܪܡܝܙ̈ܐ ܝܢܐ ܒܝܕ ܟܠܗܘܢ
f <- kappam.fleiss(d[complete.cases(d[, 2:4]), c("מקודד א", "מקודד ב", "מקודד ג")])
f$value        # ܩܐܦܐ κF = .636
f$subjects     # N = 134
f$statistic    # z = 24.97

kappa2 ܟܛܒܥ ܐܦ z ܘ-p ܐܠܐ ܠܐ ܛܥܝܘܬܐ ܬܩܝܢܬܐ asymptotic ܘܠܐ κ_max؛ kripp.alpha ܟܛܒܥ α ܒܠܚܘܕ، ܕܠܐ ܬܚܘܡܐ ܕܬܘܟܠܢܐ. ܡܢܝ̈ܢܐ ܕܓܘ ܠܘܚܐ ܕܦܐܬܐ – ASE، ܬܚܘܡܐ ܕܬܘܟܠܢܐ، κ_max ܘܬܚܘܡܐ ܕ-bootstrap ܕ-α – ܟܫܩܠܚ ܡܢ SPSS ܝܢ ܡܢ ܦܐܬܐ.

ܬܘܢܝܐ ܕܡܫܟܚ̈ܢܘܬܐ (reporting the results)

ܠܦܘܬ APA (2020، ܦܣܩܐ 6.36) κ ܘ-α – ܡܢܝ̈ܢܐ ܡܫܘܬܦ̈ܐ ܕܠܐ ܥܒܪܝ ܥܠ 1 – ܟܬܝܒ̈ܐ ܝܢܐ ܕܠܐ ܣܦܝܩܘܬܐ ܩܕܡ ܢܘܩܙܐ ܘܒܬܪܝܢ ܡܢܝ̈ܢܐ ܒܬܪ ܢܘܩܙܐ؛ ܬܚܘܡܐ ܕܬܘܟܠܢܐ ܠܦܘܬ ܦܣܩܐ 6.43. ܢܘܣܚܐ ܕܦܐܬܐ ܟܡܦܩܐ: "ܫܪܝܪܘܬܐ ܕܪܡܙܢܘܬܐ ܒܝܢܝ ܬܪܝܢ ܪܡܙ̈ܢܐ ܒܚܝܪܬܐ ܝܠܗ ܥܠ N = 139 ܚܕ̈ܝܘܬܐ. ܫܠܡܘܬܐ ܓܘܠܡܝܬܐ ܗܘܬ 72.7%؛ ܩܐܦܐ ܕ-Cohen κ = .65, 95% CI [.56, .75]؛ ܐܠܦܐ ܕ-Krippendorff (ܕܫܡܗ̈ܐ) α = .65". ܟܡܘܬܢܚ N، ܫܠܡܘܬܐ ܓܘܠܡܝܬܐ (Krippendorff ܟܡܥܕܠ ܠܗ، ܐܠܐ ܐܝܗ ܟܝܗܒܐ ܠܩܪܘܝܐ ܡܫܘܚܬܐ)، ܡܢܝܢܐ ܡܫܘܬܦܐ ܥܡ ܬܚܘܡܐ ܕܬܘܟܠܢܐ، ܘ-metric ܕ-α – ܕܠܐ metric، α ܠܐ ܡܨܝܐ ܠܦܘܫܩܐ. ܣܪܓܠܐ ܕܦܘܫܩܐ ܕܓܒܝܠܘܟܘܢ – Landis ܘ-Koch ܝܢ Krippendorff – ܟܡܥܗܕܚ ܒܫܡܐ، ܡܛܠ ܕܐܢܝ ܠܐ ܫܠܝܡ̈ܐ ܝܢܐ.

ܛܥܝ̈ܘܬܐ ܕܡܬܚܙܝܢ ܣܓܝ (Common mistakes)

ܬܘܢܝܐ ܕܫܠܡܘܬܐ ܓܘܠܡܝܬܐ ܒܠܚܘܕ. 85% ܫܠܡܘܬܐ ܥܠ ܡܫܬܚܠܦܢܐ ܥܡ ܓܢܣܐ ܫܠܝܛܐ ܡܨܝܐ ܕܗܘܝܐ κ = .20. Krippendorff (2019، ܦܣܩܐ 11.5) ܟܡܦܫܩ ܩܐ ܡܘܕܝ "%-agreement" ܠܐ ܡܨܝܐ ܠܦܘܫܩܐ – ܘܩܐ ܡܘܕܝ ܐܝܗ ܟܡܬܚܙܝܐ ܐܦ ܗܫܐ.

ܚܘܫܒܢܐ ܕܫܪܝܪܘܬܐ ܥܠ ܚܕ̈ܝܘܬܐ ܕܪܡܙ̈ܢܐ ܡܡܠܠܠܗܘܢ ܥܠܝܗܝ. ܫܠܡܘܬܐ ܕܡܛܝܬܐ ܝܠܗ ܒܡܡܠܠܐ ܫܠܡܘܬܐ ܕܟܢܫܐ ܝܠܗ (consensus)، ܠܐ ܫܪܝܪܘܬܐ. ܟܡܫܚܚ ܩܕܡ ܡܡܠܠܐ، ܥܠ ܪܡܙܢܘܬܐ ܕܚܐܪܘܬܐ.

ܡܨܥܝܬܐ ܕܫܪܝܪ̈ܘܬܐ ܕܡܫܬܚܠܦ̈ܢܐ ܦܪ̈ܝܫܐ – ܚܙܝܡܘܢ ܙܘܗܪܐ ܓܘ ܦܣܩܐ ܕܫܪ̈ܛܐ.

ܓܘܒܝܐ ܕ-metric ܠܦܘܬ ܡܢܝܢܐ ܕܟܦܠܛ. α ܓܘ metric ܕܪܘ̈ܚܩܐ ܫܘ̈ܝܐ ܩܪܝܒܐ ܟܠ ܐܝܡܢ ܪܡܬܐ ܝܠܗ ܡܢ α ܕܫܡܗ̈ܐ ܥܠ ܗܢܝ ܝܗܝ̈ܒܬܐ. Metric ܦܣܝܩܐ ܝܠܗ ܠܦܘܬ ܟܝܢܐ ܕܡܫܬܚܠܦܢܐ – ܐܢ ܦܘܪܫܢܐ ܒܝܢܝ 1 ܘ-5 ܒܫܪܪܐ ܪܒܐ ܝܠܗ ܡܢ ܦܘܪܫܢܐ ܒܝܢܝ 1 ܘ-2 – ܘܟܬܝܒܐ ܝܠܗ ܩܕܡ ܚܘܫܒܢܐ.

ܡܝܬܝܐ ܕ"ܩܠܝܠܬܐ / ܪܒܬܐ / ܩܪܝܒܐ ܓܡܝܪܬܐ" ܐܝܟ ܬܩܢܐ. ܐܢܝ ܡ̈ܠܐ ܕ-Landis ܘ-Koch ܝܢܐ ܩܐ ܚܕ ܡܬܠܐ، ܥܡ ܣܝܓܐ ܓܠܝܐ. ܡܨܝܐ ܠܡܦܠܚ ܒܓܘܝܗܝ؛ ܠܐ ܣܢܝܩ ܠܡܚܘܝܐ ܠܗܘܢ ܐܝܟ ܕܦܣܝܩ̈ܐ ܝܢܐ ܒܝܕ ܢܫ.

ܟܠܝܐ ܐܝܡܢ ܕܡܢܝܢܐ ܡܫܘܬܦܐ ܢܚܝܬܐ ܝܠܗ. ܡܢܝܢܐ ܡܫܘܬܦܐ ܢܚܝܬܐ ܫܘܪܝܐ ܝܠܗ ܕܥܒܕܐ، ܠܐ ܫܘܠܡܐ ܕܝܠܗ: ܡܐܛܪܝܟܣ ܕܫܠܡܘܬܐ ܟܡܚܘܐ ܐܝܢܝ ܓܢ̈ܣܐ ܟܡܬܚܠܛܝ، κ_max ܟܡܚܘܐ ܐܢ ܚܕ ܪܡܙܢܐ ܡܨܠܐ ܠܓܢܣܐ ܦܪܝܫܐ، ܘܙܘܓܐ ܟܪܝܗܐ ܒܝܢܝ ܬܠܬܐ ܟܡܚܘܐ ܡܢܘ ܟܦܫܩ ܒܐܘܪܚܐ ܐܚܪܬܐ. ܟܬܪܨܚ ܟܬܒܐ ܕܪ̈ܡܙܐ، ܟܡܠܦܚ، ܘܟܡܫܚܚ ܬܘܒ ܥܠ ܕܘܡܝܐ ܚܕܬܐ.

ܡܒܘ̈ܥܐ

American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.). https://doi.org/10.1037/0000165-000

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46. https://doi.org/10.1177/001316446002000104

Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213-220. https://doi.org/10.1037/h0026256

Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. Chapman & Hall. https://doi.org/10.1007/978-1-4899-4541-9

Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378-382. https://doi.org/10.1037/h0031619

Fleiss, J. L., Cohen, J., & Everitt, B. S. (1969). Large sample standard errors of kappa and weighted kappa. Psychological Bulletin, 72(5), 323-327. https://doi.org/10.1037/h0028106

Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1(1), 77-89. https://doi.org/10.1080/19312450709336664

Krippendorff, K. (2019). Content analysis: An introduction to its methodology (4th ed.). SAGE.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174. https://doi.org/10.2307/2529310

Pallant, J. (2020). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (7th ed.). Routledge.

Scott, W. A. (1955). Reliability of content analysis: The case of nominal scale coding. Public Opinion Quarterly, 19(3), 321-325. https://doi.org/10.1086/266577