Study Description

PHASE I – Determine the validity of the vulvar acute genital injury scale (VAGIS) using a modified Delphi method in a national panel of sexual assault forensic experts.

Scale Development (De Vellis, 20??)

  • Step 1 – Determine clearly what is it you want to measure – this study seeks to measure AGI after SA
  • Step 2 – Generate an item pool – the item pool will be generated through a rigorous review of literature and the investigators empirical clinical experience
  • Step 3 – Determine the format for measurement – the format for measurement will be in the form of the Vulvar Acute Genital Injury Score (VAGIS)
  • Step 4 – Have the initial item pool reviewed by experts – The investigator of the proposed study will reduce the item pool before initial review by experts. The reduced item pool will be evaluated for content validity (described later) by assessing relevance, clarity and appropriateness by an expert Delphi panel (described later).
  • Step 5 – Consider inclusion of validation items – the additional items to be validated by the expert Delphi panel will be the theoretical framework, the measurement framework, the operational definitions, scoring, format, and overall ability of the VAGIS is at measuring AGI after SA.
  • Step 6 – Administer items to a development sample – the VAGIS will be validated using content experts as described in Step 4 and Step 5.
  • Step 7 -Evaluate the items – Items on the VAGIS will be evaluated using content experts as described in Step 4 and Step 5.
  • Step 8 -Optimize scale length – The length of the VAGIS will be optimized using content experts as described in Step 4 and Step 5.


PHASE II – Validate a set of digital images for each item on the VAGIS by a panel of experienced sexual assault forensic examiners.

Consent for Use of Digital Images

CalEMA 923 requires that all victims requesting a forensic exam for the purposes of obtaining evidence for a SA acknowledge in writing that they understand that digital images may be taken during the exam and saved for purposes of:

  • use as evidence in court
  • as a means of memory recall by the examiner
  • peer review by individual groups
  • Research (the signed consent pertains to this research- improve wording)

A private forensic nurse company that contracts RN’s and NP’s to perform SA exams has agreed to using their database of digital images for the purpose of this study.

Image Validation Studies

Harris, Bates-Jensen, Parslow, Raizman, Singh & Ketchen (2010)

  • Validate a set of digital images that would reflect 11 of the 13 wound characteristics on the Bates-Jensen Wound Assessment Tool (BWAT)
  • Phases I – 75 digitalized images preselected by Harris
  • Phase II with 15 wound ostomy nurses
  • Consensus was set to 75%
  • At least one digital image representing all 65 wound characteristics

Werschler, Fagien, Thomas, Paradkar-Mitragotri, Rotunda & Beddingfield (2015 )

  • Validate a set of digital images with the Allergan Lip Fullness Scale (LFS)
  • Phase I – 200 photographs of various skin colorations were taken and evaluated for photographic quality by the PI; 95 sets were chosen
  • Phase II –
  • No consensus set
  • 12 sets of digital images remained and were validated by the panel

Phase II – Part A

The participants shall be three very experienced SA forensic examiners who work at the same Forensic Nurse Company (FNC) will convene in a one time focus group to compare decisions, discuss, remove, and reassign images to different categories as they see fit. Very experienced -performed either 1000+ SA forensic examinations, or performed 100+ cases and peer reviewed 1000+ cases.

Examiners will be asked 3 quality questions:

  1. Are you able to identify tissue landmarks within the vulva?
  2. Is there sufficient lighting within the digital image?
  3. Is the digital image in focus?

Image Selection Process:

  • Only those images where 100% of the examiners agree as ‘excellent’ in all three qualities will stay in the VAGIS digital image set
  • Images judged ‘acceptable’ will be kept for back-up by the researcher. Images where not all structures are easily identifiable, where lighting is either insufficient or overexposed and/or those where sharp borders are not present, will be discarded
  • If there is an item on the VAGIS that cannot be represented by a digital image judged as ‘excellent’, then an image judged as ‘acceptable’ will be substituted
  • If no images available in either category, then the item will be labeled as ‘unable to capture on digital image’ on the VAGIS

Phase II – Part B

The participants shall be 15-20 very experienced (defined above) SA forensic examiners known to the researcher & snowball (????)

Method of validating digital image set

  • Research Electronic Data Capture (REDCAP)-a secure web application for building and managing online surveys and databases
    • A random four-digit number and sent an online link to open the initial survey site
    • Participants will attest to the inclusion criteria and informed consent will be given via electronic signature
  • Consensus rounds – 80% agreement for the digital image to be considered ‘acceptable as representing the item’
  • Items not reaching 80 % agreement in Round 1 will be reconsidered for another category in Round 2 based on panel feedback using content analysis.
  • Any item not reaching 80% agreement in Round II will be discarded.
  • If all items meet reach 80% consensus in Round I then Phase II will be completed.


PHASE III – Determine the reliability of lay forensic examiners at using the VAGIS with a validated set of specific images

Inter-rater reliability is a measure of how consistent two different raters (observers) are at assigning a score on the same test on the same occasion.

Intra-rater reliability is the measure of how consistent one rater is at scoring a test on two different occasions.

Reliabilty studies used as a design guide:

Starling, Frasier, Jarvis & Mc Donald (2013)

  • 12 raters (MD’s)
  • 33 cases of prepubescent cases 2-9 images
  • Rating options – ‘positive’ ‘negative’ ‘indeterminate’
  • Fleiss’ Kappa 0.623, no raw agreement reported
  • Limitations – ‘indeterminate’, ‘poor image quality’

Sachs, Benson, Schriger & Wheeler (2011)

  • 8 raters (convenience sample of 4 RN’s, 2 NP’s, 2 MD’s)
  • 50 consecutive cases of 2-4 images (TEARS)
  • Rating options -‘Yes’, ‘No’ & ‘Unsure’
  • Not every case was rated by every rater
  • Percent agreement 82% & Fleiss’ Kappa of 0.66
  • Limitations – ‘unsure’ vs. ‘non-specific’; ‘poor image quality’

Round 1 (Inter-rater reliability)

  • Using RedCAP (as described in Phase II), each of the 30-40 digital images validated in Phase II will be given a random number and randomized. All raters will rate all cases in the same order.
  • Data to be analyzed using raw agreement and Fleiss’ Kappa

Round 2 (Intra-rater reliability)

  • A 2nd data collection episode will begin 14- 20 days from completion of Round I
  • Images will be re-randomized (same order for all participants)
  • Same 30-40 cases from Round I
  • Data to be analyzed using raw agreement and Fleiss’ Kappa