This paper describes statistical analyses conducted to investigate reliability and validity levels for the Traumatic Event Sequelae Inventory (TESI™) as well as procedures used to standardize TESI Total Scores. It begins with a basic description of the sample used in all analyses. In this context, an overview of differential response patterns found among various sub-groups is provided. This section is followed by a description of the instrument's reliability levels. Subsequently, TESI's validity levels are addressed. The discussion then turns to methods used to standardize TESI Total Scores. In conclusion the paper recommends additional techniques and approaches which have the potential to provide further evidence of the instruments overall level of usefulness in the clinical setting.

Sample Description and TESI Performance

The analyzed sample consisted of 1,922 individuals between the ages of 10 and 83. Seventy-nine percent (1518) were undergoing assessment for PTSD in one of 37 clinical settings located in the eastern United States, primarily in New York, New Jersey, Pennsylvania, and Florida. All had experienced an extreme and potentially traumatic stressor. The remaining 21% (404) came from the non-clinical population. Most were military personnel stationed in San Diego, California.⁽ⁱ⁾

The sample's mean age was 34.6 years. Education levels ranged from 0 to 25 years with a mean of 12.47. It included 1000 (52%) males and 918 (47.8%) females An additional 4 participants did not report their gender. The sample further included 749 (39%) African Americans, 601 (31%) Latinos, 445 (23%) non-Latino Whites and 124 (7%) individuals who reported their race or ethnicity as "other." While this ethnic composition does not reflect percentages found in the current U.S. population, over-sampling of African Americans and Latinos allowed for the usefulness of TESI with those two groups to be substantially investigated. Given our increasingly diverse population and the inadequacy with which psychological testing has historically addressed specific circumstances relevant to so-called "minority" groups, such a focus appears important.

A check of sample equivalence across gender, ethnicity, and clinical/non-clinical groups was conducted in order to identify demographic variables which might confound later analyses.⁽ⁱⁱ⁾ No significant differences in age, education, drug use, comorbidity, or suicidal/homicidal ideation were noted between men and women.

Significant differences between gender groups were found on employment [Chi-square (3, N=1810)=11.1, p=.01] and marital status [Chi-square (5, N=1846)=56.83, p<.0001]. More men than women were single and employed while more women than men were widowed. In addition, among those reporting alcohol use, men tended to indicate that they had done so for a significantly longer time (Mean=12.22 years) than women (Mean=6.42 years) [F(1,45)=4.3, p=.04]. Finally, the ethnic composition of the sample was not equivalent across gender groups [Chi-square (3, N=1915)=33.33, p<.0001]. The sample consisted of more White females than males. In addition, more males than females identified as Latino.

No significant differences between ethnic groups were noted on drug and alcohol use, comorbidity, or suicidal/homicidal ideation. Significant differences were found between ethnic groups on age [F(3,1884)=3.73, p=.01] and education [F(3,1738)=28.44, p<.0001]. Latinos in the sample represented the youngest group (Mean=33.94 years) with the lowest education levels (Mean=11.58 years) while non-Latino Whites were, on average, oldest (Mean=36.34 years) and most educated (Mean=13.32 years). In addition, African Americans and Latinos were unemployed at greater rates than non-Latino Whites [Chi-square (9, N=1811)=26.5, p=.002]. In terms of marital status, more African Americans were single than Latinos or non-Latino Whites [Chi-square(15, N=1847)=77.75, p<.0001].

Finally, no significant differences between the clinical and non-clinical sample were noted on age, gender, marital status, employment status, alcohol use, or substance use. Persons in the clinical sample were, on average, less educated (Mean=13.13 years) than those in the non-clinical sample (Mean=13.13 years) [F(1,1740)=23.1, p<.0001].

In order to determine if basic demographic sample differences were important in considering TESI response levels, bivariate correlations between age, education, and TESI Total Scores were computed. In addition, Analysis of Variance was used to check for differences in TESI Total Scores attributable to employment and marital status. No significant differences between the TESI Total Scores attributable to marital or employment status were found. While both the correlation between TESI Total Scores and age (r=.08, p=<.0001) as well as education (r=.-05, p=.03) were significantly different from zero, they were, in practical terms, quite small. Their significance is thus more likely the result of a very large sample size than any real-world relationship of substantial proportions or concern. Age, education, and marital status, and employment status thus did not appear to represent potential confounds in subsequent analyses.

To further explore TESI response patterns, a factorial ANOVA comparing TESI Total Score means between gender, ethnic, and clinical status groups was calculated. As was expected, results indicated that the average number of TESI items endorsed was significantly higher for the clinical (Mean=14.6) than non-clinical (Mean=5.28) group [F(1,1892)=305, p<0001]. Gender and ethnicity main effects, as well as all interactions were not significant in this analysis. Despite this lack of significant gender and ethnicity effects it was seen as important to fully investigate possible difference in response patterns among ethnic groups as well as men and women by considering the clinical and non-clinical sample separately. No significant differences in average TESI total item endorsement rates were found between ethnic and gender groups in the non-clinical sample. Significant differences were, however, found in the clinical sample.

On average, women in the clinical sample (Mean=15.51) endorsed significantly greater number of items on the TESI than their male counterparts (Mean=13.75) [F(1,1500)=4.09, p=.04]. In addition there were significant difference in the average number of TESI item endorsed by clinical-sample ethnic groups [F(3,1500)=3.87, p=.009]. African Americans tended to have the lowest (Mean=13.71), while non-Latino Whites tended to have the highest scores (Mean=16.1). Latinos' average score (Mean=14.54) fell between that of African Americans and non-Latino Whites. Pair-wise comparisons revealed that all of these differences were significant at the p<.05 level.

In summary, the only variable accounting for significant differences in TESI total scores when considering the whole sample was clinical status. More detailed analyses revealed that, among respondents who had experienced significant stressors, women and non-Latino Whites reported the highest scores. African Americans tended to have the lowest scores, and Latinos' average scores fell between those of the other two ethnic groups. These differences can not be readily attributed to other demographic characteristics noted in the sample. The results are consistent with literature indicating that men sometimes under-report symptoms because they do not wish to appear "weak" and non-masculine. It is also noteworthy that results support the notion that TESI, unlike most other psychometric instruments, does not inherently "over-pathologize" African Americans and Latinos. Rather, they are consistent with previous literature reporting that African Americans and Latinos tend to report fewer symptoms than their White counterparts when they are asked directly and without attempts at subtlety and deception.

Finally, results suggest that, in addition to considering the sample as a whole, norms and reliability analyses should be calculated separately for gender and ethnic groups. This is particularly important for the clinical sample. Given that the sample was one of convenience, it also appeared advantageous to so for the non-clinical group (see endnote ⁽ⁱⁱ⁾).

After initial descriptive, sample equivalence, and related analyses were concluded, TESI's reliability levels were investigated. Results from these analyses are described next.

Reliability

A Kuder-Richardson 20 reliability coefficient, calculated for the entire sample, yielded an alpha of .92. Item-total correlations were calculated in each of the above cases. All of the 39 items contributed to, or, at minimum did not diminish TESI's overall internal consistency. All item-total correlations are presented in Table 1. KR-20 coefficients calculated separately for men (a=.92) and women (a=.92) showed identical results. In addition, KR-20 coefficients computed for non-Latino Whites (a=.93), African Americans (a=.92), Latinos (a=.91) and those describing their ethnic identification as "other" (alpha=.92) reflected the same high internal consistency levels.

It is noteworthy that a further comparison of reliability of the clinical and non-clinical sample revealed that, while TESI's internal consistency for the clinical group was similar to that found in other analyses mentioned above (alpha =.91), it was substantially lower for the non-clinical group (alpha = .43). This result was not unexpected, given the low average number of items endorsed by the non-clinical sample (Mean=5.28). Such limited variability also compromises the corresponding alpha level. In addition, it makes theoretical sense that, by definition, symptoms identified as clustering together due to a specific mental disorder such as PTSD are not necessarily interconnected or related in the same way for those not experiencing the disorder. The non-clinical group's low alpha level was thus seen as reflecting the specific nature of symptom clusters associated with PTSD rather than evidence of any limitations in TESI's reliability.

Overall, TESI appears to exhibit very good internal consistency. Results indicate that it is a very reliable instrument for all ethnic and gender groups identified in the sample

Validity

Convergent & Divergent Validity

Bivariate correlations (2-tailed) were calculated between the TESI Total Score, the Beck Depression Inventory (BDI), and all Minnesota Multiphasic Personality Inventory (MMPI) clinical as well as validity sub-scales. BDI and MMPI results were only available for the clinical group, limiting analyses to that sample. Results were as follows:

Correlations with MMPI validity scales: A significant positive correlation between the TESI Total Score and the F Scale (r=.41, P<.0001) was found. Conversely, there were significant negative correlations between the TESI Total Score and L (r=-.18, P<.0001) as well as the K Scales (r=-.29, p<.0001). Using standard interpretations for L, F, and K scales one can conclude that respondents with a greater tendency to present themselves in a favorable light by denying basic human frailties and those who take a defensive test-taking posture by minimizing psychological problems tended to have lower TESI Total Scores. On the other hand, those who are very willing to admit to unusual and even bizarre psychological experiences tended to have higher TESI Total Scores.

Overall, these results are as one would predict. They suggest that, in cases where TESI Total Scores are very low, one must consider the possibility that respondents are guarded and wish to "look good " while particularly high scores suggest the possibility that the respondent is exaggerating and over-reporting symptoms.

Correlations with MMPI clinical scales: Significant positive correlations were found between the TESI Total Score and Hs (r=.33, p<.0001), D (r=.39, p<.0001), HY (r=.37, p<.0001), Pd (r=.34, p<.0001), Pa (r=.45, p=<.0001), Sc (r=.45, P<.0001), Ma (r=.31, p<.0001) , and Si (r=.40, p<.0001) scales. No significant correlation between the TESI Total Score and MF (r=-.02, p=.18) was noted. In short, TESI appears to be sensitive to fairly broad-based psychopathology.⁽ⁱⁱⁱ⁾

Correlation with BDI: There was a significant positive correlation between the TESI total score and the Beck Depression Inventory (r=.53, p<.0001).

In addition to considering significance levels, these correlations must also be evaluated in terms of their magnitude. Given the relatively large size of the sample which was used for these analyses (1491) even fairly small correlations were likely to be statistically significant. In terms of clinical evaluations, the largest positive correlations (above .39) were found between the TESI Total Score and MMPI's D, Pa, Pt, and Sc, and Si scales as well as the Beck.

A review of DSM-IV diagnostic criteria for PTSD shows substantial overlap with depressive symptoms (e.g., Criteria C 4-6 as well as D1, 3), symptoms that could be interpreted as paranoid without understanding the situational context (e.g., hypervigilance), symptoms of anxiety, the reporting of very unusual experiences, and a preference for avoiding activities, places, or people (e.g., Criterion C-2). The overall results are also consistent with the research literature which frequently reports the D, Pt., and Sc. MMPI scales as particularly elevated in the case of PTSD. While, as previously mentioned, TESI is sensitive a fairly broad band of psychopathology, the strongest relationships between it and other measures of such pathology do appear to occur in areas more specifically connected with the experience of PTSD.

Construct Validity

Individual TESI items (1-39) were factor analyzed to determine which items clustered together in a discernible structure.^(iv) Given that, by definition, the structure of typical PTSD symptoms is not meaningful for those who do not have the disorder, only the clinical sample was used in this analysis. Principal components analysis using varimax rotation yielded eight factors with eigenvalues of 1.0 or higher. Factor 1 (eigenvalue=9.19) accounted for 23% of the variance. Factor 2 (eigenvalue = 1.99) explained 5% of the variance. The remaining factors yielded figures as follows: Factor 3 (eigenvalue =1.44) 3.7%; Factor 4 (eigenvalue 1.27) 3.3%; Factor 5 (eigenvalue = 1.2) 3%; Factor 6 (eigenvalue = 1.1) 2.9%; Factor 7 (eigenvalue = 1.1) 2.7%; and Factor 8 (eigenvalue 1.0) 2.6%. The entire analysis thus accounted for 46.8% of the variance. Specific items associated with various factors are presented in Table 2. Underlying concepts for these factors might be described as follows:

Factor 1 might best be described as detachment & loss of control;

Factor 2 relates to impaired cognitive abilities;

Factor 3 can be termed physical complaints, primarily related to digestive processes;

Factor 4 captures physical complaints primarily related to anxiety & stress;

Factor 5 taps into ruminations and related dysfunctions;

Factor 6 taps into anger and frustration;

Factor 7 relates to psychomotor agitation;

Factor 8 can be labeled "marital problems."

Overall, the factor structure supports TESI as an instrument which incorporates dimensions relevant to the diagnosis of PTSD.

Given the differences in TESI item responses between gender and ethnic groups within the clinical sample, principal components analysis was also repeated separately for each ethnic and gender group. Results indicate that that TESI's factor structure differs somewhat among these sub-samples. For each group, 9 factor with eigenvalues above 1.0 were derived. Principal component analysis with African Americans accounted for 50.34% of the variance. For Latinos, the analysis accounting for 50.06% of the variance. When the analysis included only non-Latino Whites, it accounted for 53.75% of the variance. For women, the analysis accounting for 56% of the variance. Finally, principal components analysis including only men accounted for 57.6% of the variance.

An investigation of which individual TESI item loaded on specific factors found similarities and differences across groups. For example, Item 11 (Fits of anger) and Item 16 (Anger toward self) tended to hang together for all groups except African Americans. Given limitations of principal components analysis with dichotomous variables, it would not be wise to over-interpret these specific results (see footnote ^vi). It does seem plausible, however, that, while items included in the TESI consist of dimensions which are important considerations in assessing for PTSD for all groups, specific symptoms are perceived and interpreted differently by ethnic and gender groups.

Given the redundant use of items in the various groupings which have been previously developed through theoretical means (Primary Disturbances, Secondary Disturbances, Functional Disturbances, and their various sub-types) it was not possible to adequately test this proposed structure through factor analysis. This does not automatically lessen the value of such groupings in the clinical setting. Principal components analysis captures the way in which symptoms are perceived by respondents. Groupings outlining treatment considerations have to be evaluated based on their usefulness in that context. For example, "difficulties working" and "loss of interest in sex" may not automatically load on one factor. But it is probably useful if they are listed together for clinicians as "functional disturbances" which need to be considered when addressing that area in treatment planning.

Criterion-Related Validity

Logistic regressions were calculated in order to ascertain how accurately the TESI Total Score predicted 1) clinical status and 2) comorbidity. TESI accurately predicted membership in the non-clinical and clinical groups in 79.63% of the cases. The inclusion of age, gender, and ethnicity added less than 1 % to this accuracy. (It was not possible to enter results from the MMPI and Beck Depression Inventory into this model since results from those measures were only available for the clinical sub-sample). TESI Total Scores thus showed significant accuracy (p<.0001) in predicting clinical versus non-clinical group membership.

TESI alone was also a significant predictor of comorbidity (p<.0001) for the clinical group. It accurately placed 74.38% of the sample. A additional model investigated how accurately the TESI Total Score predicted comorbidity above an beyond demographic variables and the Beck Depression Inventory. (Since comorbidity was derived using the MMPI, those scales were not included). Demographic variables (age, education, employment status, race/ethnicity, and gender) were entered on a first step. The Beck Depression Inventory was entered on a second step, and the TESI total score was entered on a third and final step. Despite controlling for demographic factors and the Beck, TESI was significant (p=.0064) in predicting comorbidity. Addition of the TESI total score improved predictive accuracy of the model from 77.9% to 80.44%. In short, TESI added to the accurate prediction of comorbidity beyond considering demographics or the Beck.

Standardization & Norms

Tables 3 and 4 present TESI total raw score to linear T-score conversion tables for the clinical and non-clinical samples. In addition, T-scores for the clinical sample are further broken down into males, females, African Americans (total clinical sample), African American males, African American females, Latinos (total clinical sample), Latino males, Latino females, non-Latino Whites (total clinical sample), White males, and White females. T-scores for the non-clinical sample are broken down into males, females, African Americans, Latinos, and non-Latino Whites.^(v) Because of the smaller numbers for the non-clinical sample, a further breakdown was not advisable. Finally, Table 5 presents a comparison of T-scores for the clinical and non-clinical sample. This table is further broken down into male, female, African American, Latino, and non-Latino White sub-samples.

T-scores are based on a mean of 50 and a standard deviation of 10. They thus allow one to determine, at a glance, where individuals fall on the overall distribution. Persons with scores of 40 to 60 fall within one standard deviation of the mean. Those with scores of 20 to 39 fall into the second standard deviation below the mean, and conversely, those scoring 61 to 70 fall into the second standard deviation above the mean, and so on.

The formula for deriving such standard scores is:

                            _
   T-Score = [ (Raw score - X) / SD] X 10 + 50

It should be noted that the calculated standard scores do not automatically reflect what is often thought of as "norms." Norms imply that the sample is not skewed. In the present case the sample is significantly skewed in the positive direction indicating many lower and only some higher scores for both the clinical (Skewness=.598; Standard Error=.063) and non-clinical (Skewness=.386; Standard Error=.121) groups (see histograms on the next page).

The term "linear" used in this report indicates that the raw scores were transformed to T-Scores without attempting to do any "smoothing/normalizing" to correct for this skew and/or kurtosis in the raw-score distribution. This approach was used in the present analyses for the following reasons:

Linear T-Scores are, in part, advised when there is reason to believe that the distribution is not "normal" in the real world. This is probably the case with results from TESI for both the clinical and non-clinical samples. Psychopathology associated with PTSD is the result of an extreme traumatic stressor that tends to fall outside the range of usual human experience. It is likely most persons in the non-clinical sample experience only very few, of the problems included in TESI in the course of their every-day lives. Thus an expected positively skewed curve. Even for the clinical sample, it is generally anticipated that a majority of people experience a relatively small number and few experience a very large number of symptoms. (The measure must, however include a large range of symptoms in order to avoid a "ceiling effect"^(vi)).

Histograms showing distributions for both the clinical and non-clinical groups are presented below:

endnotes

i    A potential limitation of the sample is that it was not randomly selected. While use of available samples is often unavoidable in psychological research, results must be generalized cautiously. The literature generally recommends that when analyzing such samples, it is expedient to address the extent to which groupings account for variance in responses. Such analyses were thus conducted and are described in the body of the text.

ii    Initial data preparation included a check for normality of distribution in the case of all continuous variables. Data were visually inspected and tested for significant levels of skew and kurtosis. Skewed and/or kurtotic variables were reflected and transformed as appropriate to reduce non-normal features.

iii    The MMPI supplemental scale most often associated with Post Traumatic Stress Disorder, the Keane PTSD Scale (Pk Scale) was, unfortunately, not available for this sample. While inclusion of this scale in the current analyses would have been advantageous, correlations between the MMPI's other clinical scales and TESI were never-the-less seen as providing important information about TESI's validity since they allowed for an investigation of the degree to which specific patterns of pathology generally associated with PTSD were reflected in TESI total scores.

iv   It should be noted that using such analysis with binary (yes/no) items is somewhat controversial unless items tend to fall between the 30% to 70% endorsement rate. In the present (clinical) sample, two items were endorsed by more than 70% of the sample. TESI Item 6 was endorsed by 74% and TESI Item 7 was endorsed by 79%. No items were endorsed by less than 30% of the sample. The scale thus appears to be relatively free of problems which could pose difficulties. While occasionally a little-known technique called "monotinicity analysis" is recommended in situations such as the present, the most generally accepted rule of thumb is to proceed with factor analysis but interpret results with caution.

v   While, as previously indicated, no significant gender or ethnic differences were found on the TESI Total scores for the non-clinical group, it was seen as advantageous to calculate separate T-scores for the groups. These scores can be compared to the results of future TESI research with different (particularly randomly selected) samples to determine if the pattern is replicated.

vi   A ceiling effect would occur if TESI included too few PTSD dimensions, creating a situation in which greater levels of psychopathology could not be measured. In other words, persons with a moderate number of problems would receive the same score as those with greater numbers because the instrument would not contain all relevant symptoms for individuals (experiencing high levels of dysfunction) to check off.

Return to the top
Return to the Table of Contents