Adaptation, Validation of the PTOPS in an Italian Population
Adaptation, Validation of the PTOPS in an Italian Population
357 subjects were asked to participate to this study. Only 3 subjects refused; therefore, the trial was based upon a sample of 354 subjects, with mean age 49.96 years (SD = 13.48). Complete information was available for 345 subjects. Nonetheless, for some domains a higher number of cases is available (since some of the 354 subjects provided complete information for one domain but not for another).
Socio-demographic characteristics of the sample are described in Table 1.
56 subjects repeated the completion of the questionnaire one week after the first administration to allow the analysis of test-retest stability.
The forward backward translation process performed by 4 translators required 3 months to achieve a culturally adapted version. A further revision and adaptation after the beta version testing required 1 month.
Acceptability On the average, the PTOPS-I was completed in 6 minutes, 49 seconds (SD = 2.43). Subjects experienced difficulty or needed assistance 168 times with a mean of 0.47 (SD = 1.09) clarifications for each questionnaire. In Figure 1 the distributions of the variables are displayed.
(Enlarge Image)
Figure 1.
Frequency distributions of the variables. Each distribution is represented by stacked bars, each stack representing one of the possible values (1 = Strongly Disagree; 2 = Disagree; 3 = Uncertain; 4 = Agree; 5 = Strongly Agree) taken by the variable. The height of a stack represents the observed count of the corresponding value. The variables with the highest proportions of patients answering with "I do not know" are Q01, Q04, Q09, Q17, Q25, Q29 and Q30 which related to costs.
Following Roush and Sonstroem, the Enhancers and the Location domains have a positive connotation, whilst the Detractor and Cost domains have a negative one. Hence, high scores on the items in the first two domains, and low scores on the items in the last ones correspond to high levels of satisfaction. To make all the items coherent with their domain, the 5-points Likert-scale for items Q04, Q09, Q10, Q16, Q19, Q21, Q25, and Q28 was reversed (1 = 5, 2 = 4, 4 = 2, 5 = 1) so that the highest score (5) corresponds to the greatest satisfaction.
The items with the highest average rating were Q7, with an average of 4.57, Q2 with an average rating of 4.49, and Q23, with an average of 4.53 on the inverted scale. Items with the lowest average ratings were Q14, with a mean of 3.93, and Q29, with 3.51.
The variables with the highest proportions of patients answering with 'I do not know' are all related to costs.
For the study of the reliability and the validity, we performed two separate analyses:
Analysis 1. Analysis of the constructs identified by Roush and Sonstroem.
Internal Consistency The Cronbach's α obtained for each of the four domains-totals of the PTOPS-I are shown in Table 2 . This table also displays the values of α obtained after having deleted one item at a time from each domain. It clearly emerges from the data that some items are not particularly connected to the others: Q16 ('I have to wait too long between appointments') and Q29 ('The facility appreciates my business') in the R-S Enhancers domain; Q8 ('It is difficult for me to get into the facility from the parking lot') in the R-S Detractors domain; Q14 ('The facility is in a desirable location') in the R-S Location domain, and Q11 ('I feel my therapist overcharges me') and Q25 ('My therapist does not expect me to pay significantly more than what my insurance covers') in the R-S Cost domain. Moreover, the corrected items in total correlation show a general low-moderate α correlations. As shown by the figures in Table 2 , the PTOPS-I exhibits a good overall reliability: the α values are acceptable for the Enhancers and Cost domains (α = 0.758 and 0.706 respectively), and are good for the Detractors and Location domains (α = 0.848 and 0.886 respectively).
Construct validity was investigated by factor analysis of each domain separately. The proportion of total variance explained by the factor as well as its correlations (loadings) with the items are reported in Table 3 . In particular the R-S Locations and Detractors domains are characterized by the strongest internal coherence (for these domains the first factor explains a substantial proportion of the total variance equal to 0.6 and 0.46 respectively, and medium to high correlations between the factor and the domains-items) whilst the factors extracted for the R-S Enhancers and Costs domains appear to be weaker (i.e., exhibited moderate proportions of explained total variance, equal to 0.34 and 0.4 respectively). By analyzing the factor loadings, and by increasing the number of extracted factors for each domain to explore whether more factors were necessary to satisfactorily describe the items assigned to each domain, we noted that the relatively poor performance of the first factors were substantially due to the presence of some variables that were weakly related to the others within the same domain. These variables were the same which impacted the analysis of Cronbach's α's. Hence, factor analysis substantially confirmed the results of the Cronbach's α's reported in Table 2 , with respect both to the R-S- domains' internal consistency and to the most critical variables within each R-S- domain.
Test-Retest Stability To assess the repeatability of the PTOPS-I, the questionnaire was re-administered to a sample of 56 patients one week after the first filling. 53 out of the 56 subjects returned complete information. The relation between results gathered in the two administrations was investigated for each item and for each domain total using the ICC coefficients (see Table 4 ). Some coefficients appear not particularly high (around 0.5), even if they are all significantly higher than zero (as clearly indicated by the ICC 95% confidence intervals). Focusing on the domains sub-totals, the highest ICC was observed for the Detractors domain (with ICC = 0.891, and ICC 95% CI = 0.818–0.936), followed by the Location and Cost domains (both characterized by ICC = 0.860 and ICC 95% CI = 0.768–0.917) and by the Enhancers domain (with ICC = 0.766, and ICC 95% CI = 0.626–0.859).
ValidityConcurrent validity was assessed by calculating the Pearson correlation coefficient between the PTOPS-I totals and the scores of the other administered questionnaires (GPE and VAS). As shown in Table 5 , the GPE was significantly associated with the Enhancers (r = −0.429, P < 0.0001), Detractors (r = 0.281, P < 0.0001) and Cost (r = 0.328, P < 0.0001) totals, but was not related to the Location total. As for VAS, only the correlations with the Enhancers and the Detractors totals were statistically different from zero, but they demonstrated very low values.
Analysis 2. Determination of constructs possibly different from those identified by Roush and Sonstroem.
In the previous analysis, attention was focused on the domains determined by Roush and Sonstroem on data collected on Italian patients. We then evaluated whether a different domain specification was more suited to our data. To accomplish the objectives of this aim, factor analysis was performed using the full set of items, using the principal components extraction method and the varimax rotation criterion.
We extracted 4 factors, explaining together 48.6% of the total variance. By analyzing the correlations (loadings) between the items and the four factors (see Table 6 ), and by assigning each item to the factor to which it was most correlated, we observed a distribution of items across four factors that were similar to Roush and Sonstroem with some interesting differences. More precisely, item Q16 ('I have to wait too long between appointments') which was placed into the R-S Enhancers domain and item Q11 ('I feel my therapist overcharges me') assigned to the R-S Cost domain were instead included in our first I-domain (mostly overlapping with the R-S Detractors domain); item Q14 ('The facility is in a desirable location') assigned to the Location R-S domain was instead associated with the third I-domain (mostly overlapping with the R-S Enhancers domain).
Also, our results suggested that item Q08 ('It is difficult for me to get into the facility from the parking lot') had to be assigned to the fourth I-domain, since surprisingly it showed the highest correlation with the fourth factor (summarizing the Cost-items), even if the loading was not particularly high (0.42). Nonetheless, a weaker relation (r = 0.32) was also observed with the first factor. From the results reported in Table 3 one can note that Q08 is not particularly related to the other items placed in the R-S Detractors domain. Also, the correlation (loading) with the unique factor extracted to evaluate the construct validity of the R-S Detractors' domain was rather low (0.37) and aligned with the correlation observed with the first I-factor (0.32). Based on these considerations, for the sake of interpretation, we assigned the item to the first I-domain. Therefore, we determined that the R-S factor names were not completely relevant to an Italian context and chose 'Depersonalization', 'Inaccessibility', 'Ambience', and 'Cost' to name the domains corresponding to the four I-factors.
The internal consistency of the I-domains is higher compared to the R-S domains. The overall Cronbach's alpha reaches 0.784 for the Ambience I-domain, 0.87 for the Depersonalization I-domain, and 0.902 and 0.754 for the Inaccessibility and the Cost I-domains respectively.
Construct validity was again analyzed by applying four factor analyses, one for each I-block. The obtained results (displayed in Table 7 ) are very similar to those observed for the R-S- domains, and also in this case are consistent with the indications provided by the Cronbach's α's.
Test-Retest Stability Results are similar to those obtained for the Roush and Sonstroem blocks: as for the total scores (obtained by summing up the scores of all the items in each block) the ICC's turned out to be 0.745 (95%C.I.: 0.596–0.846) for Ambience, 0.903 (95%C.I.: 0.835–0.942) for Depersonalization, 0.875 (95%C.I.: 0.789–0.925) for Location, and 0.855 (95%C.I.: 0.758–0.913) for Cost.
Concurrent Validity As shown in Table 8 , GPE is significantly associated only to the totals based on Depersonalization (r = 0.269, P < 0.001), Ambience (r = −0.378, P < 0.001), and Cost, (r = 0.356, P < 0.001) but not related to the Inaccessibility total. VAS is not significantly related to the considered totals. Even though some correlations have relatively low p-values, very low values of the coefficients were also observed.
To evaluate the relative importance of the subtotals on the global level of satisfaction, a total score was obtained by summing up the scores on all the items, recoding some variables for analytic purposes to ensure that all the items had the same positive direction. The 4 totals (Depersonalization, Inaccessibility, Ambience, Cost) were also re-built using the modified item blocks. The correlations between the total score and the four sub-totals showed that the Depersonalization total is the one most correlated to the global one (r = 0.828, P < 0.0001), followed by Ambience (r = 0.707, P < 0.0001), Inaccessibility (r = 0.619, P < 0.0001), and Cost (r = 0.361, P < 0.0001). Note that the low impact of the last domain on the grand total is expected since the Cost-items are those characterized by the highest proportion of 'I do not know' responses. To evaluate the impact of each sub-total on the grand total accounting for the other sub-totals, the grand-total was regressed on the three most relevant sub-totals using a linear model. Our results confirmed that the Depersonalization total (and hence the items it comprises) most influences global level of satisfaction.
Analysis of the Dependency of the Indicators of Satisfaction Finally, we analyzed the dependency of satisfaction on the background variables, i.e. the variables related to the characteristics of the facility and/or of the therapist and of the patients. Because these variables are categorical, we used an ANOVA procedure, testing the null hypothesis that the means do not vary according to the levels of each explanatory variable for each sub-total. Since the distribution of the sub-totals was found not to be normal, a non-parametric approach was used, based on the Kruskal-Wallis or on the Wilcoxon test for explanatory variables having only two levels. For the sake of a compact synthesis of results, only the most relevant results are reported here.
These analyses determined differences in Ambience and Cost sub-totals across the three facilities (P = 0.0015 for Ambience; P < 0.0001 for Cost). Furthermore, we observed that for all the sub-totals but Location, the most satisfied patients are those having a female therapist (P = 0.015 for Depersonalisation, P < 0.0001 for Ambience, P = 0.0021 for Cost) and those who regularly attend the facility (P = 0.007 for Depersonalisation, P =0.004 for Ambience, P = 0.0005 for Cost). The interaction between the patient's and the therapist's gender was not significant. Moreover, all the sub-totals but not Depersonalisation differ in mean across the age strata and education level (P = <.0001 and P = 0.0036 for Ambience, P = 0.0055 and P = 0.0313 for Location, P = 0.0087 and P = 0.0086 for Cost). The most satisfied patients were the oldest ones and those with the lowest level of education ones. These findings are related as the oldest persons in our sample were those with the lowest levels of education. Finally, only for the Ambience sub-total, a significant difference in means was observed across the working status (P < .0001), the retired and the unemployed patients being the most satisfied ones.
Results
Subjects
357 subjects were asked to participate to this study. Only 3 subjects refused; therefore, the trial was based upon a sample of 354 subjects, with mean age 49.96 years (SD = 13.48). Complete information was available for 345 subjects. Nonetheless, for some domains a higher number of cases is available (since some of the 354 subjects provided complete information for one domain but not for another).
Socio-demographic characteristics of the sample are described in Table 1.
56 subjects repeated the completion of the questionnaire one week after the first administration to allow the analysis of test-retest stability.
Translation and Cross-cultural Adaptation
The forward backward translation process performed by 4 translators required 3 months to achieve a culturally adapted version. A further revision and adaptation after the beta version testing required 1 month.
Psychometric Characteristics
Acceptability On the average, the PTOPS-I was completed in 6 minutes, 49 seconds (SD = 2.43). Subjects experienced difficulty or needed assistance 168 times with a mean of 0.47 (SD = 1.09) clarifications for each questionnaire. In Figure 1 the distributions of the variables are displayed.
(Enlarge Image)
Figure 1.
Frequency distributions of the variables. Each distribution is represented by stacked bars, each stack representing one of the possible values (1 = Strongly Disagree; 2 = Disagree; 3 = Uncertain; 4 = Agree; 5 = Strongly Agree) taken by the variable. The height of a stack represents the observed count of the corresponding value. The variables with the highest proportions of patients answering with "I do not know" are Q01, Q04, Q09, Q17, Q25, Q29 and Q30 which related to costs.
Following Roush and Sonstroem, the Enhancers and the Location domains have a positive connotation, whilst the Detractor and Cost domains have a negative one. Hence, high scores on the items in the first two domains, and low scores on the items in the last ones correspond to high levels of satisfaction. To make all the items coherent with their domain, the 5-points Likert-scale for items Q04, Q09, Q10, Q16, Q19, Q21, Q25, and Q28 was reversed (1 = 5, 2 = 4, 4 = 2, 5 = 1) so that the highest score (5) corresponds to the greatest satisfaction.
The items with the highest average rating were Q7, with an average of 4.57, Q2 with an average rating of 4.49, and Q23, with an average of 4.53 on the inverted scale. Items with the lowest average ratings were Q14, with a mean of 3.93, and Q29, with 3.51.
The variables with the highest proportions of patients answering with 'I do not know' are all related to costs.
For the study of the reliability and the validity, we performed two separate analyses:
Analysis 1. To evaluate whether our results were consistent with the four domains identified by Roush and Sonstroem for the U.S. version of the questionnaire ("R-S- domains").
Analysis 2. To explore the possible existence of a different domain structure, specific to an Italian version of the PTOPS, and analyze the characteristics of these 'Italian-based' domains ("I-domains").
Analysis 1. Analysis of the constructs identified by Roush and Sonstroem.
Reliability
Internal Consistency The Cronbach's α obtained for each of the four domains-totals of the PTOPS-I are shown in Table 2 . This table also displays the values of α obtained after having deleted one item at a time from each domain. It clearly emerges from the data that some items are not particularly connected to the others: Q16 ('I have to wait too long between appointments') and Q29 ('The facility appreciates my business') in the R-S Enhancers domain; Q8 ('It is difficult for me to get into the facility from the parking lot') in the R-S Detractors domain; Q14 ('The facility is in a desirable location') in the R-S Location domain, and Q11 ('I feel my therapist overcharges me') and Q25 ('My therapist does not expect me to pay significantly more than what my insurance covers') in the R-S Cost domain. Moreover, the corrected items in total correlation show a general low-moderate α correlations. As shown by the figures in Table 2 , the PTOPS-I exhibits a good overall reliability: the α values are acceptable for the Enhancers and Cost domains (α = 0.758 and 0.706 respectively), and are good for the Detractors and Location domains (α = 0.848 and 0.886 respectively).
Construct validity was investigated by factor analysis of each domain separately. The proportion of total variance explained by the factor as well as its correlations (loadings) with the items are reported in Table 3 . In particular the R-S Locations and Detractors domains are characterized by the strongest internal coherence (for these domains the first factor explains a substantial proportion of the total variance equal to 0.6 and 0.46 respectively, and medium to high correlations between the factor and the domains-items) whilst the factors extracted for the R-S Enhancers and Costs domains appear to be weaker (i.e., exhibited moderate proportions of explained total variance, equal to 0.34 and 0.4 respectively). By analyzing the factor loadings, and by increasing the number of extracted factors for each domain to explore whether more factors were necessary to satisfactorily describe the items assigned to each domain, we noted that the relatively poor performance of the first factors were substantially due to the presence of some variables that were weakly related to the others within the same domain. These variables were the same which impacted the analysis of Cronbach's α's. Hence, factor analysis substantially confirmed the results of the Cronbach's α's reported in Table 2 , with respect both to the R-S- domains' internal consistency and to the most critical variables within each R-S- domain.
Test-Retest Stability To assess the repeatability of the PTOPS-I, the questionnaire was re-administered to a sample of 56 patients one week after the first filling. 53 out of the 56 subjects returned complete information. The relation between results gathered in the two administrations was investigated for each item and for each domain total using the ICC coefficients (see Table 4 ). Some coefficients appear not particularly high (around 0.5), even if they are all significantly higher than zero (as clearly indicated by the ICC 95% confidence intervals). Focusing on the domains sub-totals, the highest ICC was observed for the Detractors domain (with ICC = 0.891, and ICC 95% CI = 0.818–0.936), followed by the Location and Cost domains (both characterized by ICC = 0.860 and ICC 95% CI = 0.768–0.917) and by the Enhancers domain (with ICC = 0.766, and ICC 95% CI = 0.626–0.859).
ValidityConcurrent validity was assessed by calculating the Pearson correlation coefficient between the PTOPS-I totals and the scores of the other administered questionnaires (GPE and VAS). As shown in Table 5 , the GPE was significantly associated with the Enhancers (r = −0.429, P < 0.0001), Detractors (r = 0.281, P < 0.0001) and Cost (r = 0.328, P < 0.0001) totals, but was not related to the Location total. As for VAS, only the correlations with the Enhancers and the Detractors totals were statistically different from zero, but they demonstrated very low values.
Analysis 2. Determination of constructs possibly different from those identified by Roush and Sonstroem.
In the previous analysis, attention was focused on the domains determined by Roush and Sonstroem on data collected on Italian patients. We then evaluated whether a different domain specification was more suited to our data. To accomplish the objectives of this aim, factor analysis was performed using the full set of items, using the principal components extraction method and the varimax rotation criterion.
We extracted 4 factors, explaining together 48.6% of the total variance. By analyzing the correlations (loadings) between the items and the four factors (see Table 6 ), and by assigning each item to the factor to which it was most correlated, we observed a distribution of items across four factors that were similar to Roush and Sonstroem with some interesting differences. More precisely, item Q16 ('I have to wait too long between appointments') which was placed into the R-S Enhancers domain and item Q11 ('I feel my therapist overcharges me') assigned to the R-S Cost domain were instead included in our first I-domain (mostly overlapping with the R-S Detractors domain); item Q14 ('The facility is in a desirable location') assigned to the Location R-S domain was instead associated with the third I-domain (mostly overlapping with the R-S Enhancers domain).
Also, our results suggested that item Q08 ('It is difficult for me to get into the facility from the parking lot') had to be assigned to the fourth I-domain, since surprisingly it showed the highest correlation with the fourth factor (summarizing the Cost-items), even if the loading was not particularly high (0.42). Nonetheless, a weaker relation (r = 0.32) was also observed with the first factor. From the results reported in Table 3 one can note that Q08 is not particularly related to the other items placed in the R-S Detractors domain. Also, the correlation (loading) with the unique factor extracted to evaluate the construct validity of the R-S Detractors' domain was rather low (0.37) and aligned with the correlation observed with the first I-factor (0.32). Based on these considerations, for the sake of interpretation, we assigned the item to the first I-domain. Therefore, we determined that the R-S factor names were not completely relevant to an Italian context and chose 'Depersonalization', 'Inaccessibility', 'Ambience', and 'Cost' to name the domains corresponding to the four I-factors.
The internal consistency of the I-domains is higher compared to the R-S domains. The overall Cronbach's alpha reaches 0.784 for the Ambience I-domain, 0.87 for the Depersonalization I-domain, and 0.902 and 0.754 for the Inaccessibility and the Cost I-domains respectively.
Construct validity was again analyzed by applying four factor analyses, one for each I-block. The obtained results (displayed in Table 7 ) are very similar to those observed for the R-S- domains, and also in this case are consistent with the indications provided by the Cronbach's α's.
Test-Retest Stability Results are similar to those obtained for the Roush and Sonstroem blocks: as for the total scores (obtained by summing up the scores of all the items in each block) the ICC's turned out to be 0.745 (95%C.I.: 0.596–0.846) for Ambience, 0.903 (95%C.I.: 0.835–0.942) for Depersonalization, 0.875 (95%C.I.: 0.789–0.925) for Location, and 0.855 (95%C.I.: 0.758–0.913) for Cost.
Concurrent Validity As shown in Table 8 , GPE is significantly associated only to the totals based on Depersonalization (r = 0.269, P < 0.001), Ambience (r = −0.378, P < 0.001), and Cost, (r = 0.356, P < 0.001) but not related to the Inaccessibility total. VAS is not significantly related to the considered totals. Even though some correlations have relatively low p-values, very low values of the coefficients were also observed.
To evaluate the relative importance of the subtotals on the global level of satisfaction, a total score was obtained by summing up the scores on all the items, recoding some variables for analytic purposes to ensure that all the items had the same positive direction. The 4 totals (Depersonalization, Inaccessibility, Ambience, Cost) were also re-built using the modified item blocks. The correlations between the total score and the four sub-totals showed that the Depersonalization total is the one most correlated to the global one (r = 0.828, P < 0.0001), followed by Ambience (r = 0.707, P < 0.0001), Inaccessibility (r = 0.619, P < 0.0001), and Cost (r = 0.361, P < 0.0001). Note that the low impact of the last domain on the grand total is expected since the Cost-items are those characterized by the highest proportion of 'I do not know' responses. To evaluate the impact of each sub-total on the grand total accounting for the other sub-totals, the grand-total was regressed on the three most relevant sub-totals using a linear model. Our results confirmed that the Depersonalization total (and hence the items it comprises) most influences global level of satisfaction.
Analysis of the Dependency of the Indicators of Satisfaction Finally, we analyzed the dependency of satisfaction on the background variables, i.e. the variables related to the characteristics of the facility and/or of the therapist and of the patients. Because these variables are categorical, we used an ANOVA procedure, testing the null hypothesis that the means do not vary according to the levels of each explanatory variable for each sub-total. Since the distribution of the sub-totals was found not to be normal, a non-parametric approach was used, based on the Kruskal-Wallis or on the Wilcoxon test for explanatory variables having only two levels. For the sake of a compact synthesis of results, only the most relevant results are reported here.
These analyses determined differences in Ambience and Cost sub-totals across the three facilities (P = 0.0015 for Ambience; P < 0.0001 for Cost). Furthermore, we observed that for all the sub-totals but Location, the most satisfied patients are those having a female therapist (P = 0.015 for Depersonalisation, P < 0.0001 for Ambience, P = 0.0021 for Cost) and those who regularly attend the facility (P = 0.007 for Depersonalisation, P =0.004 for Ambience, P = 0.0005 for Cost). The interaction between the patient's and the therapist's gender was not significant. Moreover, all the sub-totals but not Depersonalisation differ in mean across the age strata and education level (P = <.0001 and P = 0.0036 for Ambience, P = 0.0055 and P = 0.0313 for Location, P = 0.0087 and P = 0.0086 for Cost). The most satisfied patients were the oldest ones and those with the lowest level of education ones. These findings are related as the oldest persons in our sample were those with the lowest levels of education. Finally, only for the Ambience sub-total, a significant difference in means was observed across the working status (P < .0001), the retired and the unemployed patients being the most satisfied ones.
Source...