Socioeconomic Factors and Outcomes in Higher Education
5. Main findings
Texte intégral
5.1. Construction of a single ses index
 4 It is important to point out that several attempts of catpca and efa were performed before doing th (...)
1The ses index was created by performing the mca4. As described in section 3.4, five socioeconomic variables were ordinal, and two variables were nominal. The Appendix 2 presents the category plots for this analysis.
5.1.1. Eigenvalue correction for mca
2The mca solution was created using a CA on the Burt Matrix instead of the indicator Matrix because a better approximation of the inertia explained by the factors could be obtained. As reported by Abdi and Valentine (2007):
mca codes data by creating several binary columns for each variable with the constraint that one and only one of the columns gets the value 1. This coding schema creates artificial additional dimensions because one categorical variable is coded with several columns. As a consequence, the inertia (i. e., variance) of the solution space is artificially inflated and therefore the percentage of inertia explained by the first dimension is severely underestimated. (p. 653)
3Hence the obtained value for the inertia was corrected through the formulas proposed by Benzécri (1979) and Greenacre (1993). According to Benzécri the percentage of adjusted inertia associated with a factor is obtained dividing the corrected eigenvalue by the sum of all corrected eigenvalues. However, Greenacre suggests dividing not for the sum of all corrected eigenvalues but the average inertia of the diagonal in the Burt Matrix. The corrections should be made for eigenvalues greater than 1/K, where K is the number of variables. The total number of eigenvalues in this case was 55 (sum of the number of categories minus one in each variable) but the eigenvalues greater than 1/7 were 23. The complete table with these values is presented in the Appendix 2.
4Table 1 presents the new values for the inertia after the adjustments for Dimension 1 and Dimension 2. Therefore, the amount of variance explained by the mca solution was 62.44%.
5.1.2. Discrimination measures
5Table 2 shows the discrimination measures for the mca model. Definitely, all variables have a higher value in Dimension 1 than in Dimension 2.
Table 2. Discrimination measures from the mca
Dimension 1 
Dimension 2 

Father Educational Level 
. 617 
. 332 
Mother Educational Level 
. 597 
. 306 
Father Occupation 
. 416 
. 208 
Mother Occupation 
. 324 
. 197 
Student Stratum 
. 578 
. 405 
sisben Level 
. 379 
. 174 
Monthly Family Income 
. 508 
. 290 
6The plot in Figure 6, below, shows the solution space obtained in this case. A longer vector indicates a better representation in the solution space. Thus, the parents’educational level, the student stratum and the monthly family income are the best represented variables in this solution space. In other words, those variables contribute the most to the total explained variance of the ses index.
5.1.3. Some descriptive statistics for the ses index
7The solution provided by the mca analysis was considered as the ses index. That is, the value obtained in the Dimension 1 was kept as a measure of the socioeconomic condition for each one of the students. Although this variable, Index mca, had mean 0 and a standard deviation of 1, a convenient transformation was carried out to obtain a more meaningful interpretation. In the first place, the minimum value (2.54) was added to each value of the Index mca; secondly, all the values were divided by the maximum value of the result, approximately 4.58. As a result, values from 0 to 1 were obtained with the original distribution remaining the same. The last statement is supported by a comparison between the values of the Dimension 1 (output of the mca) and the final ses index as shown in Table 3, below.
8Figure 7 shows the histogram of the ses index that follows a normal distribution with mean 0.55 and standard deviation of 0.218. Certainly, better socioeconomic conditions are closer to 1; contrary to this, worse socioeconomic conditions are closer to 0.
5.2. A preliminary bivariate approach to the association among ses, saber 11 and saber pro
 5 Although twostep clustering was also used for this classification, the results found were consider (...)
9Before establishing the relation between the socioeconomic variables and the test outcomes, a threemeans cluster was performed5. The variables used in each case were the ses index described in section 5.1 and the standardized values of the saber 11 and saber pro outcomes. As a result, three groups were conformed for each variable: high, middle and low. Final distributions for each cluster are shown below in Tables 4, 5 and 6.
5.2.1. Association between saber 11 and saber pro
10Figure 8 shows the bar chart of the relation between saber 11 and saber pro performance groups. A review of this chart reveals that students with a low score in the saber 11 test also scored low in saber pro; students with a middle (average) score in saber 11 also scored middle (average) in saber pro and students with a high score in saber 11 also scored high in saber pro. According to Figure 8, only a few students with low saber 11 scores are located in the group of students who scored high on saber pro. Similarly, there are only a handful of students with a high saber 11 who belong to the low saber pro group.
11Table 7 shows the contingency table for the saber 11 and saber pro. According to this, the largest percentage of students in each cluster of saber pro comes from the respective cluster of saber 11. To put it differently, 58.5% of students in the saber 11 high group are in the saber pro high group; 57.1% of students in saber 11 middle group are in saber pro middle group, and 74.1% of students in saber 11 low group are in the saber pro low group. This fact consequently displays a considerable association between these two tests; however, it is important to remember that saber 11 is an achievement test and saber pro is a competence test.
 6 Symmetric measures of ordinal association are based on the idea of accounting for concordance versu (...)
12The symmetric measures6 for this case (Kendall’s taub = 0.576 and Kendall’s tauc = 0.553) were significant (p < 0.01). Apparently, there is a reasonable relationship between saber 11 and saber pro performance groups.
5.2.2. Association between saber 11 and ses
13Figure 9 shows the bar chart of the relation between saber 11 groups and ses groups. Intuitively, students from better ses conditions obtain better results in the test. However, it is interesting to identify students in the high performance group of saber 11 who come from middle and low ses groups (green and gray bars). Similarly, there are students in the low performance group of saber 11 who come from high and middle ses groups (blue and green bars).
14Table 8 shows the cross tabulation for saber 11 and ses. Undoubtedly, there is a large percentage (64.3%) of students from the high ses cluster who are in the high saber 11 cluster; in contrast, only few students (9.5%) from the low ses cluster are in the high saber 11 cluster. For the saber 11 middle cluster, there is a more similar distribution of the students with respect to their ses group; that is, 39.6% of students come from the high ses cluster; 38.9% come from the middle ses group and 21.5% come from the low ses cluster. In the case of the low saber 11 cluster, the majority of the students come from the middle ses cluster (42.9%) and the low ses cluster (38.9%).
15The symmetric measures for this case (Kendall’s taub = 0.326 and Kendall’s tauc = 0.317) were significant (p < 0.01). Hence there is a noteworthy, albeit weak, relationship between ses groups and saber 11 performance groups.
5.2.3. Association between saber pro and ses
16Figure 10 shows the bar chart of the relation between saber pro performance groups and ses groups. As noted before in the saber 11 case, there are students in the high performance group of saber pro who come from the middle and the low ses group (see green and gray bars). In the same way, there are students in the low performance group of saber pro who come from high and middle ses groups (blue and green bars).
17Table 9 shows the cross tabulation for the saber pro and ses. In the saber pro high cluster, 52.9% of the students come from the high ses group; 31.6% of the students come from the middle ses group, and 15.5% of students come from the low ses group. Compared to the saber 11 middle group, in the saber pro middle group there is also a more equivalent distribution of the students regarding their ses group. In the saber pro low performance group, 23.6% of students come from ses high group; 41.3% of students come from ses middle group and 35.0% of students come from ses low group.
18The symmetric measures for this case (Kendall’s taub = 0.207 and Kendall’s tauc = 0.201) were significant (p < 0.01). Evidently, there is a weak relationship between ses groups and saber pro performance groups. As a matter of fact, this relation is even weaker than the relation between the saber 11 and ses groups.
Highlight # 1
There is a noticeable relationship between saber 11 and saber pro performance groups. A moderate relationship between ses index and saber 11 was identified, but this relationship was greater than the existing relationship between the ses index and saber pro.
5.3. The power of socioeconomic variables for discriminating the academic performance
 7 Notice that parents’occupation was measured as nominal variable. A dummy coding was used in order t (...)
19For these analyses the socioeconomic variables were considered as independent (discriminating) variables and the saber 11 and saber pro performance groups were considered as dependent variables7. Each of these results is described below.
5.3.1. Socioeconomic variables for discriminating the saber pro performance
– Significance of the discriminating variables
20The descriptive statistics (mean and standard deviation) calculated for all the discriminating variables in each of the saber pro performance groups, showed differences between the values of parents’educational level and family income and closer values for student stratum and sisben level. In order to determine whether those differences were significant, tests of equality of groups’means were performed. According to these results, there were significant differences among the variables between the saber pro performance groups; thus they were suitable for the discriminant analysis. One of the objectives of this part of the study was to determine the most important factor (s) for discriminating students among academic performance groups. Table 10 shows the test of equality of groups’means and the Wilks’lambda for each variable. Apparently, there are small differences in the variables’potential for discriminating, but the sisben level has the smallest effect and family income has the largest effect.
21In addition to the measures used for comparing the contribution of individual predictors to the discriminant model, the eigenvalues and Wilks’lambda have to be interpreted to see how well the discriminant model fits the data (ibm Corporation, 2016b). Table 11 shows the eigenvalues associated to each of the discriminant functions obtained (there are only two functions given that there are three saber pro performance groups).
22In consonance with Table 11, the first function accounted for 99.1% of the discriminant ability of the socioeconomic variables. The squared canonical correlation (CR^{2}) indicates the variation between the performance groups that is explained by the socioeconomic variables (Musso, Kyndt, Cascallar & Dochy, 2013). Hence there is an evidently weak association between the sets of variables (CR^{2} = 0.068).
23Table 12 shows the Wilks’lambda obtained for each function. According to ibm Corporation (2016b):
Wilks’lambda is a measure of how well each function separates cases into groups. It is equal to the proportion of the total variance in the discriminant scores not explained by differences among the groups. Smaller values of Wilks’lambda indicates greater discriminatory ability of the function (para. 1)
24The associated chisquare statistic tests the hypothesis that the means of the functions listed are equal across groups. Given these results, there is evidence to expect a moderate discrimination power of the socioeconomic variables in the saber pro case; even so, the discriminant function does better than a random separation of the groups.
– Discriminating Functions
25Once the previous steps have been carried out, the next procedure was to write the equations for the discriminant functions using the standardized canonical discriminant coefficients. In this case, only the equation for the Function 1 is presented, considering the limited discriminant ability found in the Function 2.
26D1 = 0.235 * Father Educational Level + 0.341 * Mother Educational Level + 0.175 * Student Stratum + 0.103 * sisben Level + 0.468 Family Income (1)
27As a consequence of Equation 1, the variables that serve to discriminate saber pro performance the most are family income and mother educational level. This finding is supported by the structure matrix where the strongest values (strongest correlations with Function 1) are precisely for those two variables.
28With the discriminant functions obtained, the next step was to plot the group graphs to see how well the functions were discriminating the data. In each case, Function 1 and Function 2 are plotted and also the group centroid. Figure 11 shows the graph for each saber pro performance group, as well as the three clusters, plotted at the same time.
29Figure 11 shows greater distance between the centroids in Function 1 than in Function 2. Such a result was expected, considering the greater discrimination ability of the first function. Even though, there is an overlapping among the centroids, indicating the limited power of discrimination of the socioeconomic variables in the saber pro case.
– Classification
30The classification results showed that 46.0% of the saber pro cases were correctly classified when using cross validation. In cross validation, each case is classified by the functions derived from all cases other than that case.
5.3.2. Socioeconomic variables for discriminating the saber 11 performance
– Significance of the discriminating variables
31The descriptive statistics (mean and standard deviation) for all the discriminating variables in each of the saber 11 performance groups indicated differences between the values of all the variables, even greater than those observed in the saber pro case. In order to test whether those differences were significant, test of equality of groups’means were performed. According to these results, there were significant differences among the variables between the saber 11 performance groups; thus they were suitable for the discriminant analysis. One of the purposes of this part of the study was to determine the most important factor (s) used for discriminating students among academic performance groups. Table 13 shows the test of equality groups means and the Wilks’lambda for each variable. Consequently, there are small differences in the variables’potential for discriminating, but the sisben level has the smallest effect and the student stratum has the largest effect. As a matter of fact, these Wilks’lambda values are smaller than those obtained in the saber pro case; therefore, better discrimination ability of the socioeconomic variables can be expected in the saber 11 case.
32Table 14 shows the eigenvalues associated to each of the discriminant functions obtained (there are only two functions given that there are three saber 11 performance groups).
33According to Table 14, the first function accounts for 99% of the discriminant ability of the socioeconomic variables. However, the value of the squared canonical correlation (CR^{2} = 0.16) indicates a moderate association between the sets of variables, but stronger than the observed in the saber pro case (Musso, Kyndt, Cascallar & Dochy, 2013).
34Table 15 shows the Wilks’lambda obtained for each function as well as the significance value of the chisquare test. Given these results, there is evidence to expect a moderate discrimination power of the socioeconomic variables in the saber 11 case but greater than the discrimination obtained in the saber pro case. Indeed, the discriminant function does better than a random separation of the groups.
– Discriminant Functions
35Once the previous steps have been done, the equations for the discriminant functions can be written using the standardized canonical discriminant coefficients. In this case, only the equation for the Function 1 is presented, considering the insufficient discriminant ability found in the Function 2.
36D1 = 0.307 * Father Educational Level + 0.286 * Mother Educational Level + 0.393 * Student Stratum + 0.059 * sisben Level + 0.298 Family Income (2).
37As a result of Equation 2, the variables that discriminate saber 11 performance the most are student stratum, father educational level and family income. This finding is supported by the structure matrix where the strongest values (strongest correlations with Function 1) are precisely for those variables.
38With the discriminant functions obtained, the next step was to plot the group graphs to see how well the functions were discriminating the data. For each case, Function 1 and Function 2 are plotted as well as the group centroid. Figure 12 shows the graph for each saber 11 performance group, and also the three clusters plotted at the same time.
39Figure 12 shows more distance between the centroids in Function 1 than in Function 2. Such a result was expected, considering the greater discrimination ability of the first function. Moreover, there is no overlapping among the centroids, indicating a larger power of discrimination for the socioeconomic variables in the saber 11 case.
– Classification
40The classification results showed that 49.8% of the saber 11 cases were correctly classified when using cross validation. In cross validation, each case is classified by the functions derived from all cases other than that case.
5.4. Assumptions of the mda
41Multiple discriminant analysis (mda) assumes that the data come from a multivariate normal distribution and that the covariance matrices of the groups are equal. This section presents a comment related with mda assumptions.
– Multivariate normal distribution
42One possible way to test the multivariate normal distribution of the socioeconomic variables is to calculate the Mahalanobis distance (D^{2}). As reported by Rovai, Baker and Ponton (2013):
It measures the distance of a case from the centroid (multidimensional mean) of a distribution, given its covariance (multidimensional variance). A case is a multivariate outlier if the probability associated with its D^{2} is 0.001 or less. D^{2} follows a chisquare distribution with degrees of freedom equal to the number of variables included in the calculation. (p. 559)
 8 The function cdf. chisq calculates the probability of a variable which follows a chisquare distrib (...)
43Using spss8, 554 cases were found with a probability less than 0.001. In agreement with Sharma (1996), violations of the normality assumption have an effect on the classification rates of a discriminant analysis and the power of test statistics.
– Homogeneity of variance
44According to ibm Corporation (2016c): “Log determinants are a measure of the variability of the groups. Larger log determinants correspond to more variable groups. Large differences in log determinants indicate groups that have different covariance matrices.” (para. 1). Table 16 shows the log determinants for the saber 11 and saber pro performance groups.
 9 Since Box’s M test was significant, analyses using separategroups covariance matrix were performed (...)
45Box’s M9 tests the assumption of equality of covariance across groups. The null hypothesis of equal population covariance matrices is tested. As a consequence of the results presented in Table 17, there is a violation to the assumption of homogeneity of variance across saber 11 and saber pro performance groups.
5.5. Logistic Regression as an alternative to the mda
46Because of the violation of the assumptions of the mda, described in section 5.4, several logistic regression models were fitted in order to explore whether changes in the socioeconomic variables could predict performance in the saber 11 and saber pro tests. In the first attempt, the dependent variable (the performance group) was considered as truly ordered, and the socioeconomic variables were treated as categorical. Therefore, an ordinal logistic regression was carried out.
47Although the results of this model showed that for increases in the socioeconomic conditions there were increases in the performance group for both tests (saber 11 and saber pro), these results have to be discussed. For Agresti (2007): “With more than two categorical predictors, the data are counts in a multiway contingency table” (p. 159). With a large number of cells, there are many cell counts that may be small (sparseness) or equal to 0 (sampling zero). This is common in tables with many variables or when the classification has several categories. Consequently, sampling zeros could cause estimates of model parameters to be infinite; moreover, empty cells and sparse tables caused bias in estimators of odds ratios. (Agresti, 2007).
48Additionally, one of the assumptions associated with an ordinal logistic regression is the proportional odds assumption or the parallel regression assumption. “In other words (…) the coefficients that describe the relationship between (…) the lowest versus all higher categories (…) are the same as those that describe the relationship between the next lowest category and the higher categories” (ucla: Statistical Consulting Group, 2014, para. 13). For both saber 11 and saber pro, the parallel test was significant (p < 0.01). In conclusion, an ordinal logistic regression was not an appropriate alternative analysis to the mda.
49In the second attempt, a multinomial logistic regression was performed. According to Starkweather and Moske (2011) this technique “is often considered an attractive analysis because it does not assume normality, linearity, or homoscedasticity (…) Indeed, multinomial logistic regression is used more frequently than discriminant function analysis because the analysis does not have such assumptions” (p. 1).
50In the multinomial logistic regression models, fitted in this study, ses index was considered as a predictor and the saber 11 and saber pro outcomes were considered as the predicted variable.
5.5.1. Results of Multinomial Logistic Regression (mlr) for saber pro
51Table 18 summarizes the results for the mlr for the saber pro case.
52Table 18 contains the logistic coefficient (B) for each predictor variable. The logistic coefficient is the expected amount of change in the logit for changes of one unit in the predictor. According to this, when there is an increase in one unit in the ses condition there is also an increase in the odds of belonging to a higher category of saber pro. Likewise, that individual is particularly more likely to be in the high performance group (38.1%) than in the middle performance group (15.6%) when the ses index increases. The likelihood ratio test for this model was significant (chisquare = 1350.681, p < 0.001). However, only the 45.7% of the cases were correctly classified by the model.
5.5.2. Results of Multinomial Logistic Regression (mlr) for saber 11
53Table 19 summarizes the results for the mlr for the saber 11 case.
54The results presented in Table 19 show the logistic coefficient (B) for each predictor variable. Therefore when an increase in the ses condition occurs, there is also an increase in the odds of membership in a higher category of saber 11. Specifically, it is more likely to be in the high performance group (75.1%) than in the middle performance group (29.5%) when the ses index increases. The likelihood ratio test for this model was significant (chisquare = 3514.374, p < 0.001). However, only the 49.3% of the cases were correctly classified by the model.
5.5.3. Classification: Volume under the surface (vus)
55Agresti (2007) defines a receiver operating characteristic (roc) curve as: “a plot of sensitivity as a function of (1–specificity) for different possible cutoff probabilities. In fact, this curve is more informative than a classification table, because it summarizes predictive power for all possible cutoffs” (p. 143). In the classic binary case, that is the outcome with only two categories, roc analysis is based on the confusion matrix of the model. This matrix is shown in the Table 20.
56With the information presented in the Table 20, the true positive rate, the true negative rate, the roc curve and the area under the curve are obtained. As a matter of fact, the area under the curve is an indicator of how well the classification procedure was conducted. For Kang and Tian (2013): “With three ordinal diagnostic categories, the most commonly used measure for the overall diagnostic accuracy is the volume under the roc surface (vus), which is the extension of the area under the roc curve (auc) for binary diagnostic outcomes” (p. 39). In the case of this study, an optimistic estimation for the volume under the surface is proposed, considering the confusion matrix for three groups (high, middle and low). Table 21 shows the confusion matrix related with the classification of the three performance groups. The values (frequencies) presented in bold are considered as true positives, whereas the remaining values in each file are considered as false negatives for each category.
 10 All possible classifications are considered equally probable. However, future estimations with marg (...)
57Given the existence of three groups, the correct classification could be performed in six different ways (hml, hlm, mhl, mlh, lhm and lmh). Moreover, each of these ways has a probability of occurrence10. For instance, for the hml classification the probability of occurrence is given by (1/6)* P (hml)=(1/6)* Correct Classification of H * Correct Classification of M * Correct Classification of L. Thus, the probability of correct classification of H is P (H)= HH/(HH + HM + HL); considering that all the cases from category H have been classified, the probability of correct classification for M would be: P (M)= MM/(MM + ML); and as all cases of categories H and L have been classified (including the combinations LH and LM) the probability of correction classification of category L would be 1. The procedure described before can be repeated for the other ways of performing the classification in order to obtain its probability of occurrence. In the end, these six values are added, and the volume under the roc surface is obtained. The vus for the saber pro and saber 11 confusion matrices are shown in Table 22.
Table 22. vus for saber 11 and saber pro classification
vus for saber 11 
0.34 
vus for saber pro 
0.25 
58Apparently, the estimated values for the vus in each classification indicate that the ses index does a better job for classifying the saber 11 performance than the saber pro performance. Similarly, Kapasný and Rezác (2013) propose procedures for estimating the vus based on the confusion matrix, Mann Whitney statistics and empirical distribution functions. Better results are achieved using Mann Whitney statistics, but computation of these statistics for large data sets has very high computational demands.
Highlight # 2
mda and Logistic Regression did not report a correct percentage of classification larger than 50% for both saber 11 and saber pro, although more correctly classified cases were found in the saber 11 case. An increase in ses conditions, i. e., an increase in the ses index, implies an increase in the academic performance. Moreover, it is more likely to improve in the saber 11 performance than in the saber pro performance when there is an improvement in the ses conditions. The Educational Level of the Mother and Family Income were the best predictors for saber pro performance. On the other hand, the variables that discriminate saber 11 performance the most were Student Stratum and Father Educational Level. The sisben Level had a low discriminating power for both tests. In summary, the ses index does a better job of classifying the saber 11 performance than the saber pro performance.
5.6. The effects of saber 11 and ses in saber pro: manova results
59Figures 13 and 14 show the means for Critical Reading and Quantitative Reasoning plotted by saber 11 and ses Interestingly, these lines are almost parallel indicating that the effect of belonging to a different saber 11 performance group on saber pro performance is no different for students of high, middle and low ses Moreover, these lines do not intersect, thus indicating a lack of interaction between saber 11 and ses on the saber pro outcomes.
60Table 23 shows the multivariate test results for the manova. Wilks’lambda and Pillai’s trace are given. Pillai’s trace is more robust than Wilks’lambda but has less power, although it is adequate to detect true differences under different conditions (Sharma, 1996).
Table 23. Multivariate results
Pillai’s Trace 
Wilks’lambda 

saber 11 
0.367 ** 
0.637 ** 
ses 
0 ** 
1 ** 
saber 11 *ses 
0.003 ** 
0.997 ** 
Note: **= p < 0.001
61According to Table 23, there are significant differences in the Quantitative Reasoning and Critical Reading scores among the saber 11 performance groups. To illustrate this fact, Figures 13 and 14 show that students from the high saber 11 group have a higher score in Quantitative Reasoning and Critical Reading than students from the middle and low saber 11 group. In contrast, there are no differences in the dependent variables scores among the ses groups; Figures 13 and 14 show that the expected scores in the competences of saber pro are practically the same for each of the ses groups. Thus, it is possible to affirm that there are no significant differences in Quantitative Reasoning and Critical Reading scores within the nine groups involved in the interaction between saber 11 and ses Furthermore, the Pillai’s trace value for this last case reveals that only 0.3% of the variance of the dependent variables is accounted for in the variance of the independent variables.
62Table 24 shows that the significant multivariate effect of saber 11 is due to differences in the means of the two dependent variables. Conversely, the nonsignificant multivariate effect of ses is due to the nonsignificant differences in the means of the two dependent variables.
– Identifying Influential Cases: Cook’s Distance
63In Figure 15 below, Cook’s distance was plotted for the two dependent variables. Cook’s distance considers the influence of a particular case on the fitted values for all observations simultaneously. Thus, this index can be used to identify influential cases (Kutner, Nachtsheim, Neter, & Li, 2005).
64Apparently, a case with a large Cook’s distance has a large residual or a large leverage or some combination of these two. Heiberger and Holland (2004), recommend that a case be regarded as unusual if the Cook’s distance exceeds 1. In this case, none of the values plotted in Figure 15 is larger than 1.
65Contrary to this, there are arguments for a smaller threshold value given by 4/(np1) or 4/n, where n is the sample size and p represents the number of independent variables (Fox, 1991). In this case, the limit value calculated was 0.00018 and several cases had a larger distance than this value in Critical Reading and Quantitative Reasoning. In order to test whether the results changed without these cases, an additional model was fitted. However, the outcomes of this new analysis showed the same effects described earlier in section 5.6.
5.7. Assumptions of manova
– Multivariate normality
66Figure 16 shows the standardized residual values against predicted values for the two dependent variables. This plot provides a good idea of accuracy of the normality assumption; any systematic pattern is a warning. Thus, there is a confirmation that randomly distributed residuals and equal dispersion in each cell do not hold for the two dependent variables.
– Homogeneity of variance
67Tacq (1997) suggests the use of plots for the analysis of residuals and Box’s M test for the control of the homogeneity assumption. Figure 17 shows the standard deviations by means for Critical Reading and Quantitative Reasoning.
68In this case, Box’s M test was significant (p < 0,001). Particularly, the Levene’s test was significant for Quantitative Reasoning (p < 0,001), but not significant for Critical Reading (p> 0,001). Consequently, the assumption of homogeneity of variance is not fulfilled by the Quantitative Reasoning score.
69Although heteroscedasticity represents a problem for the manova, it contains interesting information related with the saber pro outcomes. For example, Figure 17 shows that the variance in the scores in Critical Reading is not significantly different among the cells. In contrast, for the Quantitative Reasoning scores it is clear that when the mean increases the standard deviation also increases, indicating that there are significant differences in the variance among the cells. This finding might be indicating that the students’variance in the Critical Reading test is more homogeneous that in the Quantitative Reasoning test. Considering that the saber pro test is taken at the end of the university, an analysis of the relation at this level between saber 11 and each of the competences measured by saber pro is presented in section 5.8.
– Comment on the manova assumptions
70Given that the assumptions of the manova were not fulfilled, the quality of the results presented above was evaluated.
71Regarding the normality, the effect of violating this assumption depends on the level of nonnormality and the statistical test. For Nimon (2012): “If the normality assumption is violated, researchers may delete outlying cases, transform data, or conduct nonparametric tests as long as the process is clearly reported” (p. 3). The variables that are nonnormal and highly skewed or kurtotic distort relationships and significance tests in linear regression (Osborne & Waters, 2002). Concerning the homogeneity of variance, Field (2009) states that:
The effect of violating the homogeneity assumption is unclear, (…) As a general rule of thumb, if sample sizes are equal then disregard Box’s test, because (1) it is (highly) unstable and (2) we can assume that Hotelling’s and Pillai’s statistics are be robust. However, if group sizes are different, then robustness cannot be assumed (p. 604)
72Additionally, Tabachnick and Fidell (2001) suggest that if the larger samples produce greater variances and covariances then the probability values will be conservative and so significant findings can be trusted.
Highlight # 3
There are significant differences in the Critical Reading and Quantitative Reasoning outcomes between the saber 11 performance groups. In fact, the highest scores for the saber pro competences were estimated for the saber 11 high performance group. In contrast, there are no significant differences in the Critical Reading and Quantitative Reasoning outcomes between the ses index groups. Moreover, the effect of belonging to a different saber 11 performance group on saber pro outcomes does not differ for students of high, middle and low ses.
5.8. Relation between saber 11 and saber pro across universities: A Multilevel approach
5.8.1 Measuring Quantitative Reasoning and Critical Reading in saber 11
73Before establishing the relationship between saber 11 and saber pro across universities, it was necessary to propose a quantification for the competences measured by the saber 11 test. Instead of using the eight saber 11 scores directly (Mathematics, Physics, Spanish, Social sciences, English, Biology, Chemistry and Philosophy), two components were extracted and compared with the competences measured by saber pro test. Therefore, an Exploratory Factor Analysis (efa) was performed. As reported by DiStefano, Zhu and Mindrilla (2009):
There are two main classes of factor score computation methods: refined and nonrefined. Nonrefined methods are relatively simple, cumulative procedures to provide information about individuals’placement on the factor distribution. Refined computation methods create factor scores using more sophisticated and technical approaches. They are more exact and complex than nonrefined methods and provide estimates that are standardized scores. (p. 2)
74One of the refined methods is the AndersonRubin Scores, where the resulting factor scores are orthogonal, with a mean of 0 and a standard deviation of 1 (ibm Corporation, 2016d). Once the factor scores were obtained, using the AndersonRubin method, those values were converted using the linear transformation: F = af + b, where f is a factor score, a is the needed standard deviation and b is the wanted mean. In the case of this study, the linear transformation F = f + 10, returned values with mean 10 and standard deviation of 1. saber pro test is measured in the same scale, i. e., in a scale with a mean of 10 and a standard deviation of 1.
75Figure 18 shows the factor plot for this model, where the solution space with two factors clearly appears. For this case, correlation coefficients in the Correlation Matrix were greater than 0.3, KaiserMeyerOlkin (kmo) was 0.831 and the Bartlett’s Test of Sphericity was significant (p < 0,001). Specifically, Physics, Chemistry and Mathematics are better located in the first factor (larger values in Factor 1), and Social sciences and Spanish are better located in the second factor (larger values in Factor 2). Consequently, it was decided to rename the Factor 1 as Quantitative Reasoning in saber 11; and the Factor 2 as Critical Reading in saber 11.
5.8.2 Comparison for Quantitative Reasoning across Universities
76Table 25 provides a summary of the four multilevel models performed for the Quantitative Reasoning case, using the saber pro outcomes as the response variable.
Table 25. Summary of Multilevel modeling for Quantitative Reasoning
Significance level: *= p < 0.05;**= p < 0.01;***= p < 0.001
77Model 1: This model does not include explanatory variables, apart from the constant (i. e. representing the intercept) which is allowed to vary randomly across universities and with the levels defined as students (Level 1) and universities (Level 2). The following are differences among universities in the level of achievement in Quantitative Reasoning. The overall mean for saber pro in this case is 10.326. The total variation around this mean is 1.03. The proportion of variance at the university level is 17.66%. Assuming normality, 95% of universities lie between 9.49 and 11.162. Consequently, there are differences among universities in achievement in Quantitative Reasoning.
78Model 2: This model is Model 1 with saber 11 as an explanatory variable. This model looks at whether universities give added value to the progress in Quantitative Reasoning. The general rate of progress in Quantitative Reasoning across all universities is 0.477. There is still variation among universities after taking saber 11 into account, but it has decreased. The variance at student level also decreased. The proportion of variance at the university level for the Model 2 is 8.88%. Assuming normality, 95% of universities lie between 9.895 and 10.901. In fact, a typical student starts university with a Quantitative Score of 10, then progresses to 10.398. In the top 2.5% of the universities studied the average progress is up to 10.901. In contrast, in the bottom 2.5% of the universities studied the average score is 9.895.
79Model 3: This model is Model 2, with saber 11 varying randomly across universities. This model looks at whether there are differences among universities for the average student upon entry. The average saber pro score for all universities is 10.375. The general rate of progress across all universities is 0.447. The proportion of variance at the university level now is a function of saber 11. There are three terms at Level 2 representing the variancecovariance for universities. There is a significant variance among universities for saber 11 and saber pro scores. The covariance between the random intercepts and slopes is positive and significant, meaning that there is a fanning out effect.
80Model 4: This model is Model 3, with saber 11 varying randomly across universities and at student level. Figure 19 shows the predictions for saber pro according to the Model 4. This figure also shows the scatterplot of the saber pro by the saber 11 in a single level solution (only student level). Therefore, a multilevel approach seems to be more appropriate to analyze the relation between the two tests.
81For Model 4, Figure 20 shows in green the variance function at Level 1 (students) and in blue the variance function at Level 2 (universities). Likewise, 95% confidence bounds for the two levels are presented. This figure also shows in red the proportion of variance at university level as a function of saber 11. For all the cases the saber 11 score was centered on its mean, indicated by a dotted line at 0. A detailed analysis of Figure 20 reveals remarkable facts. For example, there is not a constant variance at Level 1; thus, the assumption of students’homoscedasticity can be rejected. In fact, it is evident that the variance in saber pro at student level increases in a quadratic way for the students who scored higher on saber 11. The 95% confidence bounds at both levels show that there is more variance for the students who scored higher than for the least able students and the fanning out effect is also visualized. The Variance Partitioning Coefficient (vpc) function shows that there is more variance at university level when the prior performance (saber 11) is better. Indeed, all these findings could explain why the Levene’s Test in the manova model was significant (see section 5.7). To sum up, the most able students have a greater variance in progress in Quantitative Reasoning than the least able students.
5.8.3 Comparison for Critical Reading across Universities
82Table 26 presents a summary of the four multilevel models performed for the Critical Reading case, using the saber pro outcomes as the response variable.
Table 26. Summary of Multilevel modeling for Critical Reading
Significance level: *= p < 0.05;**= p < 0.01;***= p < 0.001
83Model 5: This model does not include explanatory variables, apart from the constant (i. e. representing the intercept) which is allowed to vary randomly across universities and with the levels defined as students (Level 1) and universities (Level 2). The model looks at whether a difference exists among universities in achievement in Critical Reading. The overall mean for saber pro in this case is 10.375. The total variation around this mean is 0.902. The proportion of variance at the university level is 15.29%. Assuming normality, 95% of universities lie between 9.647 and 11.103. Consequently, there are differences among universities in achievement in Critical Reading.
84Model 6: This model is Model 5 with saber 11 as an explanatory variable. This model looks at whether universities give added value to the progress in Critical Reading. The general rate of progress in Critical Reading across all universities is 0.382. There is still variation among universities after taking saber 11 into account, but it has decreased. The variance at student level has also decreased. The proportion of variance at the university level for Model 6 is 7.31%. Assuming normality, 95% of universities lie between 9.98 and 10.873. In fact, a typical student starts with a saber 11 score of 10, and then progresses to 10.431. In the top 2.5% of the universities studied the average progress is up to 10.873. In contrast, in the bottom 2.5% of universities studied the average score is 9.98.
85Model 7: This model is Model 6 with saber 11 varying randomly across universities. This model looks at whether there are differences among universities for the average student upon entry. The average saber pro score for all schools is 10.450. The general rate of growth across all universities is 0.397. The proportion of variance at the university level now is a function of saber 11. There are three terms at Level 2 representing the variance and covariance for universities. There is a significant variance among universities for saber 11 and saber pro scores. The covariance between the random intercepts and slopes is negative and significant, meaning that there is a fanningin effect.
86Model 8: This model is Model 7, with the saber 11 varying randomly across universities and at student level. This model looks at for which students university makes the most difference. Figure 21 shows the predictions for saber pro according to the Model 8 on the left side. This figure also shows the scatterplot of the saber pro by the saber 11 in a single level solution (only student level). Therefore, a multilevel approach seems to be more appropriate to analyze the relation between the tests.
87For Model 8, Figure 22 shows, in green, the variance function at Level 1 (students) and, in blue, the variance function at Level 2 (universities). Likewise, the 95% confidence bounds for the two levels are presented. This figure also presents, in red, the proportion of variance at university level as a function of saber 11. For all the cases the saber 11 score was centered on its mean, indicated by a dotted line at 0. The review of the Figure 22 reveals notable facts. For instance, there is not a constant variance at level 1; thus, the assumption of students’homoscedasticity can be rejected. However, the variance function at student level seems to be more constant in this case and the variance seems to increase by a small amount, but only for those students who scored higher on saber 11. The 95% confidence bounds show that the variance is practically constant at student level but is larger for the least able students at university level, where the fanningin effect is also visualized. The Variance Partitioning Coefficient (vpc) function shows that there is more variance at university level when the prior performance (saber 11) is lower. All these findings could explain why the Levene’s Test in the manova model was not significant (see section 5.7). In conclusion, the least able students have higher variance in progress in Critical Reading than the most able students.
5.8.4 saber pro and ses across Universities
88Table 27 presents a summary of the two multilevel models performed for the saber pro outcomes using the ses index as explanatory variable.
89Model 9: In this model, the ses index varies randomly across universities and at student level. Figure 23 shows on the left side the predictions for Quantitative Reasoning according to Model 9. This figure also shows the scatterplot of the saber pro by the ses In the latter situation, no distinction is made among the universities to which the students belong.
90For Model 9, Figure 24 shows, in green, the variance function at Level 1 (students) and, in blue, the variance function at Level 2 (universities). Likewise, the 95% confidence bounds for the two levels are presented. This figure also includes, in red, the proportion of variance at university level as a function of the ses index. For all the cases the ses index was centered on its mean, indicated by a dotted line at 0.
91Figure 24 illustrates that the variance at university level is practically constant, showing only a slight increase for the highest values of the ses index. By contrast, variance at Level 1 shows a quadratic function, changing faster for wealthier students. Compared to Model 4, where saber 11 was used as a predictor, the variance functions have the same shape, but in this case the 95% confidence bounds show a constant change among the ses index, where only a slight increase for the students with the best socioeconomic conditions can be expected. The absolute values of the VPC illustrate the small value of the variance at Level 2 across the ses index.
92Model 10: In this model, the ses index varies randomly across universities and at student level. Figure 25 shows on the left side the predictions for Critical Reading according to Model 10. This figure also shows the scatterplot of the saber pro by the ses. In the latter situation, no distinction is made among the universities to which the students belong.
93For Model 10, Figure 26 shows, in green, the variance function at Level 1 (students) and, in blue, the variance function at Level 2 (universities). Likewise, the 95% confidence bounds for the two levels are presented. This figure also includes, in red, the proportion of variance at university level as a function of ses index. For all the case the ses index was centered on its mean, indicated by a dotted line at 0. Figure 26 illustrates that the variance at both levels is practically constant. However, it shows a minor decrease for the highest values of the ses index at Level 2. Compared to Model 8, where saber 11 was used as a predictor, the variance functions have the same pattern, but in this case the 95% confidence bounds show a constant change among the ses index. The vpc illustrates that there is more variance at Level 2 for students with a low ses index.
5.9. Assumptions of Multilevel
94As reported by Mass and Hox (2003):
The assumptions underlying the multilevel model are similar to the assumptions in multiple regression analysis: linear relationships, homoscedasticity, and normal distribution of the residuals (…) the maximum likelihood estimation methods used commonly in multilevel analysis are asymptotic, which translates to the assumption that the sample size is large. (p. 428)
 11 For a complete discussion on this topic please refer to Bell, B. A., Ferron, J. M., & Kromrey, J. D (...)
95A common rule of thumb about sample size11 for multilevel models is that there must be at least 20 groups and at least 30 observations per group (Heck & Thomas, 2000). A technique for graphically assessing the normality at each level is to generate a normal score plot. Similarly, a plot of the standardized Level 1 residuals against the fixed part serves as a useful tool for detecting outliers, heterogeneity and nonlinearity (Jones, 2013). Appendix 3 presents the complete MLWin outputs for these plots, as well as the number of students within each university.
Highlight # 4
There are differences between universities in achievement in Quantitative Reasoning; specifically, universities appear to add value to progress in Quantitative Reasoning for the most able students. As a matter of fact, the most able students vary more than the least able students at the university level. There are also differences between universities in achievement in Critical Reading; specifically, universities appear to add value to progress in Critical Reading for the least able students. Moreover, the least able students are varying more than the most able students at the university level. In relation to the ses index, students from different socioeconomic conditions vary in practically the same manner at the university level in both Quantitative Reasoning and Critical Reading. In fact, the least wealthy students have more variance in Critical Reading and the wealthiest students vary more in Quantitative Reasoning.
Notes
4 It is important to point out that several attempts of catpca and efa were performed before doing the mca. However, those techniques did not allow the use of the parents’ occupation level in a proper way due to the measurement level (nominal), deriving in a lower percentage of explained variance.
5 Although twostep clustering was also used for this classification, the results found were considered similar. In terms of simplicity it was decided to work with the Kmeans procedure instead.
6 Symmetric measures of ordinal association are based on the idea of accounting for concordance versus discordance. Each pairwise comparison of cases is classified as one of the following:
–A pairwise comparison is considered concordant if the case with the larger value in the row variable also has the larger value in the column variable. Concordance implies a positive association between the row and column variables.
–A pairwise comparison is considered discordant if the case with the larger value in the row variable has the smaller value in the column variable. Discordance implies a negative association between the row and column variables.
–A pairwise comparison is considered tied on one variable if the two cases take the same value on the row variable, but different values on the column variable (or vice versa). Being tied on one variable implies a weakened association between the row and column variables.
–A pairwise comparison is considered tied if the two cases take the same value on both the row and column variables. Being tied implies nothing about the association between the variables (ibm Corporation, 2016a, para. 1).
7 Notice that parents’occupation was measured as nominal variable. A dummy coding was used in order to integrate this variable into the discriminant analysis; however this additional model did not show a better percentage of correct classification.
8 The function cdf. chisq calculates the probability of a variable which follows a chisquare distribution like D^{2}. Given this function computes the cumulative probability, the result was subtract from 1 to obtain the probability in the upper tail of the distribution.
9 Since Box’s M test was significant, analyses using separategroups covariance matrix were performed to see whether classification changed. The results showed a lower percentage of correct classification for both saber 11 and saber pro.
10 All possible classifications are considered equally probable. However, future estimations with marginal probabilities or Bayesian approach could lead to better results for the vus.
11 For a complete discussion on this topic please refer to Bell, B. A., Ferron, J. M., & Kromrey, J. D. (2008). Cluster size in multilevel models: the impact of Sparse Data Structures on Point and Interval Estimates in TwoLevel models. JSM Proceedings, Section on Survey Research Methods, 11221129.
Table des illustrations
Titre  Table 1. Inertia and adjusted inertia of the mca 

URL  http://books.openedition.org/uec/docannexe/image/1516/img1.jpg 
Fichier  image/jpeg, 56k 
Titre  Figure 6. mca solution space 
URL  http://books.openedition.org/uec/docannexe/image/1516/img2.jpg 
Fichier  image/jpeg, 20k 
Titre  Table 3. Descriptive statistics of the ses index 
URL  http://books.openedition.org/uec/docannexe/image/1516/img3.jpg 
Fichier  image/jpeg, 56k 
Titre  Figure 7. Histogram for ses index 
URL  http://books.openedition.org/uec/docannexe/image/1516/img4.jpg 
Fichier  image/jpeg, 76k 
Titre  Table 4. Frequencies of saber pro clusters 
URL  http://books.openedition.org/uec/docannexe/image/1516/img5.jpg 
Fichier  image/jpeg, 16k 
Titre  Table 5. Frequencies of saber 11 clusters 
URL  http://books.openedition.org/uec/docannexe/image/1516/img6.jpg 
Fichier  image/jpeg, 16k 
Titre  Table 6. Frequencies of ses clusters 
URL  http://books.openedition.org/uec/docannexe/image/1516/img7.jpg 
Fichier  image/jpeg, 48k 
Titre  Figure 8. Relation between saber pro and saber 11 
URL  http://books.openedition.org/uec/docannexe/image/1516/img8.jpg 
Fichier  image/jpeg, 16k 
Titre  Table 7. Contingency table for saber 11 and saber pro 
URL  http://books.openedition.org/uec/docannexe/image/1516/img9.jpg 
Fichier  image/jpeg, 148k 
Titre  Figure 9. Relation between saber 11 and ses 
URL  http://books.openedition.org/uec/docannexe/image/1516/img10.jpg 
Fichier  image/jpeg, 16k 
Titre  Table 8. Contingency table for saber 11 and ses 
URL  http://books.openedition.org/uec/docannexe/image/1516/img11.jpg 
Fichier  image/jpeg, 148k 
Titre  Figure 10. Relation between saber pro and ses 
URL  http://books.openedition.org/uec/docannexe/image/1516/img12.jpg 
Fichier  image/jpeg, 16k 
Titre  Table 9. Contingency table for saber pro and ses 
URL  http://books.openedition.org/uec/docannexe/image/1516/img13.jpg 
Fichier  image/jpeg, 44k 
Titre  Table 10. Wilks’Lambda for socioeconomic variables. 
Légende  Note: **=p<0.001 
URL  http://books.openedition.org/uec/docannexe/image/1516/img14.jpg 
Fichier  image/jpeg, 64k 
Titre  Table 11. Eigenvalues for saber pro 
URL  http://books.openedition.org/uec/docannexe/image/1516/img15.jpg 
Fichier  image/jpeg, 40k 
Titre  Table 12. Wilks’lambda for discriminating functions 
URL  http://books.openedition.org/uec/docannexe/image/1516/img16.jpg 
Fichier  image/jpeg, 12k 
Titre  Figure 11. Group graphs for saber pro 
URL  http://books.openedition.org/uec/docannexe/image/1516/img17.jpg 
Fichier  image/jpeg, 28k 
Titre  Table 13. Wilks’lambda for socioeconomic variables. 
Légende  Note: **=p<0.001 
URL  http://books.openedition.org/uec/docannexe/image/1516/img18.jpg 
Fichier  image/jpeg, 60k 
Titre  Table 14. Eigenvalues for saber 11 
URL  http://books.openedition.org/uec/docannexe/image/1516/img19.jpg 
Fichier  image/jpeg, 40k 
Titre  Table 15. Wilks’lambda for discriminating functions 
URL  http://books.openedition.org/uec/docannexe/image/1516/img20.jpg 
Fichier  image/jpeg, 40k 
Titre  Figure 12. Group graphs for saber 11 
URL  http://books.openedition.org/uec/docannexe/image/1516/img21.jpg 
Fichier  image/jpeg, 24k 
Titre  Table 16. Log determinants for saber pro and saber 11 
URL  http://books.openedition.org/uec/docannexe/image/1516/img22.jpg 
Fichier  image/jpeg, 36k 
Titre  Table 17. Box’s M test summary 
URL  http://books.openedition.org/uec/docannexe/image/1516/img23.jpg 
Fichier  image/jpeg, 60k 
Titre  Table 18. Multinominal Logistic Regression for saber pro 
Légende  Reference category is Low. *** p<0.001 
URL  http://books.openedition.org/uec/docannexe/image/1516/img24.jpg 
Fichier  image/jpeg, 48k 
Titre  Table 19. Multinominal Logistic Regression for saber 11 
Légende  Reference category is Low. *** p<0.001 
URL  http://books.openedition.org/uec/docannexe/image/1516/img25.jpg 
Fichier  image/jpeg, 48k 
Titre  Table 20. Confusion matrix for 2 categories 
URL  http://books.openedition.org/uec/docannexe/image/1516/img26.jpg 
Fichier  image/jpeg, 32k 
Titre  Table 21. Confusion Matrix for 3 categories 
URL  http://books.openedition.org/uec/docannexe/image/1516/img27.jpg 
Fichier  image/jpeg, 40k 
Titre  Figure 13. Means for Critical Reading 
URL  http://books.openedition.org/uec/docannexe/image/1516/img28.jpg 
Fichier  image/jpeg, 16k 
Titre  Figure 14. Means for Quantitative Reasoning 
URL  http://books.openedition.org/uec/docannexe/image/1516/img29.jpg 
Fichier  image/jpeg, 56k 
Titre  Table 24. Multivariate effects 
Légende  Note: **= p<0.001 
URL  http://books.openedition.org/uec/docannexe/image/1516/img30.jpg 
Fichier  image/jpeg, 40k 
Titre  Figure 15. Cook’s distance for Critical Reading and Quantitative Reasoning 
URL  http://books.openedition.org/uec/docannexe/image/1516/img31.jpg 
Fichier  image/jpeg, 12k 
Titre  Figure 16. Standardized residuals and predicted values 
URL  http://books.openedition.org/uec/docannexe/image/1516/img32.jpg 
Fichier  image/jpeg, 84k 
Titre  Figure 17. Standard deviations by means 
URL  http://books.openedition.org/uec/docannexe/image/1516/img33.jpg 
Fichier  image/jpeg, 16k 
Titre  Figure 18. Factor plot for saber 11 
URL  http://books.openedition.org/uec/docannexe/image/1516/img34.jpg 
Fichier  image/jpeg, 52k 
Titre  Table 25. Summary of Multilevel modeling for Quantitative Reasoning 
Légende  Significance level: *= p < 0.05;**= p < 0.01;***= p < 0.001 
URL  http://books.openedition.org/uec/docannexe/image/1516/img35.jpg 
Fichier  image/jpeg, 112k 
Titre  Figure 19. Model 4 for Quantitative Reasoning 
URL  http://books.openedition.org/uec/docannexe/image/1516/img36.jpg 
Fichier  image/jpeg, 32k 
Titre  Figure 20. Variance Functions for Quantitative Reasoning 
URL  http://books.openedition.org/uec/docannexe/image/1516/img37.jpg 
Fichier  image/jpeg, 28k 
Titre  Table 26. Summary of Multilevel modeling for Critical Reading 
Légende  Significance level: *= p < 0.05;**= p < 0.01;***= p < 0.001 
URL  http://books.openedition.org/uec/docannexe/image/1516/img38.jpg 
Fichier  image/jpeg, 108k 
Titre  Figure 21. Model 8 for Critical Reading 
URL  http://books.openedition.org/uec/docannexe/image/1516/img39.jpg 
Fichier  image/jpeg, 28k 
Titre  Figure 22. Variance Functions for Critical Reading 
URL  http://books.openedition.org/uec/docannexe/image/1516/img40.jpg 
Fichier  image/jpeg, 28k 
Titre  Table 27. Summary of ses models 
Légende  Significance level: *= p < 0.05;**= p < 0.01;***= p < 0.001 
URL  http://books.openedition.org/uec/docannexe/image/1516/img41.jpg 
Fichier  image/jpeg, 120k 
Titre  Figure 23. Model 9 for Quantitative Reasoning 
URL  http://books.openedition.org/uec/docannexe/image/1516/img42.jpg 
Fichier  image/jpeg, 32k 
Titre  Figure 24. Variance Functions for Quantitative Reasoning 
URL  http://books.openedition.org/uec/docannexe/image/1516/img43.jpg 
Fichier  image/jpeg, 32k 
Titre  Figure 25. Model 10 for Critical Reading 
URL  http://books.openedition.org/uec/docannexe/image/1516/img44.jpg 
Fichier  image/jpeg, 32k 
Titre  Figure 26. Variance Functions for Critical Reading 
URL  http://books.openedition.org/uec/docannexe/image/1516/img45.jpg 
Fichier  image/jpeg, 25k 
© Universidad externado de Colombia, 2016
Conditions d’utilisation : http://www.openedition.org/6540
Lire
Freemium
Offert par L’éditeur de ce site