## Midterm

Measurement rules for assigning numbers to objects and properties of objects
psychological assessment collecting information for making behavioral predictions
nominal scale naming scale, each number takes on meaning of verbal label, not treated as numbers
ex. numbers on sports teams
ordinal scale tells us order in which people stand, but not how much separates them, 1st, 2nd, 3rd (difference between 1st and 2nd isn't necessarily same as difference between 2nd and 3rd)
interval scale equal numerical differences represent equal differences in the trait
ex (temperature, score on a test)
ratio scale when scale is constructed so a score of 0 means none of a trait, allow us to make proportional statements about a scale
ex( taking 10 minutes on a test is 2x as long as taking 5 minutes on a test)f
z-score the distance between his raw score and mean/standard deviation
(X-M)/(SD)
Mode the score that occurs most often
Median the middle score when all scores are listed smallest to largest
Mean the average score
norm referenced test having peers provide a standard, quality of performance is determined by comparison with others scores, designed to compare/rank individuals in relations to others
Criterion referenced test using a predetermined standard to determine the level of mastery on a specific skill, deliberately constructed to yield measurements that are directly interpretable in terms of specified performance standards,
content validity assessment of whether a test contains appropriate content and requires that appropriate processes be applied to that content, the extent to which a measure represents all facets of a given social construct.
Consists of face and logical validity
face validity the appearance of reasonableness of a test (does it seem like it measures what it should, does a math test measure your knowledge of mathematics or reading)
Logical validity does the measure adequately sample what is needed to measure the thing? Depends on expert consensus , want to ensure its comprehensively mapped
criterion-related validity consists of two facets:
predictive and concurrent validity
is said to have this when the test has demonstrated its effectiveness in predicting criterion or indicators of a construct.
predictive validity if the test info is used to forecast future criterion performance (delay between collection and use as in SAT to predict GPA)
extent to which score on assessment predicts future performance
concurrent validity scores on test and criterion are obtained at about the same time
construct validity extent to which assessment accurately assesses the construct
"Are we measuring what we thing we are?"
3 parts: group contrasts, correlational studies, experimental studies
construct underrepresentation a threat to construct validation, occurs when the scope of test is too narrow and fails to include impt aspects of the construct
convergent validity the degree to which two measures of constructs that theoretically should be related, are in fact related
ex: test the CV of a measure of self-esteem, researcher show that measures of similar constructs, such as self-worth, confidence, are also related
discriminant validity tests whether concepts or measurements that are supposed to be unrelated are, in fact, unrelated
ex: intelligence should not correlate to height
construct-irrelevant test variance a threat to construct validation, refers to presence of reliable variance that is extraneous to the construct being quantified (c-it difficult-something makes test harder than it should be like anxiety
c-it easy- something makes test easier)
nomological net a representation of the constructs of interest, their observable manifestations, and the interrelationships among and between these. a framework to organize construct validity
what are the 3 primary sources of a nomological net? group contrasts
correlational studies
experimental studies
What are the 2 main components of a nomological net? construct representation
nomothethic span
construct representation underlying theoretical construct that explains the empirical
nomothetic span network of correlates and empirical findings
Compare and contrast the construct representation and the nomothetic span. both are parts of the nomological net, the construct representation is closely tied to the theoretical understanding of a construct while the nomothetic span encompasses the correlates and empirical findings
the "twisted pear" a heteroscedastic plotting of data points where the data closer to the origin have a higher correlation and predictability than those further from the origin which are spread out further than the regression line
homoscedasticity can predict uniformly along the regression line, if range is restricted, the correlation DECREASES
heteroscedasticity observed y scores around the regression line are not uniform, a restricted range INCREASES the correlation
standard error of estimate SD of observed scores around regression line
multiple regression uses several explanatory variables to predict the outcome of a criterion variable.
(College GPA predicted by SAT and HS GPA)
suppressor variable has 4 traits: independent of criterion variable, (-)Beta weight in regression equation, significantly correlated with predictor variable, increases multiple R2 significantly
(accounts for some noise in an equation and makes the other variable better)
coefficient of determination R^2 portion of the criterion variable that can be accounted for by the predictor variables (part of circle that overlaps and explains it)
coefficient of non determination 1-R^2 portion of the criterion variable that cannot be accounted for by the predictor variables (part where circles don't over lap)
coefficient of alienation the sqrt of the coefficient of nondetermination (sqrt 1-R^2)
z-scores->SEE
how much can you reduce the confidence intervals around y predicted below chance (1)
0=perfect prediction 1=no validity
Incremental validity (via multiple regression) ?
Incremental validity (via Taylor-Russell Tables) Hit Rate-Base Rate
T-R: Base rate how successful the original screening measures are at effectively choosing
T-R: Selection ratio what percentage of the total "applicant pool" how many you must select
T-R: False Positive accepting an applicant who fails
T-R: True Positive accepting an applicant who succeeds
formative evaluation assessment to guide future instruction
summative evaluation provides a summary of student achievement
mental measurements yearbook publication from Nebraska U begun by Oscar Buros to critically review tests/questionnaires to help determine their quality
rival hypothesis hypothesis that compete with traditional views, alternative explanation for a phenomenon
meta-analysis combines information from many studies to make the confidence intervals smaller and more precise
validity generalization capacity of an instrument to generalize to other settings (cross cultural, to different schools, different regions) the study determines the bounds of the generalizability
Compare and contrast the two types of criterion related validity. Predictive validity and concurrent validity, both are important parts of criterion related validity and both are related to time. Predictive validity involves a temporal gap between when collected and compared while concurrent
compare and contrast convergent and discriminant validity both are necessary aspects for a nomological net to be accurate
convergent-the things which should be related are
Discriminant-unrelated variables do not have strong correlation
Can range truncation increase a validity coefficient? If so how? Yes! In the case of heterosckedastic distributions, a smaller range has a higher correlation
Why does the construct validation process never end? you can always come up with additional relevant studies that could contribute to the construct and further support or reduce it
What is a nomological net and what is it good for? A frame work to organize construct validity, consists of both the theoretical and empirical information related to a construct, has 3 primary sources (group contrasts, correlational studies, experimental studies) never ending
Discuss utility of Taylor-Russell tables. What kinds of info do you need to use them? Precisely estimate proportion of true +/- and false +/- that assessment procedure gives you over what is currently in place. Need BR, SR, VC to use
What is a suppressor variable, what do they do, how do you know you have one? A supressor variable, when controlled for, strengthens the relationship between the other variables, it can detract from the actual relationships (verbal skills in test taking to determine pilot success)
How do you know when you have a suppressor variable? significant correlation to predictor variable
independent of criterion variable
-Beta weight in regression equation
increases multiple R2 significantly