stud
bud
Sign up
Log in
Home
Library
Explore
SAT
More
Program Evaluation Flashcards | StudBud
Explore
/
Program Evaluation
Program Evaluation
49 terms
·
StudBud Administration
·
44 plays
Imported from external platform
Flashcards
49 cards
F
Practice quiz
P
Host live
L
Questions
49
Answers
1
the null hypotheses is rejected at the .01 level. This means that the researcher is safe to conclude that:
99% sure that the difference between the two groups is not due to chance
The results of the study are highly replicable.
there is a large magnitude of effect size
the results are inconclusive
2
collecting Inter-observer Agreement (IOA) data in an intervention study it is important that data is collected at least:
20 – 25% of both baseline and intervention sessions
15 – 20% of baseline sessions
at least 30% of sessions but no more than 50%
10 – 15 % of intervention sessions
3
which of the following would represent legitimate goals during the “establishing boundaries” phase:
getting paid
determine the availability of resources and timelines for completion
determine whether the evaluation should be conducted
Determine the purpose of the evaluation
4
consists of baseline followed by treatment
ABA Design
BA design
ABAB design
AB design
5
Which of the following threats to internal validity result in greater similarities between groups:
Resentful demoralization of the control group
Maturation
Diffusion of treatment
Selection
6
Scientific theories should have the following characteristics:
falsifiability and testability
believability and flexibility
complexity and falsifiability
Boldness and depth
7
to measure internal consistency of the items on a test it you must administer the test at two times to the same sample
True
False
8
The research methods used within individual studies do not have much impact on a systematic research synthesis.
False
True
9
if you find a significant difference between groups on pre-test scores you should
run an ANCOVA
throw out the data
run an ANOVA
run a t-test
10
A teacher believes that all children have a special way that they learn stemming from her mother.
authority
rationalism
Tenacity
empiricism
11
The omnibus score tells us the total difference between means.
True
False
12
Test-re-test reliability data would typically be captured by which statistic:
coefficient of equivalence
Coefficient of stability
Testing coefficient
Chi-square statistic
13
refers to the ability to generalize to an entire population
population validity
internal validity
reliability
ecological validity
14
When data are not normally distributed then a non-parametric statistical test should be utilized
True
False
15
Deductive theories use theory to predict findings while inductive approaches use current data to develop theories
True
False
16
examine the effectiveness of a program where you compare pre and post test means to an achievement test
t-test of single means
an independent t-test
dependent/correlated test
MANOVA
17
When selecting “standards” to be used to assess implementation and outcome goals of a program, this is considered:
Stakeholders should have input on the standards selected
Standards should reflect the intent of the program
Standards should reflect outcomes that program staff can impact
Stakeholders should not have input on the standards selected
18
When conducting a program evaluation a logic models helps evaluators:
Understand who “logically” is in charge of the program.
Collect stakeholder input
Understand theory demonstrating the link b/w inputs, outputs, and outcomes
Enhance their logical thinking
19
When an event is “unpredicted”, determinism would suggest that the causes resulting in the event were random in nature.
True
False
20
includes a baseline period, a treatment period, and a withdrawal of treatment
ABAB design
BAB design
ABA design
AB design
21
Cultural test bias may show up in
Beta coefficient
Coefficient of stability
Bias coefficient
Coefficient of equivalence
22
Which of the following is an example of a “grey” literature source:
Peer reviewed journal
a presentation from a conference
a book chapter
none of the above
23
Why is it important to search multiple databases when conducting a research synthesis?
we shouldn't
ability to combine findings from different research to form theories
some databases are unreliable
24
Being a critical consumer of research is only important for those practitioners working in research settings.
True
False
25
interested in knowing whether a particular achievement test could consistently measure achievement.
Validity data
all of the above
inter-observer agreement data
reliability data
26
Which of the following are characteristic of systematic research syntheses:
Procedures used to analyze the results are described.
They are only systematic if conducted within a test lab
Procedures used to select the studies are specified
27
effect sizes may be difficult to interpret if the dependent measures that are used across studies are not equivalent
True
False
28
Systematic variance is largely comprised of which of the following two factors:
none of the above
Variance restriction and statistical restriction
Individual influences and measurement error
The independent variable and confounding variables
29
The researcher wants to ensure an equal distribution of participants from all grades
stratified random sampling
random sampling
convenience sampling
systemic sampling
30
Which of the following is not a step in the scientific method:
Identify a problem
All of the above are steps of the process.
Define the problem
Formulate a hypothesis or research question
31
what design should you use when an intervention is already in place
BC
BAB
ABAB
AB
32
A researcher compares two different populations to determine if a test has construct validity
Misguided
Differential population study
Disability template approach
Related measures approach
33
Unsystematic variance is that portion of variance in a study that is largely unavoidable.
False
True
34
ABA design with additional treatment repeated after withdrawal phase
BAB
ABAB
BCB
BABA
35
Which of the following is not a method for determining whether your literature review includes high quality studies:
Determine whether the authors have published at least 3 additional articles
Determine whether the articles are from top journals in a field
Critically review the article yourself
Examine the impact factor of the article, if available
36
refers to the ability to generalize to different environments.
population validity
ecological validity
external validity
internal validity
37
When exploring psychological phenomenon, complex theories provide more explanatory power and are therefore preferred
True
False
38
fails to reject the null hypothesis when the null hypothesis should have been rejected
measures were not valid
Type 1 error
Type II error
correct decision
39
What is an argument against the use of effect sizes with single subject design studies?
effect sizes are difficult to interpret due to differing dependent measures
effect sizes tend to be overestimated in single subject design studies
effect sizes are easily manipulated in single subject design studies
effect sizes are not interpretable in single subject design studies
40
We say that a study has high internal validity when there is a causal relationship between the iv and the dv
True
False
41
staying apprised of the latest research findings to incorporate evidenced based practices
Rationalist
Scientist-practitioner
Idealist
Research psychologist
42
interested in examining the effects of two or more iv then the researcher should use a factorial ANOVA
True
False
43
an estimate of error associated with an individual’s obtained score
Standard error of measurement
true error
Beta coefficient
error variance
44
When replication research is conducted by a group that is independent from the group that originally did the research
reciprocal replication
Clinical replication
systematic replication
direct replication
45
During a study students experience a traumatic event in their school. What is the threat to internal validity?
maturation
mortality
History
instrumentation
46
effect sizes represent a standardized measure of difference between a control group and an experimental group
True
False
47
If a measure is valid then it must also be reliable
True
False
48
How can you use research methods to better the services that you provide to clients?
Progress monitoring of interventions
generating new knowledge
none of the above
evidenced based interventions
49
A program evaluator who is most interested in the extent to which a program is actually being implemented needs:
outcome data
Process data
Needs assessment data
efficiency data
