stud
bud
Sign up
Log in
Home
Library
Explore
SAT
More
DgXp Week 6 Quiz TXA-TXB Flashcards | StudBud
Explore
/
DgXp Week 6 Quiz TXA-TXB
DgXp Week 6 Quiz TXA-TXB
11 terms
·
StudBud Administration
Imported from external platform
Flashcards
11 cards
F
Practice quiz
P
Host live
L
Questions
11
Answers
1
What are two possibilities of allocating users in an A/B test
choice menu
use of redirects
none of the above
server backend solution
2
What does it mean when a "fly" organizations "tracks the counterfactual"
Tracking the user on a different day.
Recording what the users saw in the Control Group.
Tracking users in an A/B test
Defining Twyman's law and tracking it.
3
A company at the run stage deploys approximately X A/B tests per year
100
1000
10
1
4
In a single layered randomization with 3 simultaneous tests and 3 conditions (A/B/C) users get allocated in
108 cells
3 cells
27 cells
10 cells
5
What do fly companies do in addition to basic analysis
Manual Cleaning and Review of each data record.
Automated Analysis of KPIs, OECs and Guardrail metrics
Systematically record and archive all results
Quickly discard negative results.
6
What do fly firms analyze in an A/B test.
Power, t-test, p-values, decision
Hyperbolic test, q-values, multifactorial
Ad revenue
Guiderail Metrics, A/A' tests
7
What are the four stages of A/B testing
crawl walk run fly
get the thing done
eat drink sleep learn
walk whine talk fine
8
When launching several dozens of A/B tests per year what does the company need very urgently?
a unified OEC
much money
a standardized BQE
A testing infrastructure
9
What problems can arise with multi-layered randomizations and have to be accounted for
The wrong BSEs are tracked
Intedependence issues (Blue on Blue)
Test has too much power
none of the above.
10
Which track are you usually in?
C
D
B
A
11
Why do firms at the fly stage use multi-layer randomization
To avoid overlapping tests
To have better server economy
To handle multiple tests at the same time
It gives a better user experience
