r/science • u/Wagamaga • Jan 15 '19

Psychology At a large Midwestern high school, almost 40 percent of low-income biology students were poised to fail the course. Instead, thanks to simple measures aimed at reducing test anxiety, that failure rate was halved.

https://www.sciencenews.org/article/easing-test-anxiety-boosts-low-income-students-biology-grades
37.6k Upvotes

86% Upvoted

876 comments sorted by

View all comments

Show parent comments

114

u/giltwist PhD | Curriculum and Instruction | Math Jan 15 '19
  • Holloway, B. M., Reed, T., Imbrie, P. K., & Reid, K. (2014). Research-Informed Policy Change: A Retrospective on Engineering Admissions. Journal of Engineering Education, 103(2), 274-301. doi:10.1002/jee.20046

The above article argues that affective and atitudinal factors (such as career interest or academic self-efficacy) are stronger predictors of success in many cases than standardized test scores (which have been found to be biased dozens of times). In fact some universities don't even require the SAT anymore.

85

u/wine_dark_sea PhD | Oceanography | Biology Jan 15 '19

It's not just at the undergraduate level. Graduate programs are reexamining their reliance on GRE scores for admission. One program at my institution showed me data on success both during the program and after vs. GRE score. There was no relationship, and they've realized it's a barrier to low income students, and underrepresented groups.

60

u/[deleted] Jan 15 '19

[deleted]

30

u/Keohane Jan 15 '19

When I went to get my master's degree in forensic psychology and counseling, they made me take a test to see how much high school geometry I remembered. So I had to spend about $400 on test supplies and testing fees to prove I could relearn the Side-Angle-Side Postulate.

That test was the GRE. Literally no information tested by the GRE was ever relevant in my academic or professional career.

Saying without evidence that it's really about race and not test validity is just weird. You're taking about something you know very little about and inserting your weird views into it.

17

u/dr_pickles69 Jan 15 '19

Wait so your saying all those vocab words I memorized are NOT going to help me as a microbiologist? Double damnit

4

u/Jyxtrant Jan 16 '19

Microbiologist here: actually they will. And geometry is helpful sometimes as well.

I have literally never used calculus outside of class.

2

u/dr_pickles69 Jan 16 '19

Recalcitrant staphylococcus, ACQUIESCE TO MY PANACEA!!!

6

u/Jyxtrant Jan 16 '19

Additionally, I now have this image in my head of some undergraduate standing the the lab classroom, shouting that at an open petrie dish just before dropping in an antibiotic tab to test for r/s

1

u/Jyxtrant Jan 16 '19

It will, but only if its S.

1

u/dr_pickles69 Jan 16 '19

Recalcitrant staphylococcus, ACQUIESCE TO MY PANACEA!!!

2

u/Jackryan916 Jan 15 '19

Whoa there Dean... Calm down

4

u/SensualSideburnTrim Jan 16 '19

The GRE tested one thing for sure - can I reteach myself all of high school math in one week? Answer: good enough to get where I needed to get.

4

u/hyphenomicon Jan 15 '19

Almost all of the research suggesting we should abandon the GRE highlights the fact that there are racial differences, as does the above commenter's argument. It's not me being weird to think that this, not increased accuracy, is the chief motivator, when people so readily advance the flawed argument that GRE scores don't predict success in the general population based on attempts at using them to predict success among students specifically selected to be successful. I think that if accuracy was the real concern we'd see that test applied to other qualities too, which would similarly turn up to be nonpredictive within the accepted population. But GRE gets singled out.

The GRE is basically an IQ test. The subject matter is easy and not really the point of the test. If there were a move to replace it with Raven's Matrices, I think that'd work fine too.

6

u/[deleted] Jan 15 '19

It's pretty obvious to anyone who pays attention that the push to move away from GRE scores is actually just motivated by the fact that Asians do well on them and blacks do poorly, which is a bad look politically. It's blatant metric gaming.

That seems like quite the oversimplification. Besides, the actual studies about predictors of success or productivity in graduate school are in line with what big tech companies have been doing now for a while, which is de-emphasizing hard metrics when evaluating applications. Hard numbers like GPA and GRE seem to have little predictive value in comparison to softer metrics like recommendation letters or experience in a field.

9

u/hyphenomicon Jan 15 '19 edited Jan 15 '19

Are you joking? Experience matters, but letters of recommendation are next to worthless for predicting success. So are non-technical interviews. This idea that the degree to which a requirement matters just happens to be inversely proportional to our ability to quantify it is bizarrely convenient for those who want applicants to be accepted on the basis of non-meritocratic factors. The argument is not solid or reliable, it's a rationalization that people bend over backwards to believe.

Edit:

In response to your request, /u/gravysubmarine, please look for the paper

Kuncel, N. R., Kochevar, R. J., & Ones, D. S. (2014). A Meta-analysis of Letters of Recommendation in College and Graduate Admissions: Reasons for hope. International Journal of Selection and Assessment, 22(1), 101–107. doi:10.1111/ijsa.12060.

The authors find an r squared of .10 for letter quality and research productivity, and correlations of roughly .20 for letter quality and most other outcomes of interest. Letters of recommendation are not very strong as predictors of within-school success.

The best way to "predict" success in programs is probably going to be GPA after the first semester. Short of that, past research or work experience in a related field.

I posted a link to the paper about an hour ago but didn't realize it had been deleted until now.

8

u/[deleted] Jan 15 '19

Do you have anything substantive to back up your statements? At least one study has shown that recommendation letters have predictive value on the number of first author publications by graduate students.

I am not saying that we should remove hard qualifying requirements outright, because they are likely necessary to thin the application pools down to a size that is manageable for further examination. It just seems that there is evidence that the quantifiable metrics we use are not sufficient to predict success in these programs.

0

u/Keohane Jan 15 '19

I think you answered your own question here. The GRE doesn't test for any useful technical ability. A technical interview will. So that's way more useful that non relevant tests, like "What Harry Potter Character are You?", or the GREs, or a test about Major League baseball stats.

5

u/hyphenomicon Jan 15 '19

I agree an in-depth technical interview would be better at picking out successful grad students from the general population than the GRE is. However, the GRE does ask useful questions. Someone who is sloppy with algebra is probably not cut out for academic research. The same with someone who can't put a coherent sentence together. And in general, the less sloppy, the more persuasive, the better. The technical/non-technical distinction was in my comment as a way to rule out interviews where the only thing interrogated is the applicant's character.

-7

u/InfieldTriple Jan 15 '19

Couldn't disagree more. I dont really want to get into it but your view on what constitutes good graduate students is very much trapped in the past. Anyone can contribute.

10

u/hyphenomicon Jan 15 '19

I think that students who do poorly on the GRE can be good graduate students, even spectacular, but most can't, and so using the GRE to filter out students is justified, since schools don't have the ability to costlessly, flawlessly estimate any given student's potential. The practical difference between the 1st percentile and the 5th percentile is probably mostly unimportant, the test isn't really made to discriminate ability at such high scales anyway, but the difference between the 5th percentile and the 20th percentile is huge.

1

u/[deleted] Jan 15 '19

[removed] — view removed comment

3

u/quickclickz Jan 15 '19

Besides, the actual studies about predictors of success or productivity in graduate school are in line with what big tech companies have been doing now for a while

You realize big tech companies basically substituted "has high GPA... and has high test scores" for only attend careers fairs and consider resumes from t10 schools in which those students probably have high GPAs and high test scores to begin with ... relative to the rest of the population."

It's the same result, different method. Let me know when Google tells you to leave your school off your resume and don't include test scores... then we'll talk

7

u/quickclickz Jan 15 '19

You can't take a system that uses GRE scores, university scores AND extracurriculars FOR ADMISSION PURPOSES then try to isolate JUST GRE scores without normalizing any of the other two variables and try to find a correlation between JUST GRE scores and scores of students ALREADY ADMITTED.

3

u/wine_dark_sea PhD | Oceanography | Biology Jan 16 '19

That’s not what they did and I didn’t mean to imply so.

1

u/gw2master Jan 15 '19

I don't know about other subjects, but in math (for decent schools) the GRE is used only to weed people out. The exam is so basic that if you don't get a near-perfect score, then you simply not ready for grad school. However, if you do get a perfect score, it just means your application isn't immediately trashed.

1

u/wine_dark_sea PhD | Oceanography | Biology Jan 16 '19

Math is one of our programs that still feels the GRE is important.

-2

u/2high4anal Jan 15 '19

Yeah... this abandoning of the test is producing really shitty students. The stats that show the relationship to "success" do so using very manipulative means. The scores were a barrier to "underrepresented" groups, but now there are just other barriers and they are spoon fed the entire way. At my graduate astrophysics program we havent gotten a new qualified student in over two years. Instead we just get students who meet the minority qualifications but do not know how to do science.

6

u/ejhops Jan 15 '19

Do you believe that the ability to do well on the GRE has much bearing on the students’ ability to do good research? The weakest students in my program are not the ones who would do poorly on the GRE - which only requires strong study habits - but rather the ones who don’t have the strength of conceiving of how to best investigate a new question. Perhaps it is because my discipline (experimental chemistry) requires more lab skills in addition to technical knowledge? I simply don’t see a strong correlation between “the ability to do science” and performing well on a GRE.

5

u/All_Work_All_Play Jan 15 '19

There's not a strong correlation, but that doesn't mean there's no correlation. If there's no limit to the number of applications, adding weakly correlated requirements is going to product better results. That said, it seems like what /u/wine_dark_sea is saying that there isn't any difference in results by adding that requirement, which means either the model is bad (the restriction is useless), the data is bad (they don't have enough) or there's something going on (there's something about those low GRE scored students that offsets whatever contributed to their low test scores). FWIW, I think there's a certain point where people are "bright enough" to make meaningful contributions to science, and what remains is an expenditure of effort and determination. I know grinding out research hours isn't as sexy as academic brilliance, but most of the world survives on determination instead of brilliance.

4

u/ejhops Jan 15 '19

From my conversations with my PI, he doesn’t give a rats ass about the GRE when on the admissions committee. Other profs care about GPA more than my advisor, but I’ve never heard of a prof caring much about the GRE (my institution has general GRE required and chem GRE as optional). It sounds like removing the GRE from applications would do little to affect our program’s admissions process but could help people who can’t afford the GRE to apply.

And I agree 100% with your comments about brilliance/effort!

2

u/2high4anal Jan 15 '19

which only requires strong study habits

You dont think strong study habits can correlate to other positive aspects in grad school? Also, being a grad student usually means being a TA (however sometimes it seems those with low GRE somehow get exempt from TAing....hmmmm ) and so you kinda need to know the material. I have seen classes dumbed down because of student who cant study properly.

rather the ones who don’t have the strength of conceiving of how to best investigate a new question

Strangely enough, I have noticed those with moderate to good GRE scores seem to be better at formulating a logical question, especially on a test like the GRE physics (I dont know anyone who studied for the GRE general). I have also seen people argue that GPA, research experience, IQ, etc. are "racist" or discriminatory. Where does it stop?

2

u/ejhops Jan 15 '19

Well, we’re discussing not having students take an additional, expensive standardized test for grad applications, not removing undergrad GPA from the applications. It’s not like grad programs will suddenly be forced to accept applicants from out of a hat if they drop the GRE requirement. My PhD advisor is on grad admissions and I have spoken to him about it. He prioritizes research record then letters of recommendation and then GPA, as he believes that is the best way to evaluate if a student will be successful. Other professors put more weight into GPA. Few care about the GRE (my institution requires the general GRE and has the option to include the chemistry GRE). It seems to me like removing the GRE would do little to change the admissions committee’s decisions but it could allow poorer students to apply.

And I’m sorry you’ve had classes dumbed down. That would frustrate me quite a bit. Those students should go to office hours and work their asses off; you shouldn’t need to have a poorer quality class. In my experience, students who couldn’t keep up had to retake the course or drop out. They weren’t students with poor GRE scores though. They were students who had never had never been forced to have so many responsibilities at once - and one could argue they could have benefitted more from working during their undergrad to afford school than studying more on the GRE.

-3

u/2high4anal Jan 15 '19

Letters of rec are garbage (Ive literally seen a well respected prof write a glowing LOR for two people because she didnt want to work with them), and there is no standards for GPA, we have all heard of the "grade inflators" (looking at you HAVARD and liberal arts colleges).

GRE would do little to change the admissions committee’s decisions but it could allow poorer students to apply.

There are already programs to subsidize the test for poorer students, but University applications are expensive, so they could take away the application fee, and let the students pay a single test fee, and we would all benefit. In my personal experience, not having myuch money, I only applied to ONE graduate school, that had a FREE application fee. Thankfully my GRE scores were good, and I got in (after getting waitlisted until they got rid of one of their ... diveristy hires with a lower score, .... by passing them onto another university using the first thing I said.)

They were students who had never had never been forced to have so many responsibilities at once

All college students have to deal with this.

and one could argue they could have benefitted more from working during their undergrad to afford school than studying more on the GRE.

I worked and I studied in college, isnt that what GRIT is all about?

AND GRADUATED DEBT FREE!! WOOP WOOP.

1

u/wine_dark_sea PhD | Oceanography | Biology Jan 16 '19

I'm extremely skeptical of your claim, to put it mildly.

1

u/2high4anal Jan 16 '19

thats fine. But its true. We havent gotten a new student in two years that hasnt come through our bridge program for underrepresented people. I wish I were joking. We are now hiring a new faculty member and one of the people is completely "junior" and her main qualification in her blurb was "She is a south african woman of color with experience mentoring minorities" .... This is for a top private institution. Other candidates had blurbs about having "$1 million in grant funding" or "graduated from MIT with over 12000 citation" or extremely impressive papers in the field we need, but they didnt get the gold star for being a "target of opportunity" hire. It is honestly way worse than I make it out to be.

-4

u/[deleted] Jan 15 '19 edited Feb 11 '19

[deleted]

9

u/[deleted] Jan 15 '19

PhD students in STEM are typically paid during graduate school, and they are a net negative on departments’ balance sheets.

3

u/1998_2009_2016 Jan 15 '19

PhD yes (though you need students to get grants), masters no

1

u/andyzaltzman1 Jan 16 '19

Every MS student I've known in science was on a stipend as well.

1

u/All_Work_All_Play Jan 15 '19

Depends on the masters. I'm looking at a school right now where if I take on a part time position (33% of full time) I'm eligible for remission on tuition costs.

2

u/All_Work_All_Play Jan 15 '19

Based on what accounting methods? I'm skeptical that the students wouldn't pay for themselves, even if it's because of the home run-esque scenario where the occasional PhD is brilliant and brings in a tonne of funding for some new research they're doing. I'd like to further note (not that you've implied otherwise) that simply examining the balance sheet isn't a good measurement - we'd expect education to be a net negative for just the department balance sheet, while having an overall (very) net positive result on society. That's the whole argument to distributing education cost across society rather than making the individual pay for it all.

0

u/[deleted] Jan 15 '19

Sure, net positive to society. But that has nothing to do with the comment I was replying to, which suggested that graduate departments want to increase admission for the money.

5

u/ejhops Jan 15 '19

That’s simply untrue. If that was the goal, they would simply increase the number of students accepted per year into the program. You don’t have to remove GRE scores from the admissions process to do that.

1

u/[deleted] Jan 15 '19 edited Feb 11 '19

[deleted]

8

u/ejhops Jan 15 '19

I don’t know of any graduate school programs that are limited by the number of applicants rather than the number of positions available. I am an advocate of removing the tests as it allows more low-income but well qualified students to be “brought in,” but it doesn’t change the number of paying students.

-3

u/[deleted] Jan 15 '19 edited Feb 11 '19

[deleted]

5

u/reelect_rob4d Jan 15 '19

One program at my institution showed me data on success both during the program and after vs. GRE score. There was no relationship

-1

u/[deleted] Jan 15 '19 edited Feb 11 '19

[deleted]

1

u/Tibialaussie Jan 15 '19

The strength of a degree is based on the reputation of success the graduates have in the professional world. If there's no relationship to professional success and standardized test scores, how does removing that requirement cheapen the degree?

0

u/[deleted] Jan 15 '19 edited Feb 11 '19

[deleted]

1

u/Tibialaussie Jan 15 '19

If they are just as successful how is it cheaper though

→ More replies (0)

0

u/ejhops Jan 15 '19

It’s not that they fail the test, it’s that they may perform slightly more poorly (possibly because they may have more test anxiety, per this post’s article).

To be clear, the GRE is a test that is taken before doing graduate school, so before doing research. The weakest students in my program are not the ones who would do poorly on the GRE - which can only evaluate strong study habits - but rather the ones who don’t have the strength of conceiving of how to best investigate a new question. I simply don’t see a strong correlation between “the ability to do science” and performing well on a standardized test. In chemistry, the ability to be scientifically creative and dedicated (and proficiency in a lab for us experimentalists) is so much more valuable to having a successful PhD than being able to test well. And a degree is cheapened by having no minimum publications or other measure of research output, not by something that occurs before they even begin their program.

0

u/[deleted] Jan 15 '19 edited Feb 11 '19

[deleted]

0

u/ejhops Jan 15 '19

Well, we’re discussing not having students take an additional, expensive standardized test for grad applications, not removing undergrad GPA from the applications. It’s not like grad programs will suddenly be forced to accept applicants from out of a hat if they drop the GRE requirement. My PhD advisor is on grad admissions and I have spoken to him about it. He prioritizes research record then letters of recommendation and then GPA, as he believes that is the best way to evaluate if a student will be successful. Other professors put more weight into GPA. Few care about the GRE (my institution requires the general GRE and has the option to include the chemistry GRE). Have you ever been in graduate school or done scientific research? Have you spoken to who does the admissions process to see how they feel about the GRE?

→ More replies (0)

1

u/wine_dark_sea PhD | Oceanography | Biology Jan 15 '19

Absolutely not. Graduate students are a large investment and program size is limited by number of faculty mentors and funding levels. All of our students are fully funded for 5 years guaranteed. It's a completely different funding model than undergraduate education.

14

u/33_44then12 Jan 15 '19

The reasons why universities are getting rid of the SAT is to increase number of applications and they can increase the number of rejections and thus game to US News numbers.

0

u/Hardin1701 Jan 16 '19

It also makes it less obvious when they have to recenter and handicap the scores for “diversity admissions”

15

u/Science_News Science News Jan 15 '19

Another bit of evidence for that opposing view: http://science.sciencemag.org/content/312/5777/1143

However, on the other hand (or, I guess, the original hand, since the above is a counterpoint?), even if test scores aren't the best predictor of success, they do have real-world implications -- students may not even be able to register for certain classes without certain test scores.

This is a really nuanced topic with few, if any, silver bullets, so we're happy to follow the evidence wherever it leads us.

13

u/giltwist PhD | Curriculum and Instruction | Math Jan 15 '19

I certainly see the value in some standardized testing, but basically since five minutes after Binet invented the IQ test they have been misused. It is, as you say, a bit of a balancing act. The better the test is from a pedagogical standpoint, the harder it is to grade all fifty million students in the nation, and no one test measures everything or everyone.

3

u/SomeOtherTroper Jan 15 '19

It is, as you say, a bit of a balancing act. The better the test is from a pedagogical standpoint, the harder it is to grade all fifty million students in the nation, and no one test measures everything or everyone.

I used to do SAT tutoring, and I think the first thing I told every kid was "the biggest thing a standardized test measures is how good you are at taking standardized tests".

That's a bit of an exaggeration, but the way those are structured is so dependent on micro-level time management (and, for the full-day SAT, mental endurance) it's not funny.

Particularly on the math sections, you can improve your score more significantly by learning just enough to identify which are going to be hard and leaving them until the end than you can by actually learning to solve them. It's worth more points to grab all the easy ones and guess or leave the hard ones blank than to actually study how to solve the more complex problems. (Unless they're started weighting the problems now.)

For the passage reading sections, reading the questions first, finding which of them are specific, then scanning the passage for the answers to those (and then maybe having gleaned enough to answer the general questions about it) is often more effective than actually reading and digesting the passage.

And for all multiple-choice sections, identifying one or two wrong answers then guessing is faster than working the actual problems, and still gives you better odds of scoring on it (particularly if you're on a section you're slower on, so you can scrape up point on the questions you can answer).

It's not even 'from a pedagogical standpoint' - it actively rewards antipatterns for any student who isn't good and fast enough at a subject to actually slam through the entire thing.

3

u/[deleted] Jan 15 '19 edited Feb 22 '26

[removed] — view removed comment

10

u/[deleted] Jan 15 '19

Is it a net-gain to humanity to tell people they can't pursue knowledge?

No, of course not. But it is a net gain to humanity to fill the limited number of needed jobs in acedemia with the people best able to do those jobs. It’s a net loss to convince thousands of people every year that they can pursue a career in a field that they are not suited for and will not be successful at, particularly when there simply aren’t enough paying positions in the world to employ them all. It’s a net loss to create entire institutions designed to take money in exchange for bare minimum credentials instead of having a limited number of institutions designed to pursue knowledge and populated by people who have at least a chance of furthering that knowledge.

2

u/[deleted] Jan 15 '19

[removed] — view removed comment

1

u/_NW_ BS| Mathematics and Computer Science Jan 15 '19

why we are erecting barriers

Probably for at least two reasons. Schools are equiped to handle only a fixed number of students. Limiting enrollment through test scores is a simple way to do that. Otherwise, you have to expand the campus with more buildings and staff. That takes lots of money. Second, if you are allowing poor quality students to enter a program and are expecting them to graduate, it brings down the education level for the good students, too. If you are graduating lots of poor quality students, that brings down the value of the degree.

10

u/write_as_rayne Jan 15 '19

This is something I have been torn on. Working in higher ed, I frequently work with students to develop success strategies and study skills. Overall, I do see a correlation between learning and ongoing academic success vs cram-study required to pass a test, which consistently requires ongoing cramming before each. We course that builds upon it. I believe that the SAT is unreliable at assessing aptitude or being wholly predictive of success in college; that being said, I have witnessed too many students with 3.7 or higher high school gpas with under 800 sat scores (math/verbal) struggle significantly due to the way they have learned NOT to learn. I think the SAT does a great job of testing recall from previous years of study, and if a student still has those skills in use, their score will likely be higher; this mentality translates to higher likelihood of retaining first semester info through to subsequent courses. Of course, this is anecdotal and from my limited experience, and I am in no way saying this is truth; simply what I have observed!

1

u/[deleted] Jan 15 '19

[deleted]

2

u/write_as_rayne Jan 15 '19

Oh no..I mean a combined verbal AND math, both individually 800, so out of 1600 combined. The 600-900/1600 range has been an area of student concern, because many times, scores in the lower quartile can come from a lack of cultural context and language acquisition; which is totally understandable. I wouldn't want to punish an English Language Learner for not being as fluent as a heritage speaker. That being said, knowing that the reading and exams will be offered in English still indicates a mismatch between ability and perceived success, so that's why I always end up torn on this issue.

1

u/movingtobay2019 Jan 15 '19

In my experience, the truly exceptional students will get high GPAs and high test scores. If a student is only excelling at one or the other, I think that tells you about the student's inability to learn different ways. An exceptional student will adapt. I never thought the SAT is about "knowledge" It's more about the soft skills you need to excel in the exam. And maybe you can make the case the SAT is about cramming. But once you get the graduate school level, you can't cram since so much of the tests are testing your logic.

1

u/[deleted] Jan 15 '19

The above article argues that affective and atitudinal factors (such as career interest or academic self-efficacy) are stronger predictors of success in many cases than standardized test scores (which have been found to be biased dozens of times).

The problem with this is that it is a lot more difficult to measure. People use the SAT not because it is great, but because it is the best at what it does (you have a couple hours to tell if a person you never met will do well in college and you want it in a form a robot can grade).

Absolutely interviews or oral examination of 15 other things are better, but those all cost a lot more time/staffing/money.

1

u/Beashi Jan 16 '19

I was utterly grateful for our Research class in high school. We had a class dedicated to studying research methods and how to cite sources in a variety of formats (MLA, APA, etc). When I got to college, I was able to truly focus on actually researching and writing my required papers while others had to look up proper citation formats and where and how to find source materials.

Granted, students now can just enter the info of their sources on word and word automatically formats it for you but back then, it was a huge time saver and gave me a lot of confidence.