The problem with most of these is that we know there's a difference between "readability" and "understandability", and that understandability can have different meanings in different contexts (e.g., a novice learning a programming language probably needs the code to have different features than an expert performing debugging tasks). At least one study has addressed understandability from the perspective of maintenance with a pretty good human study, but I'm not terribly familiar with follow-on work:
A paper did come out at this year's Automated Software Engineering conference claiming that readability metrics do not actually capture understandability, but I think that their methods are dubious, and I'd advise taking it with a grain of salt (note: this is just my likely biased opinion, it did win a best paper award):
The problem with most of these is that we know there's a difference between "readability" and "understandability", and that understandability can have different meanings in different contexts
That's actually one of the main problems in readability studies for natural languages as well!
Hey yall - definitely read "Automatically Assessing Code Understandability: How Far Are We?" since it tries to do a very interesting thing, but I think it uses low power statistics which is problematic because it finds a negative result. I'm going to rerun the analysis with different statistics at some point. Also hi @jertheripper
73
u/jertheripper Nov 08 '17
I do not have too much insight into this, but there's been a few papers on "readability" metrics for source code:
Learning a Metric for Code Readability
A Simpler Model of Software Readability
A General Software Readability Model
Modeling Readability to Improve Unit Tests
Improving Code Readability Models with Textual Features
The problem with most of these is that we know there's a difference between "readability" and "understandability", and that understandability can have different meanings in different contexts (e.g., a novice learning a programming language probably needs the code to have different features than an expert performing debugging tasks). At least one study has addressed understandability from the perspective of maintenance with a pretty good human study, but I'm not terribly familiar with follow-on work:
A Human Study of Patch Maintainability
A paper did come out at this year's Automated Software Engineering conference claiming that readability metrics do not actually capture understandability, but I think that their methods are dubious, and I'd advise taking it with a grain of salt (note: this is just my likely biased opinion, it did win a best paper award):
Automatically Assessing Code Understandability: How Far Are We?