Hacker News new | past | comments | ask | show | jobs | submit login
Undisclosed tinkering in Excel behind economics paper (retractionwatch.com)
194 points by _Microft 11 months ago | hide | past | favorite | 138 comments



A related anecdote: Jönköping University is not allowed to call itself a university in Swedish (“universitet”), since that word is, by law, reserved for institutions that meet certain requirements. To get around this it uses the English name “Jönköping University” in Swedish as well. But formally it’s a so called “högskola”, more often translated as ”college”.


To be honest the difference is minimal. I studied CS at Mitthögskolan in Östersund, but didn't complete all points before graduation (still had a paper to write and hand in for one of the courses). A year after graduation, Mitthögskolan became a real university (Mittuniversitetet). I emailed the professor in charge of the course I had not yet completed, wrote my paper and handed it in, then received a university diploma in computer science, even though i technically never studied at a "real" university.


The difference for an institution that was just at the edge was minimal yes. But that's a special case.


Usually such a promotion between levels for an institution implies they observed for several years the higher standards needed for the higher level. So you studied according to the higher standard but the classification isn’t changed retroactively.


Sometimes, procrastination pays off!


Is there even a difference at that level of study, which I infer from your post is undergraduate/master?


Until recently in Ireland, third level education places were largely divided into “universities” and “institutes of technology” (previously “regional technical colleges”).

The institutes of technology have been reorganised lately and are now called “technical universities”.

The degree you get is generally much the same from any of them - but for some insane reason graduates of the institutes of technology/technical universities don’t get to vote in the national elections for the upper house (senate). Only graduates of the more prestigious ones get that right.


The Seanad thing is just absurd, given that the Constitution was changed by referendum in 1979! It's also worth noting that four universities get three seats, while another has three to itself.

The whole bloody upper house is insane in Ireland, but it's convenient for the legislators so it doesn't get reformed.


I hadn't heard about any of this and chased some details on Wikipedia; here's some more context for others:

In 1979, there was a referendum in which 92% of the voters were in favor of extending Seanad representation to graduates of other higher education institutions. This changed the constitution in a way that allowed laws to be passed to implement that, but no such laws have yet been passed. There was a ruling by the Supreme Court of Ireland in 2023 requiring a law to actually be passed, but the ruling is currently suspended until 26 May 2025.

https://en.wikipedia.org/wiki/Seventh_Amendment_of_the_Const...


I remember while studying at an IT they had a debate there about the potential dissolution or reform of the Seanad, with some senators taking part.

It was deeply funny listening to the sitting senators tell us how important the Seanad is, because for all of us - the Seanad was some useless shit we couldn’t vote for or influence in any real way.


> some useless shit we couldn’t vote for or influence in any real way.

I have no idea if it's useless shit and I admit to learning about the Seanad 15 minutes ago from Wikipedia but it seems to me that it's supposed to be a weak technocratic advisory chamber insulated from popular will. At least it seems to make more sense than the House of Lords.


Still not as bad as the English upper house!


So if you don't graduate university you have reduced voting rights?!


It was/is a thing in some countries: https://en.wikipedia.org/wiki/University_constituency#Irelan...

But the Irish Senate has a relatively unusual composition and voting system anyway: https://en.wikipedia.org/wiki/Seanad_%C3%89ireann#Compositio...


For the upper house, yes.

As a graduate of an institute of technology, having transferred from a university, I don’t get to vote in the Seanad (senate) elections at all.

To clarify: most upper house seats are elected by members of parliament. There’s a pool of seats reserved for graduates of a subset of universities to vote on.

It’s terribly undemocratic and needs to be reformed or abolished.


For one particular pool of candidates fighting over university allocated senate seats - you still have regular voting rights.


Similar as the Netherlands: there 37 "hogescholen" marketing themselves as "University of Applied Sciences" in English that are not allowed to claim to be "universiteiten" in Dutch (a name is reserved for 14 more academically advanced institutions).


The distinction becoming vague is due to the changes regarding the Bologna Process/European Qualification Framework - whereby both Universities and Technical Colleges both award bachelor degrees (EQF6) ... and if it looks like a duck, and quacks like a duck, and awards the same EQF paper at the end, then it's the same! I've taught in a programme that is offered in both Universities and Technical Colleges (Law) ... and the distinction becomes that universities are more theory based, whereas technical colleges are more practical ... and that matriculating into a university offered masters programme in NL from a technical college awarded bachelors is made harder (even with the same EQF6 paper) ... not that any other university in the EU would care about the distinction!


> reserved for 14 more academically advanced institutions

So "Technische Universiteit Delft" is not an advanced institution?

In either case, in Sweden "högskola" covers everything from small rinky-dink schools up to KTH (Kungliga Tekniska Högskola) in Stockholm and CTH (Chalmers Tekniska Högskola) which are among the top engineering schools in Sweden.

Both award doctoral degrees. I have not heard many people in Sweden argue that KTH/CTH are not as advanced as the top universities. The difference is just that the technical schools do not have departments for the humanities.


> So "Technische Universiteit Delft" is not an advanced institution?

It is, and it is also one of the 14 Dutch institutions that are allowed to use the Dutch "universiteit" name. But the English term "university" seems to be unprotected in the Netherlands.


Dutch speakers often fall into the trap of equating the terms ‘university’ and ‘universiteit’. Since the English language does not differentiate between the terms ‘hogeschool’ and ‘universiteit’, it is appropriate for both to translate into ‘university’.

(And then, of course, there’s also the term ‘volksuniversiteit’…)


But KTH and Chalmers are officially universities despite their names. Jönköping was not until recently.


> Jönköping was not until recently.

It still isn't. Unlike KTH and Chalmers, "Jönköping University" is listed as a university college, not a university: https://www.uka.se/swedish-higher-education-authority/about-...


Yeah you're right, sorry. I confused them with some other school that recently became a university.


Same in Germany where Fachhoschulen (hogescholen) marketing themselves successfully to internationals as "University of Applied Sciences"


Same in Finland. "University of applies sciences" is not a real university but rather a lower level institution (ammattikorkeakoulu/yrkeshögskola/Fachhochschule). The difference is clear for locals but I bet many international students have been tricked by the English branding.


Yep. Pretty misleading. They can use it because the law doesn't mention that word, and their management likes to pretend they are running a university, and because they were so focused on attracting foreign students.


> They can use it because the law doesn't mention that word

This is not true. The law defines which schools are allowed to call themselves "hogeschool", and also that the English translation of that word is "university of applied sciences" or, if limited to a specific job profile, "university of <said profile>".

See https://zoek.officielebekendmakingen.nl/stb-2017-97.html

> Het voeren van de naam hogeschool is voorbehouden aan de instellingen voor hoger onderwijs die zijn opgenomen in de bijlage van deze wet onder g, en aan rechtspersonen voor hoger onderwijs. Onder het voeren van de naam hogeschool wordt tevens het voeren van deze naam in samenstellingen verstaan, alsmede het voeren van de naam hogeschool in vertalingen. In afwijking van artikel 1.22, eerste lid, wordt de naam hogeschool in het Engels aangeduid met «university of applied sciences» dan wel, bij hogescholen die opleiden tot een bepaald beroepsprofiel, «university» met daarachter het vakgebied.


Ah, my bad. I didn't know the levels of delusion had gone that far.


Germany's and Austrias Universities of Applied Science use the same trick, Fachhochschulen is their German name.


Not anymore. They have all(?) rebranded to „Universität“ in German.

But that‘s intentional. Former FHs are practically identical in law to universities. They all have the right to promotion, their masters and bachelors are legally identical to universities‘ masters and bachelors.


The former "Diplom" and "Diplom (FH)" were not identical. When those still existed the distinction of "Universität" and "Fachhochschule" was meaningful. The rebranding is a consequence of the old kind of degrees having been replaced by new ones, not just a marketing trick.


Yes, you're right, the causal link is this.


Not in Austria.

There's a very clear distinction between FHs and Universities. Not necessarily in reputation or level of education, but they are separated entities. And you cannot get a PhD from an FH.


You seem to be really misinformed about the legal definitions of these tertiary education institutes in Austria.

Universities of applied sciences are governed under the "Fachhochschulgesetz"[1] (Universities of Applied Sciences Law) while universities are governed under the "Universitätsgesetz"[2] (Universities Law). This carries loads of differences with it, i.e. when it comes to academic freedom and right to promotion. Professors at universities of applied sciences are much more limited in their academic freedom and these institutions do not possess the right to promotion at all.

Not sure where you got your information as it's wildly untrue in some respects.

[1] https://ris.bka.gv.at/GeltendeFassung.wxe?Abfrage=Bundesnorm...

[2] https://ris.bka.gv.at/GeltendeFassung.wxe?Abfrage=Bundesnorm...


I haven‘t said a word about Austria.


You responded to a comment talking about "Germany's and Austria's Universities of Applied Science" - which countries did you mean if not Germany and Austria?


Germany, obviously.

Enough harassment for tonight?


I initially wanted to say that this is entirely wrong, but when I went to do a quick check, I found this: https://www.che.de/2023/haelfte-der-bundeslaender-hat-promot... So my knowledge was not entirely up to date.

In short, bachelors and masters are indeed legally identical to the ones acquired at universities, but most do not have the right to promotion. This mostly depends on the state they're in. And they are not allowed to call themselves "Universität" (though some have rebranded to "Hochschule").

All FHs that I know do have an option for getting a Phd through them, but they themselves can't grant them. Instead, an FH can cooperate with an Universität through a sort of loophole. The Phd students spend their time at the FH, but they have two supervisors and get their Phd from the "Universität".

But it seems like it is only a matter of time until difference, too, is no more.


I can't think of a single example of a former FH having rebranded to "Universität". The closest I can think of is the incorporation of FH Lausitz into BTU Cottbus. In any case, there are still >200 FHs in Germany which are not using the "Universität" label and without right to award doctorates. I'm really not sure what you are talking about.


Many have rebranded themselves as "Hochschule" Not "Universität"


Not true that FH degrees are identical in all respects: for the civil service there is still a difference in pay.

The RWTH in Germany still uses Hochschule, but does add University now to their name.


The RWTH has never been a FH, though. It‘s „Technische Hochschule“ for the same reason why the University of Stuttgart was a „Technische Hochschule“ for a long time: traditionalists insist that a university must have medicine, law and religion.


Yes, never FH, but also not university.


No, the RWTH is indeed a university. Many (old) THs were granted the status of "Universität" in the 70s and 80s, see https://de.wikipedia.org/wiki/Technische_Universit%C3%A4t

And if you don't trust Wikipedia, the RWTH refers to itself as an "Universität", see here for example: https://www.rwth-aachen.de/cms/root/Die-RWTH/Aktuell/Hochsch... Doing so would be illegal if they weren't one, so I don't see where the claim that they're not comes from.


Yes, it is (and was for a while now) a university but it didn't refer to itself as university until quite recently.


Do you have a source for that claim? This is the first time I'm hearing of a difference in pay for FH and Universität degrees there (regarding bachelor and master, old diploma system doesn't count).


Have a look here with the references to "wissenschaftliche Hochschule" and some differences between between degrees/needs: https://www.bva.bund.de/SharedDocs/Downloads/DE/Behoerden/Be...

(Slide 9, for example).

Edit: you are right that the difference might not always apply, but they are not always identical by default as far as I understand it.


This is a really unfortunate but also quite common pattern in Sweden, where there's a veneer of internationality which tricks foreigners instead of helping them. Swedes will generally understand that Jönköping university isn't formally a university and wouldn't compare it to the more prestigious schools, but for international students it's not at all obvious. They might pay a lot of money and discover too late that the education and/or credentials they get in return don't match at all what they paid for. As usual, the Swedish authorities seem unwilling or impotent to take action on this type of issue.


Sweden was a real pioneer with progressive social change and looking out for the little guy, but as with most of these things, the institutions set up to drive this change ossified, and are today more like a ceremonial roleplay of a process that concluded half a century ago.

These are the checks and balances that are in place, and unfortunately looking up from the faded old daguerreotypes of Ådalen 1931 and addressing issues from this century does not seem to be at all a priority.

You can get some action if you manage to dress the problem up as as a J.P. Morgan type industrialist opressing the workers (as happened with the Tesla blockade), but if it doesn't fit that narrative you're generally SOL.


It doesn’t help that the terms don’t translate 1:1.


I wonder if that would really stand up in court. Here in Sweden our judges are not very impressed by clever technicalities.


Chalmers on the other hand is technically a universitet, but maintains the högskola title in Swedish.


Same for Kungliga Tekniska högskolan (KTH)


In France, "Licence" and "Master" (Bachelor's and Master's degree respectively) are protected diplomas that can only be delivered by universities (or by schools that depend on universities).

To circumvent this, private non-state-certified schools deliver things called "Bachelor", "MSc", "MBA" or "Mastère" that walk and quack like regular diplomas but since they're using those specific names, they're not actually state-recognized diplomas. Which means that if a company is hiring "people with a Licence" they can reasonably deny someone with a "Bachelor" because even though it's also a 3-year degree, there is no guarantee whatsoever that it's actually worth anything. Anybody can create a company and hand out sheets of paper with "MSc in Psychoceramics" written on it.

This is a big problem for high schoolers who usually don't know the difference and end up paying thousands of euros for private schools that deliver close-to-worthless degrees (not all are worthless, but their worth is mostly correlated to how well-known the school is, whereas if you get a real state-certified degree from a random university anywhere in France it'll be recognized by any company).


Similar thing in France, however related to degrees: a Master's Degree fits within the common framework of European tertiary studies, AKA your 2-year graduate cycle that ends up totalling 5 years if you include a standard undergrad cycle (Licence). However, a "mastère spécialisé" (which is an actual trademark) is not an officially recognized diploma at all, but a label used by schools to denote (usually paid and expensive) "specialization paths". They provide no academic credits, and you can't move forward to a PhD with one. Many prestigious schools (grandes écoles) provide those kind of certifications to clueless foreign students who end up paying huge sums of money for something that holds no formal weight in any academic framework.


Something very similar in Austria - you can get a free Master's degree (1-2 years) by doing a study programme ("Studiengang") at a university, that will also qualify you for a PhD. Or you can pay for a Master's course programme ("Lehrgang") that costs ~15k€, is usually a lot easier and does not qualify you for a PhD.

The difference here is that you are allowed to wear the title of MSc either way and both programmes do confer ECTS points - only the latter does not qualify you for a PhD.


Wouldn’t that also make the professor _not_ a professor?


Are community college professors not professors?

(Edit: I wrote “community college” because I though that was what the grand-parent had said - but it was just “college”.)


It depends on the country:

"In most [UK] universities, professorships are reserved for only the most senior academic staff, and other academics are generally known as 'lecturers', 'senior lecturers' and 'readers'."

https://en.wikipedia.org/wiki/Academic_ranks_in_the_United_K...


Yes, the term professor is used more or less generously in different countries.


It depends on the country, but here in Germany no


I'm not exactly sure what a community college is, but in view of the discussion about "Fachhochschulen" above, I want to note that Fachhochschulen do indeed have professors.


Volkshochschulen...


https://en.wikipedia.org/wiki/Community_colleges_in_the_Unit...

In the United States, community colleges are primarily two-year public institutions of tertiary education. Community colleges offer undergraduate education in the form of an associate degree. [In addition community colleges also offer remedial education, GEDs, high school diplomas, technical diplomas and academic certificates, and in rare cases, a limited number of 4-year bachelor's degrees.]

https://en.wikipedia.org/wiki/Associate_degree

An associate degree or associate's degree is an undergraduate degree awarded after a course of post-secondary study lasting two to three years. It is a level of academic qualification above a high school diploma and below a bachelor's degree.


No.



Do you maybe confuse community colleges with „Volkshochschulen“?

The latter offer recreational courses, like painting or language learning or Indian cooking, usually one hour per week for a quarter year or so. The courses are held by laypeople.


in most countries the term university is reserved for institutions which provide graduate (undergrad optional) degrees in four (three in some places) major fields (sciences, humanities, medicine, etc)

nothing else.

usually just adding a business major to a place with medicine, engineering and law is enough to cross the threshold and jump from college to uni.


For comparison, would Harvard or Stanford be allowed to call themselves an universitet?


A simple test of the rules StackRanker3000 linkes to is whether you'd hold a PhD degree from those establishments in high regard independent of the field. If so, that's a universitet. If you'd only highly respect PhD degrees awarded in a few areas, like STEM fields when it comes to CalTech or MIT, that's a högskola, similar to an institute (possibly polytechnic if it is strong in more than one field).

And as was written above, some establishments, as they grew from institutes to universities, decided to keep the old name, MIT and CalTech included.


This may be helpful: https://www.studera.nu/startpage/higher-education/universiti...

> To put it very simply, the difference between universities (universitet) and university colleges (högskolor) is that universities can award degrees at the doctoral (third cycle) level because they have general permission to do so.


> the difference between universities (universitet) and university colleges (högskolor) is that universities can award degrees at the doctoral (third cycle) level because they have general permission to do so.

The text wrongly implies that "högskolor" cannot award doctoral degrees. This is not true. It's just that they don't have a built-in general permission to do so.

In practice, KTH/CTH etc have a large number of doctoral students and award many doctoral degrees every year.


Not to discredit any of what you said, but I believe both KTH and CTH are classified as universities, despite what their names imply.


This is correct. KTH hasn't changed its name because the name has an extremely long history.


No, they would be considered institutions with too low of a standing.


Oh this is nothing. Go look at the social sciences. They get data that compromises their entire profession, tell the university it’s inconclusive and wait until the research grant runs out and fail to publish. Then do the same thing again.

I have seen this with my own eyes. A bit of excel drama is usually behind their published papers. Also seen that with my own eyes.

The field has trouble collecting data too because it’s expensive and very subjective. The worst one I saw was somehow turning 7 data points via imputation and interpolation into a hundred or so. The principal researcher on that I actually had to teach BIDMAS to and her calculations were completely whacked out (excel again)


A lot of people are upset when their grant doesn’t get funded and it’s easy to suppose there’s some sort of sinister motive behind it. In truth it’s probably that the science is very difficult and only a tiny fraction of people get anywhere in social sciences.

Try marine biology, they are always looking for decent papers.


Because there are bad apples, doesn't mean you should throw out the scientific method.

Over 50% of social science studies do replicate.

Sorry that isn't high enough. We should definitely just give up on understanding ourselves.

Or, maybe fix the financial incentives.

Any industry across all fields, if the incentives are miss-matched, will produce crap.


> Over 50% of social science studies do replicate

There is some tasteful irony here. Could you link to a study on this claim? And ideally a few other studies that support its claim? Thank you.


The poster might have confused social sciences with psychology?

>Study replication rates were 23% for the Journal of Personality and Social Psychology, 48% for Journal of Experimental Psychology: Learning, Memory, and Cognition, and 38% for Psychological Science. Studies in the field of cognitive psychology had a higher replication rate (50%) than studies in the field of social psychology (25%).

https://en.wikipedia.org/wiki/Replication_crisis

Definitely better than medicine!

>Of 49 medical studies from 1990 to 2003 with more than 1000 citations, 92% found that the studied therapies were effective. Of these studies, 16% were contradicted by subsequent studies, 16% had found stronger effects than did subsequent studies, 44% were replicated, and 24% remained largely unchallenged.[76] A 2011 analysis by researchers with pharmaceutical company Bayer found that, at most, a quarter of Bayer's in-house findings replicated the original results.[77] But the analysis of Bayer's results found that the results that did replicate could often be successfully used for clinical applications.[78]

25% of experiments replicating in-house is crazy low, but explainable due to very different incentives (the researchers need to get the paper out; the Bayer scientists need to have a functioning product)


Didn't realize there was so much variation from field to field. Even inside 'social sciences'.

I'd like to see this across all, including STEM.

Note: I'm using 'social science' colloquially. On HN, seems like Psychology is a 'social science'. Anything dealing with people is deemed a 'social science'. Even though that isn't technically correct, they are separate fields. There is a 'Social Science' field and a 'Psychology' field.


[flagged]


Why cite something if any source will be rejected.

Where are we when no studies are trusted, and no review body of any study is trusted.

Don't think it is just social science. That is hubris, in our post truth world.

""A study published in 2018 in Nature Human Behaviour replicated 21 social and behavioral science papers from Nature and Science, finding that only about 62% could successfully reproduce original results."

Roger A (27 August 2018). "The Science Behind Social Science Gets Shaken Up—Again". Wired. Retrieved 2018-08-28. Camerer CF, Dreber A, Holzmeister F, Ho TH, Huber J, Johannesson M, et al. (September 2018). "Evaluating the replicability of social science experiments in

Nature and Science between 2010 and 2015". Nature Human Behaviour. 2 (9): 637–644. doi:10.1038/s41562-018-0399-z. PMID 31346273. S2CID 52098703.

https://newsroom.haas.berkeley.edu/research/amid-a-replicati...

https://royalsocietypublishing.org/doi/10.1098/rsos.200566

EDIT

"For instance, a major reproducibility project that sought to replicate 193 experiments from 53 high-profile cancer biology research papers ran into multiple barriers."

https://bigthink.com/hard-science/the-replication-crisis-is-...

EDIT 2:

Physics Discussion https://pubs.aip.org/physicstoday/article/72/12/8/811763/The...


Respect for citations. I'll read into this over the next few days.

I tend to treat papers as disinteresting until someone has asked me to tear a new hole in one (I can only really do that mathematically) or they have been reproduced. Decreases cognitive load when someone says "check this out".


Well, that is the problem.

Time and "Decreases cognitive load".

We really trust scientific papers to have done the leg work. If it is in a respected peer reviewed publication, then we can trust it and then build on it.

We can't all individually reproduce every study.

I really fear for the future.

Because already there are tons of garbage studies being faked by AI flooding the world. It will become very difficult to know anything.

My only beef here is that it is not isolated to social sciences. HN really piles on social sciences while having a blind spot for STEM. Right when neuroscience and AI are converging and both sides need to have more understanding.

Edit:

at Minute 5

https://www.youtube.com/watch?v=g7TghURVC6Y


What profession have you seen compromised?


"Journal of Cleaner Production" is not "highly ranked" in economics. Economists are pretty hierarchical about their journals and know which are the top 5, which are top field journals etc. That one is a non-entity, I have literally never heard of it.


"Highly ranked" probably refers to a formal metric like the impact factor. See https://en.wikipedia.org/wiki/Journal_ranking


It appears not to be an economics journal at all, at least it is not featured in RePEc's rankings: https://ideas.repec.org/top/top.journals.simple.html


What are the top 5?


Depends a bit on the branch/focus of economics you're looking at, and which nation/region you want to read about, but a broadly uncontroversial list would be:

The Quarterly Journal of Economics, Oxford University Press (colloquial known as "the QUGE")

Econometrica, Econometric Society

Journal of Economic Literature, American Economic Association

Journal of Economic Growth, Springer

Journal of Financial Economics, Elsevier

Maybe swap in the Journal of Political Economy if you're looking for more political leaning research. Swap in the American Economic Review if you're interested in more American-centric research.


That's incorrect, the Top 5 in econ are (in no particular order):

- Econometrica

- American Economic Review

- Quarterly Journal of Economics

- Journal of Political Economy

- Review of Economic Studies


Just to add some context, the term "Top 5" is not subjective, it's a reference to those five journals. Tenure and promotion decisions in top departments based their decisions on publications in those journals.[1]

[1] https://newsroom.iza.org/en/archive/research/the-tyranny-of-...


This is the correct answer and would get the agreement of probably better than 95% of the profession.

The suggestion above that JEL is a top five is bizarre. The CS equivalent is saying that the book “learn python in 21 days” is a top outlet for CS research.

The authors of this paper should be deeply embarrassed about the way they handled that data, but I’m slightly relieved to see that this garbage was published in a tenth-rate journal no one has ever heard of. (Though to be clear there are definitely problems with what gets through in top journals too!)


QJE and Econometrica yes. JEL is excellent but not one of the "top 5": it's literature reviews. JFE is big in finance but not top 5. JEG, not even close - it's a field journal.


The title is a bit odd. The article details a sloppy use of substitute data from adjacent rows or even columns to fill gaps in data, excel has little to do with it. The worst example is using United Kingdom data to fill gaps in USA data because they were adjacent in the list.


Excel has everything to do with it. From the article:

> In email correspondence seen by Retraction Watch and a follow-up Zoom call, Heshmati told the student he had used Excel’s autofill function to mend the data.

The most charitable interpretation is that the professor has no idea what autofill is or isn't capable of doing, so he misused it.


That’s not even the worst case though, worse by far than interpolating (whatever method is used) is simply taking data from countries with a similar name to fill gaps.


"What was your rational for using data from the Netherlands to fill in the missing values in New Zealand?"

"Um, both countries start with the letter N"


Should’ve used Denmark rather than Netherlands, as it’s the Old Zealand.


Do _you_ have an idea of what autofill is and what is or isn’t capable of doing?

When he dragged down a selection he _saw_ the values being copied or a trend being extrapolated.


Flash Fill and Series Fill are great at recognizing simple patterns like 1,2,3,4,5 or 2,4,6,8,10 and can sometimes (unreliably!) recognize more complicated multiplications, repetitions or formulaic string modifications (like house numbers and street names). I use both frequently.

But neither these tools nor an LLM has real understanding, they're not going out into the real world and collecting data. The results need to be verified. Automatically filling an index column with 1,2,3... is one thing, automatically filling a data column with guesses from pattern matching is different. This problem is only going to get worse as LLMs proliferate and can more reliably but still imperfectly and still opaquely fill in more complicated data.


I don't use autofill precisely because I have no insight into the algorithm behind it. I suspect it's very "stupid," but I don't know exactly how stupid it is.

It sounds like he didn't realize or notice that autofill was using adjacent cells (from other countries, in this instance). That behavior would shock me as well. I thought autofill only took columns of data into account.


> It sounds like he didn't realize or notice that autofill was using adjacent cells (from other countries, in this instance).

No, it doesn’t sound like that: “in several instances where there were no observations to use for the autofill operation, the professor had taken the values from an adjacent country in the spreadsheet.”

It was not Excel’s fault.


Right, but the mistake wasn't an issue with Excel, it was with the professor misusing it.


Yes, I understand that and agree.

The headline isn't blaming Excel. "Undisclosed tinkering in Excel" is not far from the accurate headline: "Misuse of commonly-used Excel tool..."

The interesting thing is that a supposedly sophisticated person was using an unsophisticated tool and then relying on the data it produced. Failing to mention "Excel" in this case is doing a disservice to the thrust of the problem.


The accurate headline would be incomplete data falsified.

The tools have nothing to do with it, and the same could have been done with paper and pencil. Autocomplete is the problem, it just repeats a single number or a series, and when using it you see exactly what the results were.

This was not done by mistake or in ignorance or because the tool encouraged them, it was done to knowingly falsify incomplete data so that they could finish their paper and publish.


Might as well blame ChatGPT for writing your term paper.

If they'd been using R/Matlab/Python written by a copilot it would have been just as much gibberish.


Such things are encouraged by the excel UI because it fails to make relationships and manual corrections easily visible. The human computer interface matters beyond UX.


I get what you're saying, but I would disagree that the focus should be on Excel and its UI/UX. As annoying as Excel is when you're working with _specific_ data that suffers from Excel's attempts to be helpful, I'd wager for most use cases Excel is doing exactly what most users expect it to, or the change is benign at best for most use cases.

That isn't to diminish when Excel's help has disastrous consequences, but rather to say it shouldn't be unexpected. If your data requires precision, there are better tools than Excel for working with your data and there should be steps taken by more experienced members of the team to ensure the team doesn't get bit by Excel's features. Excel's behaviors are well known and I'd have to imagine at least one member of any research team has been bitten by Excel's assistance features before during research. I would be surprised if any serious educational institution that teaches about ethics and common pitfalls in research *didn't* caution about how Excel might corrupt research [0].

0 - Okay, maybe not _that_ surprised, but I would be disappointed and think less of the institution/instructor.


Surely this can’t be justified as a ux driven “oversight” - I am not an economics professor but I would be feeling totally sketchy if I manipulated a data source in such an apparently arbitrary manner.

In combination with the (reportedly)blatant lack of any reference to the methodology in his paper this feels like wilful gross misconduct in professor-land.

IMO this sort of thing rightly undermines confidence and credibility of research in a much broader context and should be a sackable matter if shown to be true as asserted.


Also Netherlands to New Zealand. Hilarious.


And United States <~> United Kingdom.

To compare OECD countries.

What’s the point of even doing the research?


Personally, I prefer using data from Uruguay to cover the holes in the United States data.


To write papers


Excel makes such things convenient


This reminds me of an error in the Excel sheet used by the EU to justify austerity programs back in 2008 the led to trouble in the PIGS countries.

Someone double checked that and found that there was an error in the equation is correction show that austerity programs were mildly worse for recovering from economic disasters.

--

On an editorial note, it seems to me that economics is just highfalutin astrology and this does nothing to convince me otherwise


Reinhart, Rogoff... and Herndon: The student who caught out the profs (2013)

https://www.bbc.co.uk/news/magazine-22223190


If you see conclusions drawn from country level data, you’ve gotta be careful. Unfortunately, there is a field of economics that uses such data extensively. Other fields have a, shall we say, weaker taste for it.


Many excuses have been used to justify austerity. If this study didn't work, they'd use another. If 99 studies said austerity was bad and 1 study said it was good, I suspect that one would be more likely to get published; in either case it would get used to justify austerity. It seems to me like austerity is the pre-determined conclusion, and then they go looking for reasons.


I expected to read a paper about some obscure Excel trick to manipulate stats output. Instead, this is just old-fashioned manipulation by hand or "imputation" as the paper describes it.

> In email correspondence seen by Retraction Watch and a follow-up Zoom call, Heshmati told the student he had used Excel’s autofill function to mend the data. He had marked anywhere from two to four observations before or after the missing values and dragged the selected cells down or up, depending on the case. The program then filled in the blanks. If the new numbers turned negative, Heshmati replaced them with the last positive value Excel had spit out.


The crazy thing about it is that the author doesn't seem to understand why it's bad. He doesn't appear to be hiding it. He just says "yeah that's what I did, whoops, I forgot to say it in the paper". He's either decided that acting like a complete moron is better than being thought of as an an intentional fraud, or else he really does think it was totally above board.


I had a similar thought. My interpretation is that he genuinely thinks what he did was ok, because Excel has computer magic.

This quote seems unintentionally telling:

> "If we do not use imputation, such data is almost useless,” Heshmati said. He added that the description of the data in the paper as “balanced” referred to “the final data” – that is, the mended dataset.


""Is the evidence for austerity based on an Excel spreadsheet error?"

Lets not forget that people/politicians also will ignore errors if they support their point of view. This can lead to policy decisions that impact the global economy.

The US 'Right' believe strongly in biblical/moral individual responsibility, so any study that supports that gets a pass.

This lead to 'austerity' measures after 2008. Because "everyone should just be more responsible in their spending".

""They actually found, using a different method, that the economic growth would be around 2.2 percent. Reinhart and Rogoff admitted that their spreadsheet was accidentally omitting 5 rows in an Average formula, and claimed that the corrected result would be a positive growth but of just 0.2 percent.""

https://www.powerusersoftwares.com/post/2016/08/11/the-excel....

https://www.theguardian.com/politics/2013/apr/18/uncovered-e...

https://www.washingtonpost.com/news/wonk/wp/2013/04/16/is-th...


You post on HN, people see it as a UI problem. Everything is tech, everything is tooling.

But if you knew anything the history of science, you'd know that a sloppy scientist with good tools gets you sloppy science, and a diligent scientist with bad tools might still get you as-good-as-possible science.


People aren't uniform, but they're all human.

Improvements in tooling raise the baseline universally, making the "sloppy" scientists malfeasance obvious and helping the "diligent" scientist avoid inadvertent mistakes.

Moralizing about humans being humans and wishing we were better gets us nowhere, but technology carries us into the future.


I'm not moralizing. There's no moral anywhere in what I wrote. Being sloppy isn't a moral judgement.


“The reason it’s cheating isn’t that he’s done it, but that he hasn’t written it down,” Sorry but no. Every day my team takes decisions that have significant risk management impacts and we clearly state the subjective implications. Even when we have numbers to back things up, we always state the nature of our decisions.


There is plenty of software that can do imputation for you. Video of how to do it in our Easy Data Transform software at:

https://www.youtube.com/watch?v=WXAGhtqI5xw

Is is jaw-dropping that a professor would use Excel auto-fill.


> If I had other intensions [sic] and did not believe in my imputation approach, I would not share the data with him.

Admittedly this isn’t my field. But that sounds a lot like “I only show my work to people who will validate it,” does it not?


I disagree, I think it means “my actions have demonstrated that I’m not trying to cover anything up”


That's my interpretation as well.

It doesn't put him in a much better light, though. It's like he's so incompetent that he didn't even realise that there was something that needed covering up.


There wasn't something that needed covering up. If he'd reported the methodology in the paper there would have been no issues.

It's likely that this was omitted from the paper as an oversight; such imputation likely wouldn't have affected the outcome of peer review.



I'm not surprised that results of this study were not surprising.


This is called "econometrics".


This is not econometrics though. Actual econometrics is upfront that you have missing data, performs partial identification rather than pretending that you have point identification, and gives you upper and lower bounds on what the true parameter value might be.

Only if you did that with this paper, you’d have bounds so wide that no useful conclusion could be drawn, as indicated by the author’s statement that without imputation such data is useless.


[flagged]


Did you really just make an account to post this comment? That's cowardly, and sad.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: