The good news is I think that with classes being in person again things will go relatively back to normal. Honestly I don't think I can fault students who cheated during the pandemic, it was just so abnormal that most people did whatever they had to to cope.

The exams were incredibly difficult for bad reasons: you had to calculate many iterations of an algorithm by hand. You also had to memorize a bunch of algorithms (rather than understanding how they worked or how to modify them to solve different problems).

The professor also had a "can't do anything about it" attitude, and so the entire class was blatantly cheating as a result, and I got the worst grade of my academic career.

It sort of broke the camel's back for me; I consider our current approach to academics to be fundamentally broken.

It's harder to design exams where people cannot cheat, but it's certainly possible. Randomize the orders of questions on the test. Have multiple versions the same question (e.g. in a math exam, change a 2 to a 4, or a sin to a cos). Now you have unique exams for each student.

Most importantly, design questions that require critical thinking, and allow everyone to use their computers/books/notes/whatever. That's far more representative of real life anyway.

These simple changes would both make cheating very difficult, and result in better learning: memorizing a bunch of crap the day before an exam will have basically no impact in my life. Learning how to reason critically about the subject matter will give me lasting benefits, even if that subject never comes up again.

This ProctorU stuff is just quadrupling down on a fundamentally broken approach and making it much worse.

We were allowed calculators too, but the problems were designed that anyone even mildly good at mental math didn't need them. In theory you could teach all of calculus without ever even using numbers (perhaps except 0).

Yes.

It's been so long since I took calculus but that's the reason why we weren't allowed calculators in my Calculus class. All the formulas needed for the test were also provided (as we weren't being graded on our memorization skills either)

My professor also said don't bother doing simple arithmetic just to simplify an answer - leave it at 12/56 or whatever.

Turns out, it was a really, really great system.

Mathematica, on the other hand...

Lets see if the kids know how to multiply fractions:

factorial(256) times ( 1 divided by factorial(256) ) equals ?

And yes, there will unfortunately be kids trying to brute force it by typing it in instead of cancelling the factorial(256).

I think the 89 is awesome for algebra because typing in an equation and doing an ans(1) with an operation will let you do the algebra process without making stupid arithmetic mistakes. I wish someone made a calculator that does this only and not one that does all the other CAS tricks for integration and derivatives. Oh and I wish they would make it RPN but that is a whole other story.

Another common trick: looking at your keyboard isn't cheating. Any small display secreted on your keyboard will go unnoticed. You will just look like you aren't a great typist, someone who needs to look at the keys.

Want to go real hardcore? Cut into your laptop and send the screen/keyboard/mouse signals to a different monitor. Then you can have an actor sit in front of the webcam pretending to take the test. I doubt they are correlating keypress sounds to characters appearing on the screen... yet. As any security expert would tell you: if the attacker has unfettered access to your hardware then all bets are off.

I had such a numerical methods exam, I had a python script ready for it. No regrets on automating something that should be automated. I understood how the method works and wrote code for it.

We were allowed (and encouraged) to write cheatsheets for most of our exams at university. However, they had to be a single sheet and handwritten.

The reasoning was that in the process of creating a cheatsheet, students would reinforce their learning. The handwriting requirement was to encourage us to write them ourselves, instead of just printing one off, and the size requirement was so that we had to think about how to distill a semester of learning instead of just trying to copy the entire textbook by hand.

I really liked the system, and I feel like it did help more than just allowing open book exams. I remember of the few exams that were open book (some even allowed us to use the internet, but with a ban on communicating with others), I usually got a worse grade, as I ended up putting less effort into studying.

https://nakedsecurity.sophos.com/2011/03/09/a-complicated-ca...

He convinced one professor, possibly two, but if he had any more luck than that, I wasn’t in those classes with him.

Most of what we do is synthesizing facts into knowledge. But I know from copious personal experience that there is a wide grey area between recalling a fact at will and forgetting it exists entirely. I am fairly good at recalling concepts in general terms. Y might be applicable in this case, but I need to check the details to be sure/explain how.

Which is also why I think banning computers from planning meetings is stupid. You want us to agree to do things without being able to spot check any of our hunches that will drastically effect our estimates? That’s a winning plan /s.

My school's CS department defaulted to allowing students to use almost any available resource (unless specifically indicated otherwise). But you were required to write down what resources you used.

So a professor might assign a whole pile of nasty algorithm proofs, and say, "You're allowed to work in groups to find the general solution. But you need to write down your group members, and you need to do your own writeups." Tests were in person but not especially proctored. The penalty for cheating was a heavy suspension and a transcript annotation, with permanent expulsion for a second offense.

This actually worked pretty well, back in the day.

That’s such a strange response to that expectation. For the class I teach, I made my exams open book, open notes, open internet and gave them a extended time to work on the exam. If they’re gonna cheat, they’re gonna cheat. Don’t fight the tide. But the questions I asked where analysis/synthesis questions that couldn’t simply be looked up, and student performance was on a bell curve with a 80 average (in line with previous years). I think a lot of professors need to embrace the situation and adapt rather than forcing a police state on students during exam times.

Cheating is never OK. Look what that lead to. A few bad apples ruined it for the other students. People have to learn, that their behavior in such a setting does not only affect themselves. In this case it very negatively impacted everyone studying with them.

at some point it's like defensive driving. Sure, they are at fault and an idiot, but for the sake of your own safety you shouldn't "stick to your morals" or just "hope they get theirs". Look out for yourself, because for some people that's the only person that will do it.

With that lens, yes. I don't blame people who need to do what they do for one day in some artificial environment so they aren't impacted in their transition to real world work where they are otherwise moral.

Except, for many competitive positions, that's a zero-sum game.

Their solution was to let us use whatever we wanted except other students. Cellphones, textbooks, laptop, Internet.

You had 2 hours to answer 2 or 3 questions, it was definitely reasonable if you studied... But the questions were insane.

You know how homework would be assigned 1-50 odds? These were like the problems 74 and 76, 1/4 page long and difficult, multiple parts.

That seems as similar as engineering school got to the real would outside our sophomore and senior design projects.

Yeah, it's crazy that cheaters are the protagonist of the ban e-proctoring story.

What if the university only admitted students who don't cheat?

How do we know that policy works for real at a place like Caltech?

Academic culture is accommodating, and you feel that way too. But universities must know that cheating harms everyone, it's a losing proposition. The right response should have been to suspend grading.

It’s a common fallacy to attribute behavior to innate qualities of people, rather than situations.

Personally, I removed exams entirely in the last courses I taught in Spring 2020, and it seemed to work out OK. I had no faith that the proctoring systems would catch everything, and I had no desire to inflict them on my students in any case.

One variety was legal - you’d have a doctors note to get a reasonable accommodation for ADD. In the 90s, it was pretty trivial to do that. Once you had that, you were taking tests in a facility that made cheating very easy (open book) if you were motivated.

The idea was that the A section was the basic plug-n-chug versions of the problem that everyone should be able to get if you were even half paying attention in class; the B section is stuff that requires a bit more reasoning and analysis, and may have a pitfall or two, but they'll be things that were at least mentioned; and the C section is effectively extra credit— a problem where you needed to have a deeper understanding in order to figure out something novel about it, or like you needed to re-derive something from first principles in a way that was slightly different from what was shown in class, that kind of thing.

And the sections weren't equally weighted; I think it was usually like 50/30/20, or even 60/30/10.

Some people hated this, but I really liked how explicit it was. There was no mystery meat— the whole evaluation scheme was very open about its design and intent.

On one hand I really like the former approach and could understand the latter (organizing resits is likely a huge PITA). On the other that's the only professor I heard of that was attacked with an axe by one of his students.

How did that end, if it's not too weird to ask

> normalizing the lab scores between different reviewers

Sounds like a good idea to me :-)

I had experience taking undergraduate courses at 4 institutions in the USA and of course many departments.

I had some professors that expected the grades to fall like grade school did with 90-100 % being a high percentage. Others gave 4-6 difficult multi part problems and graded on a transparent curve. Others gave difficult problems and were not transparent about the grading scheme.

But let's look at the reality of a student at a normal institution or trying to get into college today; it's a series of high stakes exams that decides your future. And a prisoners' dilemma since as soon as someone cheats you have to do it as well just to keep up. This used to be high risk high reward but now, thanks to the pandemic, it suddenly became low risk high reward. No wonders everyone is doing it.

Really, it's a great equalizer. Pre-Pandemic, it was a common strategy to get "diagnosed" with ADHD to get extra time during exams. But for that you needed well connected parents. Now everyone can cheat.

these are university students pressured to have high grades for internships/first jobs, not some impressionable elementary schooler. Thinking way too obtusely for this to be a factor and compare this to some middle schooler getting into drugs.

We had the students turn on their webcams just to discourage them sitting in the same room, which basically would negate our former efforts. Neither did we flag any behavior nor did we record anything or used some automatic software.

I believe that worked quite nicely. A few cheaters were easily spotted when they had the wrong constants in their answer.

I understand that not every course and exam style is suitable for that. But for our databases course, it was applicable.

I think that this was not much more invasive than a traditional exam. Anything more, like watching the screen, or having audio, would make me feel uncomfortable.

When I was studying (physics, early 2000s in Germany), we didn't even have a honor system, we had the assumption that everybody is there to learn something. If you cheat, you harm yourself.

Progress was assessed by weekly excercise sheets where you were encouraged to work together with others, and to learn how to use the library to find solutions. After each semester there would be a test for each course, and you would get a certificate if passed. Attendence to lectures was usually not checked. The tests were "closed" and watched, but if you were determined you could probably have easily cheated. Anyway regular cheaters would have failed miserably at the latest in the oral exams or the lab courses.

I think a major function of universities is to teach people to be independent and responsible, and if we school-ize them we loose a lot.

I'm speaking, of course, of measures like expulsion. Too much of a hit to the bottom line, too much messy litigation, and too much bad PR.

All of this is another way of making your point, which is that the core mission of universities has drifted away from idealized learning to big business and sports.

I suspect the entire thing needs to be nuked from orbit before it can be rebuilt and I'm happy to see COVID accelerating the process.

Another issue with the purportedly "optimal" testing situation of creating unique challenging and deep questions for each test (and even for each student). It's difficult to make good questions. Sometimes they are too hard, are not as clear as they seem since students might misunderstand and go off in the wrong direction giving you some trivial answer to a similar-looking question. The safest questions are the most shallow. And deeper problem-solving questions may take a few iterations of "testing" (as in trying them out on exams and seeing how students do). So what seems like the best kind of question will actually be the riskiest and least-tested.

It's like having unit tests and integration tests in software testing.

It should not matter if you have the exact line "To be or not to be" verbatim if you can explain what Shakespeare tried to convey about suffering and the meaning of life. More importantly, you should be allowed to have the book! You can't expect to jump from confusion to comprehension by reading the whole thing during the exam.

It shouldn't matter if you can regurgitate "Miranda v. Arizona, 1966" or not if you can explain when a person is giving voluntary statements or is under custodial interrogation. Access to the full PACER database wouldn't enable you to understand those distinctions.

And as comes up here all the time, it shouldn't matter in a whiteboard interview whether you have the arguments reversed in your strcmp(), an editor, compiler, or reference (or just running the code) would show you which way it was supposed to be. The important thing is the planning, algorithm tradeoffs, data structures, and debugging process that you used to write the thing. Heck, it shouldn't even matter if you can remember that the function is called 'strcmp'.

Reliance on these factoids usually indicates that the grader is taking the easy way out - scanning for expected quotes, for case names, or compiling the code and seeing if it works or not - rather than gauging comprehension. I'll grant that it is far easier to do the former, but that's not what the goal is supposed to be.

Edit: What can't be allowed is regurgitating someone else's comprehension of the question while having none of your own. If that's a possibility, you need to either ask different questions or have some way to prevent people from accessing that explanation. Remote exams need more of the former and less of the latter.

Isn't that just testing for memory rather than comprehension?

All of my BSc Applied Physics exams including the final (Exeter Uni. 1977) were open note and this really helped sort the competent students from those who thought that they were at university to learn facts. The latter brought fifty litre rucksacks full of notes in to the finals and without exception either failed entirely or scraped through to third class honours.

They seemed to have not realized that the faculty would not set questions that could be directly answered by looking up the question in one's notes. The best example was the final quantum mechanics paper which did not contain a single question that had a direct answer in the notes we had taken or the textbooks we had used.

My most recent exam was a C# course and that exam was open book. It wasn't as extreme as my QM final but had basically the same idea: apply what you have learnt.

I want my lawyer to double check the sources. Even if he has the case name memorized, joe vs sally is easy to get confused with joe vs sue (or sally vs joe...), and those details are important to get right, lest I lose the case on a technicality.

There is a time to memorize details, but for the most part I want people who verify before making a statement.

You are making assumptions based on your own experiences, and they are incorrect.

You have a lot of faith in for-profit entities.

Poland has a completely different culture from Western Europe or the US. It's rapidly Westernizing, but it's still a world of difference. I think it's a pity. I prefer Slavic cultures to Western. One of the things which bugs me most is how modern Poland is starting to have the same surface, insincere friendliness as the West.

- EU membership is clearly a net win, but doesn't help preserve Polish culture.

- Many of the cultural differences are not recognized on either side -- people say the same things sometimes, but it takes deep digging to understand that they mean very different things.

- Historically, Poles also looked up to and tried to emulate the West, which is part of the reason the culture is shifting so rapidly.

So the change was rapid, but far from complete.

Apply again next year? In Germany, a university expulsion usually comes with a lifetime ban on studying the same subject again.

This of course may have changed dramatically in the last two years given the major shift in how we all live but I would suspect that this is more common than you think.

Where there ever any rumors of that kind of thing. In a large enough class someone will be tempted to cheat, and if the school is taking decisive action there will be rumors even if they are making great effort to protect the students privacy and make the discipline a learning experience

The people I knew on a personal basis weren't the kind to do that or really care about it enough to do so and without going into a story about the people I know and how we're all doing well in life you'll have to take my word for it.

Well, that wouldn't be a problem to the person you're answering to, considering German universities aren't businesses who take people in for capital (both financial and social).

In France it's certainly very easy to get yourself thrown out for cheating or not showing up.

Typically there's a faculty union or senate which negotiates policies with the administration. Individual professors aren't all alone in the fight. Though they may individually choose to give lax punishments or look the other way.

If you actually learn something in a class, that distinguishes you from many other students.

I agree though, university is not a good system for determining who is capable and deserves the best jobs. It’s unfortunate that our society leans on it so much as a sorting hat.

Applying a panopticon to children to prevent cheating is questionable since it normalizes authorities using electronics to bully people. Applying it to adults at a university is arguably a human rights violation and a failure to write good course materials.

That would have been fun! The only time I did something like this was when I tested out of one of the intro to programming classes. I was one of the last students to talk to the professor that ran it before he left.

Vivas work,they are just expensive. The alternatives are cheaper but don't work.

Of course it could be that different people have different goals in mind. Some just want the diploma, others want to master the subject. So it depends on what the diploma is supposed to mean. If it just means: turned up then a viva is of course unnecessary and wasteful.

The problem is, I knew some of the guys who graduated, with extremely low GPAs, who didn't cheat, have the same degree as me, exited with the same student loan debt as me, and can't code their way out of a wet paper bag. I've ran into some of them working retail jobs back in my home town.

It's a fucking travesty. They had to be saddled with a student loan debt load they would have no chance of paying back. I only paid mine back so early because of a lucky break on an insurance claim. I'd still be paying on them, 20 years later, if I hadn't lost everything I owned in a flood.

Universties talk about how they don't want to be seen as vocational training facilities. Well, they need to admit the reality. Primary education is not preparing people for the work force. We give people a free education for 13 years, and if they want to have economic opportunity, they want to do something more than just manual labor, we can't give them another 2 to 4 years? It's bullshit.

Your type of social group is not the one that this behavior happens in.

Same results as always. Exactly the same.

Someone will have cheated. Well, as always: I am sure people cheat at my exams as they do at any other.

I do not repent and I do not think I would do otherwise.

*However*: I DID spend a lot of time preparing materials, attending tutorials, replying to emails, correcting my lecture notes, recording videos...

I honestly think my students acted in accordance to my attitude: if you are honest, we are going to be as well.

People asked if he was concerned that people cheated or wouldn't learn anything and he once gave an insightful comment m that stuck with me: if people cheat, and they will, they're only cheating themselves.

He was right because while almost everyone passed his classes, not everyone ended up with successful careers. If they did, they probably had to pickup the sort of knowledge he crafted together like a masterful chef and hand fed us. The education is the product, not the degree. The degree is often a necessary rubber stamp to get past the first interview filters but it's not going to help you in most positions for the subsequent interview or actual job demands. You're really only cheating yourself of the education because the education is the real product here.

Things have changed a bit and you can find pretty much anything online readily, but you have to chop up the information, prep it, and craft it into a meal. Over time these skills improve and you too can become a knowledge chef but your (good) university programs give you the tools to become a knowledge chef in your field. They start you out as a cook who knows how to do certain things but they also give techniques you need to make your own new dishes. That's what you're paying for.

That can hurt the school's reputation if it becomes known for passing students who aren't actually prepared to do the work. Employers would prefer to recruit from schools where having a diploma actually means one is capable of doing what the job requires.

But I think, most importantly, I put a lot of emphasis on group work & home work. This has also its own issues. I like to do group homework because I believe that in the real world, you rarely work alone, and group work allow to have more fun project than what you can do in a 1-2h exam. On the other hand, I know some student hate it because there is always a slacker in every group. And home work is also an issue because not every student has time to work outside of school sadly. I try to give time during the classes for them to work on their homework but it is not all the time possible (limited number of hour for the whole course).

Overall, I hate grading and exams, I consider that my job is teaching, not grading, but this doesn't really work in the real world sadly.

My experience is that there is sometimes somebody who is not a slacker, and it's usually the only one doing the work for the entire group.

IRL:

- people has much higher stakes, while in school tanking a class project is no big deal for most slackers. IRL, they can get excluded, isolated, shamed, fired and so on.

- such group would exist rarely in the first place, because institutions filter people;

- if you are in the rare situation it happens, you can get out of it, but you it's very hard to move in the school structure, not to mention out of it.

Anecdotally, I've seen the scenario playing out many times in schools, but rarely in a restaurant, a dev team, a NGO or a sport club. There are productivity differences, sometimes immense, and an occasional slacker, but that's not the same order of magnitude at all.

We (as a species, really) are bad at interviewing lots of people that we don't already know well - just like we're bad at testing academically, surprise! - and we're also bad at coaching/firing/managing them.

Real world work is nothing like group work. In real world, you can carve part of project on yourself and be judged on that. You can complain about co workers doing nothing. The teamleader deals with interpersonal or productivity issues.

And ultimately, you can leave team or find new job if above dont happen.

> group work allow to have more fun project than what you can do in a 1-2h exam.

Or someone else from the group do the fun parts and will try to offload only boring parts on others.

School group work is less fun then individual, unless you are lucky to work with really good matching people. It has none of the processes and structures that mediates political issues in real work.

It seems like with software, you could easily write math questions with different inputs, as you suggest (in fact, our homework software already did that for math last semester), but it seems that those tools are not built into the testing software so much. It definitely seems like a solvable issue though!

International students pay significantly more, and there are often 100 students or more in a class.

So it’s also not a great solution to put the burden of overworked professors onto the students by invading privacy and making them use poorly built suffrage to do so.

The trouble is at the institution level.

https://community.canvaslms.com/t5/Instructor-Guide/How-do-I...

It's a whole platform for e-learning. The exam system is ... usable. The fine arts, law, and social silences like it very much, because long form answers are readable.

The natural sciences with the math bit have to work around it. We manage.

I don't believe “e-proctoring” companies are capable of it, either. I got disqualified from an exam for five `onblur` events (while my screen was being recorded), and it kept counting while the “DO NOT CHEAT” lockout message was displayed; if that's “works reasonably” then I don't know what isn't.

We spent 6 months on it and as far as I heard was still in use as late as 2015.

You don't have to create a whole system; just a database that prepares variants of questions, and a method to put it into the existing exam software.

Why is it unreasonable? What level of duty do you perceive course instructors having to achieving positive outcomes for their students? Who is responsible for pushing improvements beyond the trough of the status quo?

I'm of the opinion that e-proctoring, whatever its privacy concerns, is fundamentally an exercise in "doing the wrong thing faster". Our exams are a poor reflection of student ability. Our exams have no bearing on actual proficiency in the subject matter. Our exams are ineffectual at catching cheaters. Leaving aside any discussion of privacy, e-proctoring (and traditional proctoring) fails to accomplish its fundamental goals.

To automate these exams, to make them ever-more scalable and easier to distribute, is not a victory. We've merely perpetuated a flawed system.

If we want a meaningful assessment of student ability then we need to use better methods. Dethrone exams from our curriculum. Leverage project-based assessments. Use oral exams where feasible. Replace the infinitely looping lecture halls with recorded videos and open-source textbooks. Use the recovered instructor time for something meaningful. This is not a Gordian knot. It can be solved with a little courage and a little pragmatism.

A friend of mine is a University lecturer for physics and one of his students gave him an invite to the discord channel where half his students are in there sharing all the answers during the final exam.

He basically told me there is nothing he can do about it.

He talked to the head of department and they basically said if you have proof of specific people cheating then they can escalate it for those people but short of that, nothing can be done.

Not that I’m saying these systems are great either. But I really don’t know how exams in the current form can be conducted remotely without endemic cheating.

Maybe systematizing everything about education is the wrong approach.

For a class of 30, this took the professor 15 hours (three hours a day over the course of a week) to have all of the students go through the exam process during exam week.

Trying to scale this up for something that can handle ~100 students becomes difficult.

Another part with this is the objective vs subjective evaluation of the student.

With the "I got a poor grade because the professor didn't like me" type issues that come up, being able to objectively point to "this is the score on the homework" and "this is the score on the final" allows that subjective grading to be removed and protects the professors from students who feel that the grade they received was influenced by external factors.

While systematizing the verification of material learned may be the wrong approach, it is where we are for a number of factors that the university doesn't have too many options (allegations of professor bias, even if unfounded, still take up a lot of time; class sizes for some classes go beyond what can be reasonably verified on a 1:1 basis)

It's not scalable, and it's way too open to professor bias.

Except for in some niche seminar classes, it's a non-starter.

I pretty much know if I learned the material or not. A test is nice to confirm this for myself but the way we have put test scores above everything is just ridiculous.

It is all the symptoms of credentialism. Imagine taking the same class for free that you don't get "credit" for. It is not "cheating" then if someone else gives you the answers to things you don't know. It is just a waste of your own time. Why even bother taking the class.

Instead we have people paying tens of thousands of dollars basically to focus on test taking to get credentials that no one really cares about anyway instead of actually becoming more educated.

The irony to me is I always got good grades when I was younger and in school because I have always been a great test taker. I didn't have to "waste" much time actually learning the material. "waste" time actually benefiting from the material.

Yes, but have you never met someone that was convinced they knew the material when they in fact didn't? They're everywhere! Accurately assessing oneself is a skill that needs to be taught for a lot of people, and an important part of it is by having people fail tests they're not capable of passing.

A major component of the university experience is collaboration and networking. Dividing tasks and plugging holes in each other's understanding. Bouncing ideas off other people to test your own understanding and verify against theirs.

Any good university isn't simply about grades and there's a limit to what you can learn online. Credentials exist as a minimum filter, not as a mark of competence. One cannot be tested for an expressed numerically.

Testing is itself a form of learning, because you are asked to recall facts, concepts, and practice skills.

Signing up for a class with a hurdle at the end I need to jump over is exactly what I need. Free online classes are like gym memberships, but even cheaper. Most people won't stick to it.

• Unless it's recorded, you can't easily get another opinion on how to grade it, except from those at the event

• Some students will naturally be better at interview-style exams, independent of their skill in the relevant material. This is also the case with written exams, of course, but I imagine the 'skilled bullshitter' types might do better in an interview-style exam, and a nervous student might do much worse.

• I imagine it would be tougher on students who don't speak the language natively

• The 'imprecision' of a wandering organic conversation might make it harder for the student to know what to study

• Different students don't get the same exam, which might raise questions of fairness in itself, but it's worsened by that the examiner's mood is likely to change. (With written exams, an examiner can mark all students on the first question before moving on to marking the second question, to protect against this.)

• Physical appearance and deportment might skew the examiner's judgement. (This is widely believed to be an issue with juries, after all.)

• You can't hide the student's identity from the examiner, as a measure against favouritism. With written exams this can, and should, be done

The largest part of the grades came from delivering projects, where you were graded both on result and process (you would present an end-product, and deliver a report documenting the product & the process of creating it). At the start of each project you would get a document explaining what categories the product & presentation would be graded on and what a failing, passing and excellent grade would look like, so you had a clear goal.

I have the feeling the system only works with great teachers (which was the case in 90% of the teacher I had during my 4 years).

They would also have feedback rounds after each semester and actually change things based on feedback, applying the design process on the course itself.

It was a really great experience and I would whole-heartedly recommend that program to anyone interested in design/programming.

We had oral exams in university. They are less fair then written ones for sure. For the record, I did well and tendes to know what to say. Could also bluff my way when encountering edge of my knowledge.

And seen a lot of bluffing, appeals to emotion, negotiation for better grades or special advantage. Once I have seen a guy negotiate C all the way to A - and no, no additional questions were asked.

Oral exams are not fair in any way.

Though I suppose it would make people a lot better at job interviews if done for every class.

Another thing to note is that undergrades' first job usually depends on having good grades. Some people will get lower grades and those will complain to the point of suing the university into having a more seemingly fair and unbiased process. (Note actually fair, but I'm pretty sure whoever will get a low grade on the Socrates style conversation will be pissed and demand something else. This kind of mentoring only makes sense in the PhD level really, where grades matter less)

When I was in university (some time ago!), it was always the large courses that were easiest to game. They were also generally the subjects I was least interested in.

My small comp sci courses were amazing in contrast. Easily worth the cost of admission. I think the education system would be much improved if we had fewer, but significantly smaller courses.

So, you pay the same price, but instead of 100 courses of massive size, you get 25 courses of small size.

A friend of mine copied an assignment right down to the name and student number as he didn't even bother to read the whole block of code so also copied the comments with the submission information.

He got 100%, but was told not to do it again.

My experience as a TA are quite different, though this in in Europe. Especially with the algorithmic courses there was quite a lot of copying since there were multiple TA's and they thought we didn't compare notes. Several of them got reported (you get a mark, two marks is suspension, three is out) and I know of at least one case who got caught again at a different course and was suspended.

For a programming course for first year engineering students we didn't care they copied each others code but just asked students to explain specific parts of the programs they handed in. Those who copied failed mostly instantly and got one retry, and most at least tried to understand what it was about after that. Though copy/pasting and not even changing the student's ID numbers was an instant fail.

I was a part-time visiting lecturer in computing, paid by the lesson. Cheating was rife; I often got duplicate submission of coding exercises. Scoring was mainly by coursework, not exams.

I asked my colleagues what to do about it. They said:

1. You can ignore it.

2. You can report it; you will be accused of racism (the majority of the class were people with brown skin). You will be required to sit on an exam board for several days during the holiday season, unpaid.

3. You can explain to them that collaboration is encouraged, but submitting work you didn't do as if it were your own is cheating, and not permitted.

I took option 3.

That's just sad.

> 3. You can explain to them that collaboration is encouraged, but submitting work you didn't do as if it were your own is cheating, and not permitted.

That gives you a great excuse to fail duplicate copies, since they clearly didn't follow the rules. Just split the grade across all duplicates (so something you received three times gets 1/3 of the points). Add a note saying that they can get more points if the other duplicate authors agree to transfer them points.

Watch the chaos.

Basically, I was advised that it's fine to explain the rules, but to actually attempt to enforce them would cause chaos - for me.

No, I don't really advocate doing that, but it's what first came to mind.

If the answers to your final are easily Googled then you’ve written a shitty final.

Allowing students to take exams alone, at home, unsupervised is just a recipe for disaster without surveillance. It's not just Google but any sort of help in the room or remote, which can go all the way to someone else taking the exam. That really simply is the reality. I don't think this is contentious and I am surprised by the debate.

On the other hand, it is of course possible to discuss how far surveillance should go and how perhaps to avoid too much intrusion (as mentioned in my previous comment). That said, with the expensive connectivity people have these days, in addition to all they can think of if the exam takes place at home this is really an arm race situation.

So, again, the alternative is good old in-person, on-site exams, where students are also under surveillance, by the way, but that feels less intrusive because students aren't in their own private space at home.

Since when? My discord account doesn't have my name or phone linked at all, just email.

Then to our relief the education authorities in our region straight banned the practice so we just drew up a document with various ideas about how to go about online assessment in a humane and reasonable way and we went through thousands of online exams with very few incidents.

One of the profs decided to make it impossible to cheat by basically making the exam so hard that even the A+ students would have a hard time solving it. This worked well in preventing cheating because for the A+ students to collaborate with others they would need spare time. On the other hand it left lot of students very demoralized.

Some of the other profs came up with question randomization schemes. So the assesment tool would shuffle questions for different students. People were still on discord channels but to not much effect. Of course this required that the professors (or the TAs) were conversant with basic programming skills.

For humanities oriented assessments however, I know most of the profs chose to do things via project work (it's a bit hard to cheat when writing essays. You can't possibly have the same argument as someone else so...)

One thing I can't help but wonder is that I realized when we moved things online we missed out a lot on student interaction. Students were generally far more aggressive for their own grades, often at the cost of other students. Furthermore, I found students were far more ready to abuse the teaching staff than when we were physically there. Conversely, teaching staff was far more strict and stressed out than before. I can't help but wonder if the same courses were conducted face to face whether there would be more kindness all round.

The professor handed back the exam the next lecture with the percent on it. I was looking at a 50% on my exam. This was a Tuesday Thursday class (rather than MWF) and so was an hour and a half long. I'm staring at my exam thinking "F, maybe a D, hope I got a C, I'd settle for a D, probably an F... hope I got a C" all through the lecture. Just before the bell rang, the professor wrote the curve on the chalkboard.

* 100 - 75 : A

* 50 - 75 : B

I don't remember any of the rest... I was stuck on "I got a B!" As I left, my TA congratulated me for getting a B.

This is true of copying and collusion generally, but not of commissioning unique work from an essay mill, which is shockingly prevalent in humanities subjects for this very reason. I am aware of no effective defence against this other than in-person exams or oral vivas. I’m glad I’m not a humanities lecturer right now.

I think if universities want to mandate use of e-proctoring software, they need to provide temporary/burner devices too.

The issue is the fact that I have my camera on and there are people who I do not know who can see me and my room. I don't know how this information is being stored and who is seeing it.

Regardless of the client-side software that you can't trust, there are also third-party servers that will store all the recordings with personal information for a potentially long time, which is unacceptable and isn't fixable by temporary hardware (as we don't have a temporary face).

The proctoring process is actually pretty simple: I have a Google Chrome extension installed that I enable when I have to take an exam. It takes 5 extra minutes before the exam: I have to show my identification, the materials I'm using on my desk, my ears to check whether I'm using wireless earphones, and do a quick sweep around the room. It records my screen, my webcam, and my microphone. Of course, the system is not fool proof (I've heard some students use post-its on their display), but communicating with other students becomes nearly impossible.

My roommate is actually jealous of my proctoring. He does not cheat, but knows most others in his year do. There are groups of students who meet up and take exams with each other. As a result, some of his peers consistently get higher grades, while my roommate clearly put in more effort and is more capable of achieving a high grade on his own. Because the barrier to cheating is so low, it almost becomes a requirement to cheat if you want to achieve grades that are high relative to your peers.

I do not believe proctoring is a breach of my privacy. Google Chrome's sandbox is good at explaining what information the extension is requesting, and when it is turned on. Chrome's battle-tested sandboxing makes me confident that the extension is not snooping through my files, for example. It only sees my screen. I can hide things I do not want the online proctor to see before the exam starts. Similarly with my room, you can hide everything that would breach your privacy before the exam starts. Of course, online proctoring is invasive, but I believe students should think more carefully about the dilemma our teachers are facing. Lack of online proctoring discourages smart students, discourages learning, and hurts the reputation of the university in the long term with unreasonable diploma's. This pandemic requires flexibility from everyone, and simply crying "privacy" without considering both sides is short-sighted. The data recorded for online proctoring is reasonable, and does not bring us closer to any kind of "big brother" scenario.

The need to hide things is itself an effect of privacy having been breached. Privacy isn't just to keep information secret, but also to provide a space in which you can be comfortable because it is your own space. Needing to hide things away is a reaction to privacy being breached, not a way to preserve privacy.

Furthermore, you do not have to take the exam in your own room. Any quiet room where there are no other people is theoretically fine. Having access to such a room is your own responsibility, just like having access to a laptop to do your study is your own responsibility. In practice, hiding private items in your study room is usually the most practical way to get access to a comfortable room, but I do not believe requiring students to have access to a private room for 2 hours breaches their privacy.

I think I see the difference here. You're seeing "privacy" in terms of information being leaked. So long as no information is revealed, then no privacy has been lost. The information security is key. Is that an accurate way of describing your views?

For me, "privacy" is a state in which effort does not need to be expended to maintain information security. The cost is not the personal information being leaked, but rather the effort needed in order to prevent information from being leaked. Of knowing that your sanctuary has been violated, that your comfortable place has been exposed to others. And sure, you may try to minimize other effects of that breach of privacy by hiding away aspects of yourself, but that is a response to an invasion that has already occurred.

To use your analogy, suppose I'm going on a trip through the desert. I know that water may not be readily available, so I take several gallons of water with me. On the other hand, if I'm going to a restaurant, I can assume that water will be available and do not pack my own. In neither case have I died of thirst, but in one case I have needed to spend additional effort to ensure that was the case. If I hide things away from the camera, I haven't had information leaks (died of thirst), but I have needed to take extra precautions (carrying several gallons of water) due to the breach of privacy (travel through the desert) that has already occurred.

> Furthermore, you do not have to take the exam in your own room. Any quiet room where there are no other people is theoretically fine. Having access to such a room is your own responsibility, just like having access to a laptop to do your study is your own responsibility.

During normal times, when there may be publicly available study rooms at a university, those options exist. When those same study rooms are closed to stop the spread of a pandemic, or when those study rooms cannot be booked due to lack of availability, a person's private room may be the only room available. In that case, the requirement of exposing one's exam space implicitly requires exposing one's personal space.

> I do not believe requiring students to have access to a private room for 2 hours breaches their privacy.

Requiring students to have access to a private room isn't the issue. Requiring proctors to have access to the student's private room is.

It might be a little much to ask for during a pandemic where people are stuck in places they didn’t expect to be for a long while though, with a whole bunch of other people also stuck inside.

I did not take the exam.

I assume at significant cost?

I feel similarly strongly about my dominion over my own computing devices and the boundaries placed on their intrusion into my environment.

I wonder, at what point should this reasonably be considered equivalent to a religious belief and provided the same sort of protection.

Same here. If I ever need to take remote exams like described I'll get a cheap laptop, or something like a Pi attached to a TV/monitor, and take the exam on that. They can have me install anything they like, it will be on a guest wireless network that can't cross to the rest of the local network, will be freshly installed with an account that is not one of my primaries, and will be wiped down when the exams are done. In fact I have an old slow laptop that would do, assuming it still works.

Of course, not everyone has the luxury of being financially and technically able to do that.

I'm the same with work. If work wants specific control on the devices I use, then they need to provide devices instead of taking control of my personal ones. I did install the comms software on an old phone in 2020 but if the ability to remote-wipe this granted is ever used (accidentally or purposefully) I'll lose nothing but my "get up for work" alarm as that is the only other thing on the device. If I work from home it is via RDC over VPN to my machine at the office - if there becomes a requirement for more control of the machine I'm VPN+RDCing from then either they provide a machine they control or I stop being able to work out of hours (not a problem for me!) or work from home should that be convenient/required (which isn't often, other than during lockdowns over the last 18 months).

I don't have an adversarial relationship with work (which might be implied by the above paragraph), at least not as far as I can tell, but I find it better for my mental health (and my work, I think, though that is the secondary concern from my PoV) to silo my work and personal life somewhat. I can see a day when extra requirements will turn up, as we work with clients in strongly regulated industries and many of them are actively tightening their internal controls and making increasing demands on their suppliers (including us) to do the same.

> and the boundaries placed on their intrusion into my environment.

That you can do less about in many cases, though again I'd have options. I only have a two-bedroom flat, but the little bedroom is used as a training room, so I could setup a chair and something as a desk in there. All they'll see in the tour of the room is the treadmill, weights, mats, and related bits & bobs.

Again, many do not have the luxury of this option.

I charge $10/megabit/minute as i am an exclusive artisianal cloud. And thats not including badwidth or installation fee.

A relative of mine recently took an English course, and constantly had to lobby for re-evaluation of low grades caused by false positives with Turnitin[1].

The issues were varied, but the most frustrating one was that passages in her paper that were quoted and footnoted were marked as "sourced from the internet". Turns out that would happen with any passage quoted from a book that someone put on the internet, somewhere.

Of course, instructors are supposed to manually review for this sort of thing, but it's such a basic miss. One that's going to get worse over time. People put existing source material on the internet, and it gets indexed.

Also, see this example from Reddit: https://www.reddit.com/r/WTF/comments/h731k/thank_god_for_tu... Argh.

[1] https://www.turnitin.com/

When I (as a TA) had to mark dozens and dozens of papers pre-screening for obviously copied paragraphs was a very nice time saver. Otherwise the only students I would catch are the ones with poor English skills (grammar switching from flawless to flawed and back again is a pretty good tell that the paragraph was written by somebody else).

But the tool is shite. It misses some very basic stuff. Like "stuff in quotes followed by a footnote/citation" is copied, by definition, on purpose.

It also notes things like missing commas as if it's 100% authoritative. Even though the passage in question passed Grammarly, Word, etc. There's no notion of nuance or suggestion, just "right/wrong".

[...] a form of exam proctoring which involves monitoring student behaviour during exams administered electronically [...]

Though it is a bit US centric I think. Here in the UK what I think is referred to as a proctor would instead be called an invigilator. Though again, unless the target audience is international, that probably does not matter.

One would think that after the issues with whorepresents.com and expertsexchange.com (and dozens of other popular examples) from at least a decade ago, people would be somewhat more careful as to how their domain name could be misread.

* proctoring software can be installed before an exam and uninstalled afterwards. It doesn’t need to stay on your pc/browser, and its activation is very obvious. I have no info about permanent security issues caused by such software.

* what proctoring does is mostly recording video and audio (possibly with room scans at the beginning ) and uploading it to a remote server for later verification. Then, such footage is automatically scanned for anomalies. The teacher can then check what happened in flagged content parts and choose what to do. The system does not impose a “cheater” label by itself.

* proctoring systems don’t force a closed book approach. I took open book exams where the only enforcement was “be alone and no collaboration with other people during the exam”.

It seems that some people are concerned with some very specific details of some implementations, or with shitty teachers that say some people cheated just because they were flagged for whatever reason.

Some things that happened to me and I still passed the exam:

* a coworker accidentally entered the room where I was taking the exam and I had to talk with him to send him away.

* I had to change my position since I had setup my laptop in a way that was unbearable for a many-hours exam, and I briefly exited the webcam view.

I contacted the teachers when this happened and I got no issue at all.

Of course some teachers happened to be silly, at the beginning of the pandemics: I heard things like “no drinking, no eating, no restroom” for 4 hours exams. Blame the people, not the software.

2. You are far luckier than most when it comes to your circumstances, i have seen myself people failing exams for similar mishaps to yours.

2. Shitty instructors are not the software’s fault.

2. This has nothing to do with instructors (in my experience instructors have little to no say or participation in the usage of these systems), badly written algorithms will frequently kick students out of tests for non-adverse behavior or failure in facial recognition (once again oft-mentioned in this thread).

2. Some issues, as I said, are specific to certain implementations and don’t invalidate eproctoring as a whole.

There are other concerns, such as they often only support a few proprietary operating systems, however many programs require these anyways, so that isn't a major step back in most cases.

The permanent security issue is that violating a security assertion (eg running an untrusted program) on your machine means you can never trust that machine until it gets completely wiped. I'm well past the age where I have to worry about bullshit like eproctoring and my computing environment has become much more deliberate, but even in college I was running Linux as my primary. Reading how it's being normalized that in order to pass a class you're forced to install random spyware on your own machine gives me the willies. This is not what we should be teaching kids for information security practice!

That hasn't been the case ever since firmware became updatable. Nowadays, even wiping the machine completely is not enough, because the firmware might have been maliciously modified by whatever violated the security boundary. And it might not even be the firmware you'd expect (the SPI chip containing the UEFI); see the recent post here (https://news.ycombinator.com/item?id=27740295) about someone who modified his laptop's EC firmware and his laptop's wireless card firmware, making the first intercept all key presses (the EC is also the keyboard controller) and send them to the second to be broadcast over the radio, completely invisible to both the operating system and the UEFI firmware.

The extension will just share your screen, use your camera and mic.

I wouldn't trust a computer that had spyware installed on it without a full system wipe.

> Soon after Russinovich's first post, there were several trojans and worms exploiting XCP's security holes.

I agree that the tools themselves aren't terrible, but I do believe they're an example of "doing the wrong thing faster". My claim would be that all time-limited exams are bad news.

1. The strictness and quality of exams varies significantly by instructor (as you mentioned in your post)

2. Exams aren't representative of any real-world analogue (how often does your boss lock you in a room and tell you to solve the Lagrangian by hand, alone, under a time limit)

3. They don't actually stop determined cheaters (anyone with the foresight to put up notesheets in their bathroom)

In short, proctored exams do not effectively assess student proficiency in a subject, and do not effectively prevent cheating. Making proctoring more scalable means propogating a bad practice at ever-greater rates.

What I would suggest instead, and what many classes in OMSA have already done, is to leverage project-based assessments.

* Demonstrates learning by actually using the material, instead of hollow repeat-backs

* Encourages further learning since practice entails dealing with real-world complications

* Helps students develop a portfolio so they have more to show at graduation than transcripts

There are many other options though if that particular solution doesn't tickle your fancy.

Though I actually agree here with the people saying this type of software is usually a rootkit by design and unacceptable for that reason alone. I understand why the school did it, but in my experience, we never caught a person cheating on an exam. Where we caught cheating was in project submissions, and that was often quite blatant with people turning in identical code or code identical to code on the Internet, sometimes even identical comments. No need for malware on student's personal home networks to catch that.

There is somewhat of a fundamental conflict in the structure of the OMSCS courses, though. Personally, I don't think there is much value in even having the exams at all on top of the usually extremely comprehensive practical project assignments, for the terminal masters students. But that's everyone in OMSCS. The exams tend to test your knowledge of prior research, which is mostly valuable for people going on to a PhD afterward.

Of course, some students do that, even if you need to separately apply after completing the terminal masters and can't just move straight into it like the on-campus grad students. So I don't really know the answer, but the structure right now is imposing a heavily research-focused curriculum on students who are overwhelmingly not going into research. And the OMSCS program largely exists because of sponsorship from industry that wanted stronger industry-focused CS education for their own workers from a school more reputable than a plain MOOC with no university behind it.

They do an amazingly good job of best of both worlds right now, but it is a conflict as I think many of the exams are unnecessarily difficult for people who are never going to become researchers. I guess it's the classic Google approach to hiring, though. You end up filtering a bunch of students who may have been perfectly good as programmers, but you're probably never going to let anyone through who wasn't, and that is ultimately more important for maintaining the school's strong reputation in industry.

If it installs drivers, I have news for you...

Btw: implementation details. It doesn’t invalidate all eproctoring solutions.

Also, the domain is poorly chosen, “Bane Proctoring” sounds like a particularly terrifying proctoring service.