Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm curious what kind of programs the author is writing (and not testing) here.

A few years ago, I implemented tests in a Django website I manage on the side, and am around 90%+ "coverage". Since then, I've been able to upgrade major/minor versions, add new features, and have deployed around 100 updates. Automated testing before deploying has caught numerous small typos and errors before anything went live.

On the other hand, I'm also writing some games right now, and those don't have any tests. I can't see as clear of a reason to implement them.

Would others agree that certain areas of dev are more suited for testing than others?



> Would others agree that certain areas of dev are more suited for testing than others?

I wouldn't agree at all with that. Code are rules written in way that can be translated by programs into a machine executable format.

Saying that code for some areas is more suitable for testing than code for other areas would imply that the rules are different somehow.

I don't see why that would be the case. I'd rather suggest that certain development processes and -methodologies (or lack thereof) may discourage systematic testing.

The rules that govern games are often expressed on a different level and by different people than the implementation. It's therefore hard to come up with reasonable assumptions to test in the first place.

Add to that the goals are often quite different (e.g. correct code vs the software doesn't crash and runs at an acceptable performance) and you may get the impression that tests just aren't well suited for say games.

In reality, the rules of the game, playability and fun need to be tested anyway. So from a cost-perspective it can make sense to skip out on automated code tests as errors and crashes would surface during gameplay tests anyway and overall correctness isn't a requirement for games anyhow.


Integration tests become difficult when the output is visual - such as a formatted PDF document or image. The best I've managed to do is store a known-good output and regression test against unexpected changes. I work in document composition, and it's very important to our clients that a random business rule does not break the overall look and feel of the document. Unit tests are easy, but don't help with that.

When even minor differences can completely alter text flow you have to basically invert the testing process. You run the program, use your human brain to decide if it looks good, then take that output as the 'good' output for automated testing. That's in contrast to a lot of testing where you can set up the expected results as you write the tests, before the code.

So for me, that definitely falls under the domain of being less suitable for testing. A lot of the usual tricks don't work, and the benefits are less certain.

Since it seems like it'd be very similar in practice, I'm curious how browser devs test the visual aspect (not the parsing or DOM) of browser/CSS rendering - how do they decide the graphical result shown is 'good'?


For CSS and the like, A/B-tests are the norm.

It's comparing output rendered to bitmap images and verified on a per-pixel basis.


A big one is that the game part of making a game requires you to test it by running the game and actually using it. My impression of other areas of development is that it's much less likely that they execute the code they write as close to an end user as that.

My other impression is that a lot of the unit testing advocates are coming from dynamic languages and need to runtime execute their code to find basic errors like typos. I know myself from writing in JS and Lua how much of a pain this is.

The last part is that gameplay code changes a lot, its very much experimentally driven. In my experience of following strict processes like TDD it becomes laborious, hurts iteration time and is error prone because rewriting tests often introduces bugs just by itself.

That said integration tests for bits and pieces of the game engine are gold.


I've unit tested the core domain of puzzle games and rpg games before. It's like testing business code. It's uniquely suited to that. But they can't tell you if some visual thing looks strange.

It seems like testing in games is a huge problem, in that game QA testing is a big industry. I bet puzzle games and rpg games could gain a huge competitive advantage by testing their core domain. If they're long lasting games with lots of changes coming up.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: