Always incredible to read about how problems were first solved. Things like
Lesson Two: TCP is evil, don't use TCP for a game...
Lesson Three: Use UDP...
and the like are all "of course" solutions now, but they weren't then. How do you build something fast and reliable over something slow and creaky? Who knows.
The shoulders of giants have gotten a lot taller since then.
I can confirm this, having been mostly responsible for the network code in Widelands. We exclusively used TCP with parallel simulation (i.e., only player actions are transmitted), and it worked just fine as early as 2007 even without any special tweaks. There are two factors that matter, I believe:
First, Widelands is a real-time game but with a slow pace, so relatively high latency is easy to tolerate. Second, the internet is simply more reliable today than it was 15 years ago, especially if the bandwidth that you use is small relative to the Mb/s connections that are common today.
> First, Widelands is a real-time game but with a slow pace, so relatively high latency is easy to tolerate. Second, the internet is simply more reliable today than it was 15 years ago, especially if the bandwidth that you use is small relative to the Mb/s connections that are common today.
Exactly. TCP is fine if you can tolerate 3-5 round trip times' worth of latency. RTT to the other side of the world is ~135 milliseconds at the speed of light. That's roughly one second of latency for a less than optimal network connection. Probably just fine for an RTS game or a not quite real time MMO.
It is not, however, good enough for fast paced FPS games, racing sims or (space) flight sims. They will keep on working better with UDP as long as there will be latency and packet loss in the Internet.
Also, nowadays, there are (usually) no more slow and brittle dialup modems in place any more. A connection without packet loss is way more common these days than one with.
This makes TCP much more feasible (and sometimes necessary as getting UDP packets through various NAT routers can be difficult).
Back then, they were probably right to use UDP, but I found it funny how all the reasons they gave why TCP sucked were things their workarounds were equally bad at.
All of this engineering and technical analysis is wonderful and interesting, but it doesn't change the fact that Lucas Arts sold this game for a big chunk of change to my teenage self (I did farm work to earn my money) and it was a HUGE, DISAPPOINTING RIPOFF. The single player AND multi-player aspects of the game were shit. It was rushed to production by the publisher and it showed.
I realize that it was a huge technological leap for early 1997, I really do. But the product was shit and we paid good money for it.
Edit: I meant to add something. 11 months after XvT was released, Volition Software released Descent: Freespace. It was everything that XvT should have been. Nobody noticed it but whatever. (I admit I'm probably being unfair since the internet was changing so fast back then)
Problem is that you usually only want a subset of what TCP does, and sometimes the simplest/most straightforward way of getting that is a little reliability sauce over a standard UDP connection.
Most games that use UDP do not reimplement TCP because data that is too old is simply not relevant to the game anymore so it is not resent. If TCP had a way to indicate messages shouldn't be delivered anymore that would be great.
I haven't used a dial-up modem since ~1999, and have had 7 different connections in 6 different cities on the west coast in that time. ~0.5-3% packetloss has never stopped rearing its head, especially during prime time. If you've got a magic source of packetloss-free last-mile connections, I'd love to hear about it.
I live in switzerland and I have been using DSL and tv cable modems since 2001 and I never had packet loss (unless there was something broken on the providers end and subsequently fixed by them - this was something that lasted for maybe a day or two)
Yes, there's pretty much no packet loss in normal conditions if you measure using ping.
But in real world scenarios there are lots of reasons for packet loss, congestion being the most important. Have your little brother turn on a bit torrent client on a DSL line when you're playing counter strike and you will experience packet loss for sure.
Your understanding of TCP is wrong. Packet loss is a natural part of the process. When a link is saturated, a router should drop packets and notify the sender to slow down.
When ISPs engineer their networks to not drop packets they add larger buffers, packets wait at the last hop and instead of high speed you get high latency. Games lag at the expence of bandwidth.
Read up about buffer bloat.
Games should use UDP, as its not important if 0.5% of position updates don't get through, but it is important the get through quickly.
TCP is good enough for the large majority of not-too-demanding use cases. But in some corner cases it becomes more of a problem than a solution, because it is more or less a black box; one has very little control over the number of repetitions (sometimes it's even advantageous not to repeat), the repetition timings, the window sizes, etc.
In these corner-case situations you prefer UDP because you have better control on what happens. Also if you work with some obscure TCP/IP stack implementations you wouldn't bet your life on (that happens in the embedded world), and that you don't feel like debugging, or if datagrams are more convenient than streams for your app(sometimes new programmers are surprised to receive one packet when they sent two - or the other way round), UDP doesn't look that bad at all.
I read once a rant that argued half-jokingly that TCP is useless, because in the end the services it offers are redone at application level. Repetition? most applications do retry a failed operation? Acknowledgement? Ditto. Encryption? Same.
There are, of course, a [heap; I originally wrote "stack"] of reasonably standard protocols other than TCP and UDP with behavior that may be more appropriate for games; many are based on top of UDP. RDP comes to mind, although it's not much of a protocol itself.
> I read once a rant that argued half-jokingly that TCP is useless, because in the end the services it offers are redone at application level.
Half-jokingly? Essentially all services provided at "layers" below the application are, at best, performance optimizations. See the End-to-end argument.[1]
Also setsockopt TCP_MAXSEG = 536 and do small sends with your own ID numbers. When one IP fragment from a fragmented packet goes missing everything stops and then everything from that packet is resent. On lossy networks that bigger packet has a bigger chance of again having a missing fragment so there will be an additional delay.
It works just fine.. for non-realtime games (and has been widely used for such games since forever). I've done the tests - you cannot just "tweak" it, and even if you could you'd be relying on the networking stack to do something it's not intended to do.
You might be surprised. World of Warcraft and Guild Wars 2 are both TCP. It's pretty common for MMOs. It can also work well for some types of RTS games where the added latency isn't a major issue.
They're not real-time, though, in the sense that a delayed message is as good as no message. The tolerance is high enough that a few delays usually don't affect gameplay.
All flash games use TCP (as you can't use UDP in flash). There are many realtime flash games. Many MMORPGs use TCP for compatibility reasons: they are very much real-time, especially something like PVP in world of warcraft, etc. Guild Wars also uses TCP and is very much a 'twitch' based realtime game as well.
WoW is the type of game where it can reatroactively change things when data is received and have it mostly be unnoticeable.
Your spell interruption example illustrates it perfectly - spells can be interrupted if the interrupt is within the 1-2s cast time of the spell. It's adjudicated on server side, where the server knows the spellcast start and sends feedback on the spell results (including if any interrupts happened). If there is minimal delay then it works without being noticed; however when I had significant packet loss, then the spellcasting would 'end' for me only after the response (including check if it was interrupted) was received, and it could be multiple seconds later.
The point is, that WoW can mostly function if the orders and feedback are received with delays, since if some opponent in reality(server) is somewhere else than I see, then it usually doesn't change anything - but for a rapid movement FPS game, that would change hits to misses which is the core of that game.
Didn't play much wow but aren't spell cast times like 2-3 secs in wow? Even with a 0.5-1 sec delay, it would be playable. Never played guild wars
Maybe my definition of real time is incorrect. But you don't point and click in MMOs generally, it is all aoe spells or targetted abilities. There are big margins of time or spell aoe to overcome delays of network packets.
1.5 second cast times with abilities to push that down to around 1.0s. A lot of the interrupts are off the global cooldown in order for players to use spell interrupts effectively.
.5 - 1 seconds of delay may not seem like much, but it actually adds up to be quite significant because of the global cooldown would also be enforce client side and so a player with higher delay would have done fewer actions in the same amount of time.
I disagree. Even back in 2005-2006 - almost a decade ago!, in Guild Wars pvp, it was possible to interrupt a 1 second spell with a 1/4 second one, leaving you with 750ms for human reaction time and latency.
World of Warcraft is not a "twitch" game. Ever multi-screened it? Why make all this stuff up? This may surprise you, but Flash games don't set the standard for quality network code.
How do you build something fast and reliable over something slow and creaky? Who knows
You use TCP, which they already had.
The article describes TCP as evil, but then they go on to reimplement TCP over UDP in a way more inefficient manner, not realizing their problems with TCP were problems with the internet. They'll hardly have been the first ones to do so, either.
> The article describes TCP as evil, but then they go on to reimplement TCP over UDP in a way more inefficient manner, not realizing their problems with TCP were problems with the internet.
Almost every real-time game out there "re-implements" parts of TCP over UDP because the mechanisms that TCP use are unsuitable for real time communication.
If a packet goes lost over a 150 ms latency (something modern game engines manage gracefully), the server will receive a resent packet that contains information from 450 ms ago. Additionally a packet going lost with TCP will inhibit the delivery of successfully received packets that were sent after the one gone missing. Re-sending an old packet (like TCP does) is worse than not receiving the packet at all in multi player gaming.
You do need (semi-) reliable delivery, connection management and flow control over UDP but implemented differently than TCP. When a packet goes lost, you send the up-to-date information instead, not what was relevant 300 ms ago.
TCP is a wonderful protocol but absolutely useless for multi player gaming. Everyone who has been writing netcode for real-time games (there are exceptions like RTS, MMORPG, etc) knows that.
I think “real-time” does not mean what you think it means.
I’ve read a lot of things like “TCP is not suitable for real-time online games”, in this thread, where the author of this sentence excludes RTS, MMORPG, etc. As if these were not real-time games.
It’s not just you, but I had to choose one of the posts in this thread for my response. Saying “TCP is a wonderful protocol but absolutely useless for multi player gaming” is just plain wrong. A lot of online, real-time games are using TCP. Saying that TCP is useless for multi player gaming is really misleading and I’m tired of reading this. There are a lot of valid reason to choose TCP over UDP (for example the fact that your game doesn’t require any round-trip to be shorter than, say, 500ms, because that’s how the engine works). Of course, TCP would not really be suitable for an FPS like quake, but that’s only one case of real-time game, there are a lot of different kind of games, for which TCP is the best choice.
> I think “real-time” does not mean what you think it means.
There are various definitions of "real-time" and we should not make this an argument about semantics. I meant real-time in the sense of "a soft real time software system" or the "Quake" sense of the word.
In other words, "real-time" in the sense that missing deadlines will mean that the quality of service is degraded. Deadlines here mean delivering the packets in time for the next simulation timestep.
Yes, TCP can be successfully used in games classified as "real time" as opposed to turn based, such as RTS games or MMORPGs. But when analyzing it them from a software standpoint, they might be just very fast turn based games.
The solution they used was essentially a 15 frames per second lock step loop. It looks like real time to the player but from a software standpoint it really isn't.
You and I agree on everything except the definition of "real-time". There are applications where TCP works well and applications where UDP is better. I'm sorry if my words were a bit overgeneralized and it annoys you.
> In other words, "real-time" in the sense that missing deadlines will mean that the quality of service is degraded. Deadlines here mean delivering the packets in time for the next simulation timestep.
Yeah, that’s what happens in most RTS, if the feedback for the action you sent doesn’t come back before Xms, then the game pauses and waits (for example Starcraft 2 does that).
And I know the lock step mechanism used in some RTS, I’m implementing one myself. But I don’t see how it makes it non-real-time. If quake or teeworlds associates a network packet (containing a player-position) with a frame, does that make it really-fast turn-based game (with turns being frames, or ticks, or what you want to call them)?
That being said,
> You and I agree on everything except the definition of "real-time".
No, the “real-time” word is really not that important to me, I mostly disagree with you saying “TCP is […] useless for multi player gaming.”
RTS, or MMORPG, or turn-based tactical RPG, or card games are not “some exceptions”. They probably even represent a huge proportion of the multiplayer games out there. I now understand that you were talking about “fast-paced action games like quake or the likes”, but that really didn’t sound like it.
> I'm sorry if my words were a bit overgeneralized and it annoys you.
This is not just you, I’ve seen a lot of these “— TCP sucks for all online game purposes!
— but what about all these cases were it’s fine?
— Oh, I was obviously talking about those other cases”
But that’s ok, I’ve made my point public instead of grumbling in my head, now I’m fine and we can move on. :)
> > In other words, "real-time" in the sense that missing deadlines will mean that the quality of service is degraded. Deadlines here mean delivering the packets in time for the next simulation timestep.
> Yeah, that’s what happens in most RTS, if the feedback for the action you sent doesn’t come back before Xms, then the game pauses and waits (for example Starcraft 2 does that).
To elaborate on this further, I would not consider Starcraft 2 a real time system, because it can stop the world and wait. It is annoying when it starts lagging but produces the "correct" results anyway.
This is not the case for simulators that deal with "continuous" physics updates. All clients are simulating the world in parallel and no one has the authority to stop everything. If lag gets significant, the results will become more and more incorrect and the view of the world that the clients have will diverge more. There's always some inaccuracy but that is tolerable to some degree.
This is by no means the only or necessarily the correct definition of real time but that's where I draw the distinction when it comes to multiplayer games. Someone might not agree with me on this one.
For some definition of real-time, a racing simulator is real-time because it will produce incorrect results (with catastrophic consequences) if there is too much latency. An RTS like Age of Empires is not because it will only lag annoyingly but still produce correct results.
The article even contains an explanation why TCP does not, and NEVER should, include forward error correction :
> Our simple re-sending mechanism just didn’t perform well enough under these conditions. It was quite common for a re-sent packet to be dropped, and we saw several cases where the original packet and 4 or 5 re-sends of that packet would all be dropped. We were re-sending so many packets, we were starting to exceed the bandwidth of the modem, and then the latency would start to climb, and all hell would break loose.
(then they implemented a -very bad- form of forward error correction in their packets to almost never need retransmissions)
This effectively ignores available bandwidth, and it worked because they were only using tiny amounts of bandwidth, so cheating helped them in this case, without ill effects. It is a dangerous game however. The problem is one of unfairness. If you implement your connections with forward error correction you will have an advantage over others, as you will not follow congestion signals while others do.
I am not suggesting Forward Error correction is a good idea on TCP. They put UDP and forward error correction together which is a good idea (see video/audio streaming over internet). The parent seemed to think they had reimplemented TCP badly, which they didn't, they implemented quite a different protocol which utilised forward error correction which was a good idea and something they would not have been able to do by building on vanilla TCP.
So their prototcol probably ended up messier than it should of been but they were pioneers fixing problems as they cropped up so that's understandable.
I loved the series, I remember in X-Wing there was a complete campaign where you start with hit'n'run tactics against some containers (!) and end in an ambush and the first kill of an ISD. This storyline made you feel hopelessly outnumbered and you just stood (flew) there in awe whenever something bigger than a Nebulon B arrived at the battle scene. Your AI wingmen just panicked and ran and so did you, possibly directing all your shield and weapon energy to your engines if you were unfortunate enough to fly a Y-Wing.
Later on you could count on corvettes and cruisers that reinforced your side (though they were always late) and you
felt the tide of the war turning.
I never got that feeling again, maybe because I am older. Or maybe because DX 11.1b support is much more important for a game and story is considered overrated.
I loved this series too. I even remember reading this very article back in the day when I was learning about game development.
Sometimes when I go to my old room at my parent's house and I see the game boxes in the shelf, I am tempted to install at least one of them and try to get back that feeling you talk about. I never have done it because I know deep down it will never be the same
There is a whole generation of people who haven't played XVT and that is a real shame. XVT has aged well over the course of time. I once dreamed of the day LucasArts was going to release XVT for the 21st century. Once they hosed that studio I gave up on the dream.
I'm having high hopes for both Star Citizen and the indie game Rogue System - the later one is very simulation heavy and could bring back the warm feeling of getting a damaged craft under control, only on steroids this time. None will have the SW licence, but hey - can't have everything, right?
Btw. how come nobody really cashed in on the Battlestar Galactica hype?
I worry that SC is way too ambitious. I never really thought Wing Commander ever delivered on it's hype, and I think SC is more of the same. I look forward to being proven wrong though.
BSG online has a bit of a following, but being browser based I gave it a miss
As someone who's always hated the Star Wars movies (except Empire, natch), I've nevertheless lost a lot of my life and worn out at least two joysticks to Tie Fighter. Fantastic piece of work.
I have yet to find a joystick that is half as nice as the (Microsoft!) one I had back then. Now it seems like either the springs are too hard or force feedback makes the dead zone all sloppy. I feel like the Goldilocks of joysticks.
I'm told the new Galactic Starfighter expansion for Star Wars: The Old Republic has a lot of the things people liked about XVT in it. It's free-to-play now, might be worth checking out: http://www.swtor.com/galactic-starfighter
Disclaimer: used to work on that project, still have friends who do.
I was very hoping for this. But the (few) maps are horrible for a "space" sim. The MMO fighter/item-part killed the rest of my possible motivation.
I really hope Disney will come up with something real in the future. I've been building Maps vor TIE-Fighter, X-Wing and XvT back in their days for The Emperors Hammer (that still exists out there!) so I hope they won't forget about the creativity in the community.
Well, except it replaced any level of skill you needed in XvT with clicky-the-thingy-oh-looky-it-explodes. XvT was the Quake of spacesims, garnered with some real sim elements like tuning your energy consumers. No game since has delivered on this again - for some reason all publishers thought there is no market for this - looking at Robert's kickstarter I think this has clearly been proven wrong now.
XVT had a two fold problem, half was the poor internet implementation but the other half was that the product didn't fulfill the genre's expectations.
Its predecessors (Tie Fight and X-Wing before it) had large, complex and well written story lines which following engaging missions paths. This was 80% of the game, as the graphics were relatively poor compared to sprite-based shooters of the time.
I remember the deep disappointment of booting XVT for the first time and realizing it had no story, and a terrible UX. I was launched into an offline lobby and only had the option to play some very uninteresting and un-engaging missions.
Put that on top of a completely unplayable multiplayer mode, it ended being a poor decision to make a game based on a technology they couldn't support -- when a perfectly valid offline story mode would have made up for it fine. So instead they charged another 49.99 on top of 69.99 for an add-on (and people complain about IAP) that gave the features the original box promised.
If you like the space shooter genre and you like them to have an interesting story line, then also take a look at Freespace and Freespace 2.
For me, those two are the pinnacle of the space-shooter genre, even more so as Freespace 2 was open sourced and attracted a very large community of not just programmers but also artists which resulted in a game that's still being active developped with quite current graphics and sound.
You can get the game on gog.com and use the Java-based[1] installer to get the open source engine and assets installed
The multiplayer is so great that we still gather together every year and fly missions for a night. Balance of Power added great coop missions that needed team-play and sticking to your role.
The most interesting thing - historically - about XvT was that it was one of the first games that completely skipped single-player campaigns for multiplayer. Everyone screamed "madness" back then. See where it got us ;).
This is timely as I have been replaying X-Wing lately (specifically, the Collector's Series version that uses the XvT engine, aka xwing95). X-Wing was one of the first PC games I played when it first came out (I was in middle school) and thus I have an endearing love of space sims. (I have high hopes for Star Citizen and Enemy Starfighter.)
I think an interesting project would be creating a modern graphics engine that can parse and run the various X-Wing game data files. The XvT and X-Wing Alliance engines run ok on modern Windows but still have some compatibility issues (graphics glitches, etc). If a modern graphics engine (written in Unity or some such) could capture the look and feel of the X-Wing games, that would be great. It would be even better if it was an official game and not someone's open source project that never gets finished :(
LOL! I wrote that for a presentation at the Game Developers Conference way back when. It was really a post-mortem, where the point is to show others what mistakes we made so that they wont repeat them. Anyway, its kind of flattering to see it come to life again... like some kind of Frankenstein internet cat. Thanks to all the positive commenters, and all you haters can fall into a timewarp and have to suffer through 28K modem hell all over again. ;) Or worse yet, be forced to watch Titanic...
It took the video game industry nearly a decade for most games to come out with network code as effective as X-Wing vs. TIE Fighter's and QuakeWorld's at hiding latency, packet loss, and bandwidth limits. Even recently, many fighting and RTS's games struggle on poor connections.
Until I played Killer Instinct, the only decent fighting games that I've run across have been based on the GGPO. Judging from my own and others experience, KI's network code is this generation's equivalent of QuakeWorld. I hope the developers of KI publish a paper like this one, detailing their technique.
If you find this interesting, you may want to take a look at a series of articles and a live demo I wrote on the topic of client-server networking for games (http://gabrielgambetta.com/fpm1.html) which is very often cited on gamedev.stackexchange and similar places. It's more about First-Person Shooters, but most of it applies anyway.
X-Wing, Tie Fighter (and its mission packs) and X-Wing Alliance were awesome. XvT not so much :(
> On some connections one in five packets are dropped
And then shortly afterwards that
> This would only fail if two consecutive packets were dropped, and this seemed unlikely.
Hrm. I think I'm beginning to see the problem here, and it isn't the internet. Still, fun to read about the battle scars of remote gaming's early days.
You've destroyed an important bit of information in your paraphrasing. The literal quote:
> It turned out that on some connections, about every fifth packet we sent would just disappear into the Ethernet.
Due to the way packet-queueing works, it is actually a reasonable failure-expectation. If the network is on the edge of congestion, a packet will be dropped as there's no space in the queue, but another packet will be sent and clear space in the queue before a second packet has a chance to be lost. In many near-congestion failure modes, it is literally every fifth or tenth or twentieth packet that gets dropped. This is actually the basis of how TCP's congestion-control works, too.
These guys were at the forefront of online game technology back in '97. And not only that but they were also willing to share what they learned. Mistakes, warts, and all.
There's something magical about the internet where any technical post results in a slew of commenters trying to show they are smarter than the topic author. Good job.
I think their requirements were to transmit every action the other player(s) took, so it seems like things would still be failing all over the place. But maybe occasionally insane results are better than occasionally blocked.
I realize this is about game development, where missions and what-not are a major chunk of development. But still, I think most of the problems here are basic engineering failures: if you have a big infrastructure issue, get that issue right before you go crazy on other stuff. From the article:
"We actually used the 'synchronous' version for quite awhile. It was good enough to test with, so finishing the network code was considered a lower priority than the other issues we needed to address at that stage of development."
And the result was, rather predictably:
"When we finally came back to the network code we were behind schedule, and that affected some of the decisions we make later in the process. And it meant we were absolutely committed to the complexity of the missions, and the user interface."
I got to the third page before I realized how clueless the person who wrote that is.
First of all, 1999 was 15 years ago but since I started an ISP from the ground up in 1996, I know for a fact that the Internet wasn't quite the dark ages at that time he tries to make it seem. 28.8k modems were around in '94 and by '96 to '97, 56k were standard. And maybe I was just in a lucky area, but I had a Comcast cable modem in 1996 (http://corporate.comcast.com/news-information/timeline).
Getting beyond that, he complains about everything. TCP sucks, UDP sucks, the Internet sucks. A clue for anyone with introspection is when you think everything but you sucks, it's actually you that sucks.
For an example of much more highly skilled engineers working on an arguably even bigger engineering problem in 1996 on a much larger game, read here:
One correction, the 1999 is when the article was written, the game was created mostly in 1996 and released early 1997. Your point still stands of course.
Currently working on an iOS project using a similar system, which so far works great. A challenge I am facing right now is pathfinding in conjunction with synchronous multiplayer. Pathfinding can take a decent chunk of cpu time on lower-end devices, but you can't really multi-thread it since you need all players to calculate the same paths on the same simulation ticks lol... so might come down to rate-limiting the amount of units that can re-path on a given sim-tick.
it makes me wonder if there's a way of encoding some kind of extremely compressed lossy version of the entire game state that can patch each player's state to be incrementally more "correct". Then on slow connections you're still keeping positions broadly where they should be, while faster connections will bring the game state to a more precisely correct state. How would you create such a packet without knowledge of each player's differences?
Just a thought experiment I was having. What if the game state was represented as a very large N dimensional vector. The most significant bits are sent until the players report that they match. then the next significant bits. then the next after that. Adapt the granularity of an update to the current throughput. If a player disconnects for 10, 20 seconds their game state re-syncs smoothly and gradually instead of in a jump cut?
It was quite common for a re-sent packet to be dropped, and we saw several cases where the original packet and 4 or 5 re-sends of that packet would all be dropped. We were re-sending so many packets, we were starting to exceed the bandwidth of the modem, and then the latency would start to climb, and all hell would break loose.
That is exactly why TCP has the "slow start" protocol. I am sympathetic to their problems, though. I understand that the implementations of TCP they had access to had way too much latency, and that back then, there probably wasn't much tuning one could do on the outside. What they did is a clever hack (message n has a copy of n-1) that takes advantage of the fact that they don't have to send much data in total.
> > Lesson two: TCP is evil.
> Does this mean WebSockets are unusable for most type of multiplayer games?
Yes, it means that WebSockets are unusable for real-time multiplayer games like fast paced FPS games. That may not be "most" multiplayer games, though.
If you're planning a game on top of WebSockets (or TCP), the gameplay should be tolerant of 3-5 times the network round trip time.
As well written as this article is, I do have a problem with its tone. The writer seems willing ignorant of why the Internet “sucks” for gaming; it was not designed for real time small packet transmission. Most of TCP’s design, in particular, was intended to facilitate the transmission of comparatively sizable files over FTP, NNTP, and SMTP. Even stuff like IRC was designed not to be so concerned about timeliness that the chatlog on each end would match up exactly.
The internet wasn't designed for high-latency, high-compression modem sessions. TCP works just fine over a 56 kbps leased line. It's terrifying on a 56k home modem, despite similar "bandwidth."
The leased line has 50 ms of very predictable latency and 56 kbps of very predictable throughput. The 56k PPP session has an unpredictable 300-500 ms of latency and a constantly varying capacity of 2.4 to 56 kbps thanks to high noise and ugly compression schemes.
Lots of things that worked perfectly well on the relatively crude internet of the 1980s were real pains in the ass on "high-tech" modems in the 1990s.
--
edit: a specific example would be X11. An emacs X11 session was totally usable on a leased line or ISDN, despite the low bandwidth, because they delivered low latency and packet loss. It was hellish on any kind of a home dial-up.
During the late '90's, I was a sysadmin for a fairly major computer science department. One of my periodic tasks, and my test of Internet health, was to download and build gcc. When I started, getting gcc took a while. By 1999 and later, when the Internet had been discovered, downloading gcc took several hours and frequently several retries.
I remember making a PC game in 1998 with multiplayer.. we had to handle 500 ms lag in an action game! Later I worked on a game that kept three versions of the simulated in memory (the old confirmed world, the new confirmed worlds, and an interpolation) to deal with lag while keeping in sync.
Their implemented solutions at the end remind me of Halo 2, where players would slide and warp around if they were too laggy and where a bad enough connection would boot you out into a blue screen until it recovered (or kicked out the player with the worst connection).
> When we started the X-Wing vs. TIE Fighter project, our goal was to create the first multi-player space combat simulator to be playable over the Internet.
They obviously hadn't heard of Netrek, the third multiplayer internet game ever, first released in 1988. http://en.wikipedia.org/wiki/Netrek
Ah, memories. My inspiration to study CS, though probably fortunate that it was mostly dead by sophomore year, or my college GPA would have been much worse.
Netrek's macro system was another feature I wish were more widely adopted in games. Receiver-configurable distresses (RCDs) allowed a sender to publish named kinds of messages ("help at location, intercepting X, ship status, bombing planet, etc.) but receivers control the formatting. So I can set up my client to only show your hull+fuel for general distress calls but more data based on the kind of message.
Actually, I _loved_ netrek... it was an inspiration! It was more RTS than flight sim tho. But it was quite possibly the first MMO as well! I never played it over a modem, tho, just on a LAN... which is definitely not the same thing as a 28K internet connection circa 1999. But that game was genius, and quite possibly _would_ have worked over slow, lossy and highly latent connections, I never tried it.
Lesson Two: TCP is evil, don't use TCP for a game...
Lesson Three: Use UDP...
and the like are all "of course" solutions now, but they weren't then. How do you build something fast and reliable over something slow and creaky? Who knows.
The shoulders of giants have gotten a lot taller since then.