Thank you for your interest guys! We went from about 20 stars to 200+ stars, landing us on the 'trending' section of github! We could vastly improve the program by hooking into the game's memory (Such as the totally sick mapping done in https://www.youtube.com/watch?v=3YYyGGgoWfs). But this project really just explores computer vision and ML (Kinda, tensorflow is still being worked on).
It's amazing to see a game with the roads of half continent. I've been playing driving games for so many years that I become fed up with all the grinding (licenses, career path, races blocked by other races, etc) so I'm not buying those games any more. I want all the cars, all the roads, no traffic, no police, only driving. I could pay an extra for that. I wish there is something like Euro Truck with only roads, some fast car and curated lists of interesting road to drive on.
And kudos to the authors of ChosunTruck, obviously.
> I want all the cars, all the roads, no traffic, no police, only driving.
Then you don't want to play ETS2.
> It's amazing to see a game with the roads of half continent
As someone else noted, the road network is only an approximation of the highways. Each city is essentially the same layout with slightly different buildings to make it look like the namesake city (outside the drivable world, basically a 3D skybox).
The AI vehicles are quite stupid. I've had them suddenly brake in front of me, or merge into the trailer from behind. Oh, and you get penalised for this (ticket & damage repair costs).
Apart from the PS of trucks, there's virtually no difference between them. Want a basic Volvo? It'll haul trailers. Want the top of the line Volvo Globetrotter with sleeping cabin? It'll haul trailers. Oh, and you still need to stop at the designated rest areas to sleep, even though you have a bloody sleeping compartment in the Globetrotter!
You have to follow all the laws, all the time. Speeding? That's a ticket. Running a red light? Ticket. Bumping another car? Ticket!
I wish they would include an arcade or "god mode" so you could actually have fun.
/rant
Can anyone comment on how this is actually obtaining the video output from ETS2?
Is there some way you can feed the output of an OpenGL application into this? Or is it just capturing the desktop area where ETS2 is running?
> Can anyone comment on how this is actually obtaining the video output from ETS2?
In Windows, it's done by grabbing the handle of the window with the "Prism3D" classname.
This then goes to the hwnd2mat function[1], which does `GetDC`[2] to get the device content and then creates a bitmap with `CreateCompatibleBitmap`[3].
---
On Linux[4], it's done by grabbing the screen with `XOpenDisplay` and the current window with `DefaultRootWindow`. It then uses `XGetImage` and memcpys the result into the vector of pixels.
Note that the `main` function in the Windows and Linux version is slightly different to accommodate the difference in how these images are grabbed.
> You have to follow all the laws, all the time. Speeding? That's a ticket. Running a red light? Ticket. Bumping another car? Ticket!
> I wish they would include an arcade or "god mode" so you could actually have fun.
They do. Look in settings, you can disable vehicle offenses (so no fines for running red light, crashing, speeding) and you can disable traffic too (might need a mod, i dont recall)
When it comes to the city layouts, the DLCs for Scandinavia and France have better and different layouts than the main game.
What I'm missing is road construction. You can't go 100 km pretty much anywhere without encountering something out of the ordinary - narrower lanes, blocked lanes, lanes running on the opposite side. Also the ramps look more or less the same no exit within exit or those weird X-shaped exits - where an off-ramp is an on-ramp at the same time.
Have you looked in the options? you can disable most of those things your complaining about in the vanilla game.
you can even disable the limiter if you're feeling brave!
> The AI vehicles are quite stupid. I've had them suddenly brake in front of me, or merge into the trailer from behind.
I'm sure the issue here is that it is "more common than the real world", but you could replace "AI vehicles" with "other drivers" and that would be a valid real-world complaint.
I'm surprised that - given the existence of OpenStreetMap - nobody's tried to make a global driving simulator. Real roads, real scale. Buildings are a potential issue (OSM includes the outlines, but I don't know about full 3D models), but it otherwise seems at least plausible (even if highly unlikely) to pull off reasonably well.
My impression was that it can capture road elevation, but most OSM contributors don't. I know it supports capturing the elevation for specific points (as well as the heights of things like buildings), and I'm pretty sure OSM includes terrain elevation as well.
Well, it'd make a for a good starting point for a developer - they could then mangle, simplify and merge Ordnance Survey (UK) or national equivalent elevation data..
..and then, like with the bots that supposedly roamed No Man's Sky's development, ensuring no random variables came out with anything rude or inappopriate, make bots like that here to check the merged output isn't quirking too much :)
There certainly used to be some effort to achieve this in Outerra's Anteworld- a full earth simulation. I don't know if it ever got anywhere. There was a Google Maps overlay you could use to painstakingly add roads in their real locations. I never attempted anything elaborate, but I certainly enjoyed placing my own (non realistic) racing loops around random Scottish Lochs or Tibetan mountains and driving on them at my leisure: http://www.outerra.com
I've played on Europe Server 1 and 3 and on both I've seen normal cars available (although from what I've seen ES1 kicks people back into singleplayer, if they attempt to use a car in multiplayer); ES3 supposedly allows to drive cars in multiplayer as well.
Interesting choice of a name. "Chosun" is not widely used in contemporary South Korea unless you're in a self-deprecating mood ("Hell Chosun") or referring to the Kim dynasty up north. You guys planning to drive to Pyongyang one day in an autonomous truck? :)
Anyway, cool project! It's really nice to see what used to be very expensive proprietary technologies become available for anyone with an Ubuntu box to play and experiment with.
> "Chosun" is not widely used in contemporary South Korea unless you're in a self-deprecating mood ("Hell Chosun") or referring to the Kim dynasty up north.
I rather see it as a tongue-in-cheek name, it doesn't necessarily convey a self-deprecating mood (though it somehow implies the comparison itself).
My thought was, driver must be Australian - as fast lane hogs and a complete, fundamental lack of lane discipline is unfortunately the order of the day here.
Still a pretty cool idea to see Euro Truck Sim being used for AI training. I'm impressed.
Yep, it was on integrating GTA V into Universe[0].
Interestingly, that post links to a specific blog post on GTA V integration[1], but going to that link redirects to the initial "Introducing OpenAI" post[2]. I can't find the GTA V blog post[3] on their blog anymore.
Edit:
In addition, the GitHub repository[4] for spinning up a GTA V server instance from the post[3] no longer exists.
I appreciate this is a big ask but I find this incredibly interesting and would love to know how it works, would someone mind walking me through some of the main functions please? Which files on GitHub are actually doing the work to keep the truck within the correct lines? Does this learn as it goes or have they somehow trained it before hand?
1. Grab the game window with `FindWindow("prism3d", NULL)`.
2. Get an image of said game window with `hwnd2mat` (found in windows/src/hwnd2mat.cpp). This part is complicated, as it deals with the Windows API.
2. Apply Median Blur to the image, as shown with this transformation[2].
3. After creating the 4-point correspondence between the input image (`image`) and the output overlaid image, (`outputImg`), create an IPM object that will perform the Inverse Perspective Mapping. This is a function that turns a flat-angle shot into an overhead shot [3].
4. Resize the `outputImg` to 320x240 for the smaller window.
5. Convert `outputImg` to grayscale, storing the result in `gray`[4].
6. Perform the `cv::blur`[5] operation on `gray`, storing the result in `blur`. This reduces noise in the image; there are artifacts gotten from the Inverse Perspective Mapping.
7. Perform the `cv::Sobel`[6] operation on `gray`, storing the result in `sobel`. This gets all of the outlines, including (surprise!) the lines on the road.
8. Perform the `cv::threshold`[7] operation on `sobel`, storing the result in `contours`.
Result at this point: A flat perspective of the road has been turned into an overhead perspective, converted to grayscale, blurred, turned into an outline, and then thresholded to solely have the lines on the road. Note that there's an option to perform Canny[8] as well, which is another alternative.
9. Create a `LineFinder` object `ld` (found in /windows/src/linefinder.cpp).
10. Perform `ld.findLines` on `contours`. This performs the `cv::HoughLinesP` operation[9], returning a vector of lines (`vec4i` is a vector of 4 integers that stores x_1, y_1, x_2, y_2). It'll use these to draw the detected lines for informational purposes.
---
Okay, now we have a black-and-white image of road-line contours, stored in `contours`. Unfortunately, keyboard and mouse input doesn't work in Windows, so we have to go over to linux/src/main2.cc. Note that the line-finding part is the same thing but requires some Linux-specific stuff to get the original image of the game.
Starting from Line 198:
11. Define the center of the image to be `bottom_center` at column 160.
Our x-value for the pixel values is going to be `j`. Our y-value is going to be `i`.
12. Create a for loop that iterates through all values from y = 240 to y = 30. Inside this is effectively two loops - one that acts on the x-values from 160 to 10, and one that acts on the x-values from 160 to 310. This loop looks at the pixel in the (j, i) spot in `contours` and sees if it detects a line there. If it does, and it hasn't found a line yet, it sets the `center_to_right` and `center_to_left` values, respectively. This finds the x-values of the left and right lines. I think you know where we're going from here...
13. Create a `centerline`, which is going to be the average of the two.
14. For informational purposes, gather the average `centerline` values and the average `center_to_left` and `center_to_right` values.
15. Move the mouse according to how far away `centerline` is from the center of the image. I don't know how to move in ETS, so I'm going to take their word for it that their gnarly-looking trig function is on-point.
Right now I'm changing SendInput to use mouse simulation instead of pressing keys. Hopefully this will fix some of the lagging caused by pressing keys too fast / too much.
This sounds real similar to Project 4 of Term 1 in the Udacity Nanodegree (Advanced Lane Finding) - basically using OpenCV on a video to find the lane markings and highlight the road.
The only real difference is some of the order (in the project, you first have to fix the camera distortion, then do the image thresholding, then warping the image from "perspective view" to "overhead view") and the lane finding (where you do a "windowing" method to find the lines after doing a histogram search to find the start of the actual lane-lines; after that, you essentially do a polynomial fit of the found pixels, which you then use to render the road shading, then re-warp that from flat back to perspective and overlay it on top of the road image).
Can you tell I just recently completed all these steps...
I'd just like to say that your code is extremely easy to read and follow, and this was a pleasure to do.
Thanks a ton for showing a real-world example of how to use OpenCV. The examples given by the official documentation are cute, but they're individual tools that you have to combine intelligently to make something useful.
*) http://www.newyorker.com/tech/elements/desert-bus-the-very-w...