In the third semester at university in Germany there was a course "Introduction to technical computer science", where one would learn about things like Karnaugh Diagrams, compression algorithms, D-Latches and so on and built a 16bit optionally pipelined RISC CPU in lab assignments. The professor teaching the course had developed custom signal processors for the LHC, so that was pretty cool aswell.
They did for us; right in the first two semesters we had 4–6 hours a week just low-level stuff. They also introduced us to algorithms and data structures with Pascal, not Java (which I'm very thankful for).
It was quite enlightening to see over the course of three weeks or so how a (simple) CPU could be built from ever-larger building blocks, but starting at the very beginning with transistors.
It's same for me, too. How long did it take for you to decide that way? I do not think about giving up until I get my master degree and plan my future again.