I enjoyed Linear Algebra Done Wrong [0], to be used in combination with a more traditional textbook, like Linear Algebra and its Applications [1] (which has some good diagrams). I've already seen it mentioned, but I'd like to add that 3b1b's Essence of Linear Algebra [2] videos are well made and make for a good supplementary resource early on.
The title of Linear Algebra Done Wrong is an unacknowledged nod to Sheldon Axler's Linear Algebra Done Right. I know Sheldon; he believes it's a crime to teach people determinants. I teach people determinants. Wrong features determinants in Chapter 3.
I was once part of an interactive learning software demo, where Sheldon had provided the sample linear algebra problem. I solved it in seconds using determinants. That really made my day.
I don't think Sheldon Axler wants to literally banish determinants, they are a useful tool.
But they do obscure the meaning in a lot of proofs. If you start using them immediately, it is also hard to motivate where they come from, what's the intuition behind them and how the students could arrive at the definition themselves.
And even if you use them and they produce short one-line proofs, you should also prove all the properties behind them first, which is where all the complexity is hiding.
The nice point about Linear Algebra Done Right is that it explains how Linear Algebra works without resorting to "determinant magic". In the end, you will also understand why determinants work so well in so many proofs.
In any case, his book is really suited for a second course in Linear Algebra, so maybe it is beneficial to start using determinants right away. Personally I understood them better with his book.
Disclaimer: I don't know Sheldon Axler, but I read his book.
this is definitely one of the problems I have always had. There seems to be so much "lore" hidden in various axioms about these things that often some derived concept like determinants that seems to relate to nothing intuitive are taught to the exclusion of insight generating concepts. They seem to think it's a clever way to teach to do this and then later on reveal the magic behind the curtains, saying "see this is how determinants work!". But for me I was lost and gave up trying to understand 6 lectures ago.
Agreed which is why I love Artin’s Algebra. It introduces the matrix determinant early and uses it as the prototypical example of a group homomorphism and the book uses matrix groups liberally.
Obviously not a beginner friendly book if you just want to understand matrix equations s. But I found the pedagogical approach great and there are some real gems in the exercises.
I found it a better for learning versus the standard recommendations of Axler/Lang/Halmos for LA. (Lang’s exposition is kinda weird and Halmos’s choice of exercises isn’t my favorite).
To echo sibling comment but make it more prominent - "Linear Algebra Done Right" is a great second book, I wouldn't use it as a first source of learning Linear Algebra. (I'm pretty sure it says as much in the intro, IIRC.)
I personally learned from Strang, then this book, which complemented each other very well. Strang is very focused on practicality, actually computing things, always seeing the matrix behind everything. "Done Right" is great at ignoring most of the "matrix view" of things completely in favor of pure linear transformations instead. Both views have their use - so learning one, then the other, is a great idea.
The book is open access (https://linear.axler.net/), I am going to quote directly the author in the preface:
<< all linear algebra books use determinants to prove that every linear operator on a finite-dimensional complex vector space has an eigenvalue. Determinants are difficult, nonintuitive, and often defined without motivation. To prove the theorem about existence of eigenvalues on complex vector spaces, most books must define determinants, prove that a linear map is not invertible if and only if its determinant equals 0, and then define the characteristic polynomial. This tortuous (torturous?) path gives students little feeling for why eigenvalues exist.
In contrast, the simple determinant-free proofs presented here (for example, see 5.21) offer more insight. Once determinants have been banished to the end of the book, a new route opens to the main goal of linear algebra— understanding the structure of linear operators.>>
If you like mathematics, it is actually a pretty nice book.
The philosophy of LADR is described in his paper Down with determinants: https://www.axler.net/DwD.html. In short, he thinks they obscure proofs. I love the book, but AFAIK only the compactified version (excluding all proofs, examples, and exercises, along with most comments) is open access: https://linear.axler.net/LinearAbridged.pdf.
IMHO, since the OP wants to apply linear algebra to real world problems, a better approach is to go with a matrix analysis book. Strang is very popular, but my favorite is http://matrixanalysis.com/Contents.html. Axler is a few notches higher in terms of abstraction. Hence, you won't learn lots of important practical results about matrices. In case of going with Axler, I'd use the previous edition. It's a shame they have ruined the typesetting by adding so many distracting color boxes and different fonts.
Personally, I'd go with Hubbard & Hubbard: https://matrixeditions.com/5thUnifiedApproach.html. It's a work of art that takes you from pre-calculus till multivariate calculus and analysis, along with all necessary linear algebra. Great mix of rigor, intuitions and practical details. At this level, as Hubbard points out, it's very useful to combine linear algebra with calculus & real analysis.
Another vote here for the matrixanalysis.com book. It is a really excellent book and takes a reasonably pragmatic approach to linear algebra. Coming from a programming background, you're more likely to find some things that resonate with you here. For example, this is one of the few introductory linear algebra books that deals with sensitivity analysis, which is useful to think about when dealing with floating point arithmetic instead of real arithmetic.
Would Hubbard and Hubbard be too difficult if I found Spivaks calculus too difficult?
I have been struggling to find a linear algebra book that isn't too abstract or too verbose. I did take LA and calculus a decade ago and I am trying to build up a background strong enough for probability and statistics.
It depends. Skim through H&H to see. I find it more intuitive and modern, but it also covers way more territory. At some point, the material will be hard because it's very advanced mathematics.
However, by then perhaps you have already adjusted. There's also a solution manual. Furthermore, many difficult proofs are in the appendix. So it's more of a calculus book if you want to ignore the analysis part.
I am hoping to get some experience with probalistic modelling, maximum likelihood etc. I work in bioinformatics and spent most of my time on algorithms development but probability/stats/ML are becoming the norm now. I find it hard to follow papers and develop new methods.
The skills needed will vary a lot. Hence my concern about studying H&H. It's a good idea, as real analysis is the foundation. But it will take too much of your time to get to something useful. Probably you should try to learn more applied material in parallel and let both threads merge in the future.
For maximum likelihood, you need to learn convex optimization right after real analysis. The canonical reference is [1], but there's also a very simple and pragmatic linear algebra textbook by the same author that also covers some of the optimization basics [2]. This might be a good entry point, certainly easier than Spivak or H&H. There's also [3,4], which you probably know about. These are great and emphasize the modeling part. Maximum likelihood (via EM) is in the appendix, and you don't really need to know a lot of math to get going.
If you prefer a Bayesian or a variational point of view, modeling is the really important part. MCMC and message passing algorithms tend to be reused. For high level modeling of study results (e.g. differential expression on complicated designs), Gelman's Stan books [5] are a delight to learn from. If you need to roll your own custom inference, you should learn about graphical data structures such as factor graphs [6,7]. Here, knowhow from H&H is also required.
Thanks so much, it's nice to get suggestions from someone in the same area!
These are the most helpful and practical suggestions I have encountered. You've hit the nail on the head with the exact problem I have been having working through books like Spivak and Axler. It always felt like I wasn't learning anything practical towards my work and that anything useful was a long ways away. I do enjoy the books and the material and the suggestion to pursue them in parallel is something I wish I thought about.
Determinants are usually introduced in Linear algebra out of the blue because you can't get to other important topics in Linear Algebra without them. Calculating them is a complex mess best left for a calculator. Sheldon teaches Linear Algebra as a theoretical math course, along the lines of learning Abstract Algebra. He approaches those other important topics from a different direction entirely, and determinants are just a trivial part of his book because of the different approach.
An eigenvector of a linear operator is a nonzero vectors s.t. the operator multiplies it by a scalar. It's existence for a complex finite dimensional operator can be proven without the determinant.
The minimal and the characteristic polynomials can be defined and described without the determinant.
Well at some point you may want to actually calculate the eigenvalues. Unless you just dive in and try and compute the characteristic polynomial directly.
That is really a wonderful book which I perused when learning linear algebra, maybe a bit on the mathy side for OP expecially as he is asking a book for a "visual learner".
Fortunately linear algebra can be grasped intuitively in dimensions above 3 even if it can't be visualized, but maybe I'm biased as it is bread and butter for me now.
I have a text at https://hefferon.net/linearalgebra/index.html. It is aimed at beginners. It comes with perhaps two dozen exercises per lecture along with complete worked answers to every question, with videos of the lectures, a lab manual using Sage, and some other ancillaries.
Like others here I recommend 3B1B, which may be what you are looking for visually, but whatever you end up with it is absolutely crucial that you do exercises. Do many of them. It is the only way to get better.
The point of the hook statement "You are not a visual learner" in the Veritasium video is not to "disprove the myth that you're a visual learner."
The point is that there's little evidence behind different people having different learning styles, and that in general everyone is every "style".
This implies that vision, in addition to many other sensory modalities, is useful. As you point out, the utility of of 3b1b is in line with this point.
Just chiming in to say that you can dive directly into Strang's Youtube lecture series, without a book or anything else; like, an immediate next step you could take if you wanted to is just to pull up his first lecture right now and watch it. (I mostly watched him at 2.5x speed).
Although it’s not a book, a good series on YouTube is 3Blue1Brown Essence of Linear Algebra. That explains it in a very visual way. That, in addition to Linear Algebra and its applications by Gilbert Strang, would be a strong mix. I would also recommend 3000 solved problems in Linear Algebra by Seymour Lipschutz as a strong foundation in linear algebra requires practice.
It should be noted that the sum of the 3B1B videos is like 2 hours, and that Grant himself says that these videos are for summarizing and providing intuition after you have already taken the course.
Essence of linear algebra is an absolutely wonderful series. It gave me an intuition of the subject in a matter of hours in way years of university didn’t do.
Personally I liked the No Bullshit Guide to Linear Algebra. It kind of builds up things slowly and in a conversational manner, but you can also skip thru pretty quickly if you just need a reference.
I don't think I've been able to find any particularly good visual LinAlg books - most of what you're trying to achieve is actually quite abstract and I found the classic books a little confusing.
As an addendum - if you live stateside, classes at community colleges may be quite inexpensive and fairly approachable.
I highly recommend getting a print copy because it will help you concentrate on the material and read deeply without distractions. I also have a free-PDF-if-with-purchase-of-print-copy policy, so if you send get the print version and send me an email I'll hook you up with the PDF for free (PDF good for reference and for searching).
Happy to take questions about anything related to the book. Post here or send email. Contacts in profile.
It's worth calling out that the video itself doesn't support some of the blunt arguments being made here. The point of the video is that it's likely that everyone does better with a multimodal approach. It thus remains reasonable to seek out books that do a good job with visual representations! No visual components, or, worse, bad diagrams are, according to the video, an impediment to everyone, not just people who have used a disfavored term in their Ask HN question. :)
I like 3Blue1Brown as much as everyone else, it's an achievement and kind of a joy to watch, but my experience was that, after many go-rounds over the years, the thing that made any of this actually stick was doing exercises. I tend to bang the less abstract ones out in Sage: https://www.sagemath.org (you have to do a little bit of extra work to make sure Sage isn't doing too much of the work for you.
I'm a fan of Strang's approach. But I'm bad at linear algebra, so, grain of salt.
I wouldn't reccomend Strang's "Introduction to Linear Algebra" textbook to a beginner. Strang has a very odd, dense way of writing, often with references to material that has yet to be introduced. I think this is a consequence of it's intended use as an aid to his lectures, and can't really stand on its own. The goodreads reviews on the textbook seems to share my opinion: https://www.goodreads.com/book/show/179700.Introduction_to_L...
I think it's great for an intermediate student, or someone who's also watching Strang's lectures.
I agree, but would recommend his video series to beginners. The books themselves are less important than the exercises in the book, which unfortunately sort of demand that you have the book because they refer back to them. But the package of (exercises, video lectures, book), in descending order of importance maybe, I think is a worthy recommendation. Ultimately, the book is the only part of that you actually have to "acquire", so it might be ok that it doesn't stand on its own.
This is highly important. Linear algebra is applicable to so many fields, but learning linear algebra for say... Graphics Programmers, is a completely different feel from learning linear algebra for an Electrical Engineer Signals-and-systems engineer.
Graphics programmers largely need to learn "how to use" matricies. Emphasis on associative properties. Emphasis on non-communitive operations.
In contrast, Electrical Engineers / Signals-and-systems want to learn linear-algebra as a stepping stone to differential calculus. In this case, you're going to be focusing more on eigen-values, spring-mass systems / resonant frequencies, applicability to calculus and other tidbits (how linear algebra relates to the Fourier Transform).
----------
The graphics programmer (probably) doesn't need to learn eigenvalues. So any textbook written as "linear algebra for graphics programmers" can safely skip over that.
The electrical engineer however needs all of this other stuff as "part" of the linear algebra class.
I'm sure other fields (statistics, error-correction codes/galois fields, abstract algebra, etc. etc.) have "their own ways" of teaching linear algebra that is most applicable to them.
Yes, "linear algebra" is broadly applicable. But instead of trying to "learn all of it", you should instead focus on the "bits of linear algebra that is most applicable to the problems you face". That shrinks down the field, increases the "pragmatism" of your studies.
Later, when you're more familiar with "some bits" of linear algebra, you can then take the next step of generalizing off of your "seed knowledge".
--------
I personally never was able to learn linear algebra from a linear algebra book.
Instead, I relearned linear algebra 4 or 5 times as the "basis" of other maths I've learned. I learned it for differential calculus. I relearned linear algebra for signals. I relearned linear algebra for Galois fields/CRC-codes/Reed Solomon. I relearned linear algebra for graphics.
Yes, it seems inefficient, but I think my "focus" isn't strong enough to just study it in the abstract. I needed to see the "applicable" practice to encourage myself to learn. Besides, each time you "relearn" linear algebra, its a lot faster than the last time.
Thank you, this is a great point! I am in the category of someone who needs linear algebra in order to apply it for day-to-day stuff, hands on not blue sky. Currently my primary use case is image filtering but a bit down the line signal processing will come up.
> Currently my primary use case is image filtering but a bit down the line signal processing will come up.
Image filtering _is_ signal processing, two-dimensional signal processing to be precise.
Traditionally, a college would take you through linear algebra -> differential equations -> signals and systems, to approach this subject.
I found it easier to go through the reverse: start at signals-and-systems (to see what you have to learn), then work your way back down to linear algebra, and then work your way back up to signals and systems.
---------------
From a "signals and systems" point of view, your image filtering functions are 99% going to just be a "kernel" applied to an image.
Where blue-box is the original signal, and red-box is the convolution-kernel, and the black-line is the output of blue convolve with red.
From there, you generalize the 1-dimensional convolution, into a 2-dimensional convolution. To do so, you need to study linear algebra and matricies. But now that you're "focused" upon the convolution idea, as well as the idea of a "kernel", everything should be "more obvious" to you as you go through your studies.
You can see that a "Matrix", in your specific field of study, represents a kernel to a discrete system. The image you want to manipulate is a 2-dimensional signal. A "matrix" is many different things to many different mathematicians / engineers. "Focusing" upon your particular application is key to learning as quickly as possible. (You can generalize later after you've mastered your particular field).
Still, the study of signals / systems is a very generalized and large field. Mechanical engineers study this, because it turns out that an "impulse" that is "convoluted" with a "kernel" is descriptive of how a speed-bump affects your car's suspension system (!!!!). (EDIT: A youtube video demonstrating the same math for earthquakes vs buildings: https://www.youtube.com/watch?v=f1U4SAgy60c)
So studying signals-and-systems is still a very abstract goal of yours. It sounds like you need to focus upon the image-processing portions of signals-and-systems.
---------
IMO, you'll find that there's probably very little linear algebra you actually need to learn for your particular path.
I largely agree with your comments in this thread -- I'd been thinking about trying to express the same thing myself. I'd been guessing the motivation would be ML, where I feel that most people substantially overestimate how much they'd need.
Signals processing, though, is one of the places where I actually think a decent understanding of some of the higher-level concepts in linear algebra is really helpful. Linearity itself comes to mind, and maybe it just reflects my physicist's education, but it's hard for me to imagine having a working understanding of Fourier transforms without getting the idea of changing bases. I feel like you're about halfway through a first linear algebra course before you'd get there.
EDIT: that said, a good signals processing book probably covers a lot of this in sufficient depth if you can figure it out. The other catch-all comment I'd make is that linear algebra from a math class can look somewhat different from practical linear algebra on a computer. (That it's often a bad idea to directly invert a matrix for many computing applications is non-obvious from math class.) A book like Trefethen and Bau is great on that latter subject but is not a good starting point for OP.
> I personally never was able to learn linear algebra from a linear algebra book.
> Instead, I relearned linear algebra 4 or 5 times as the "basis" of other maths I've learned. I learned it for differential calculus. I relearned linear algebra for signals. I relearned linear algebra for Galois fields/CRC-codes/Reed Solomon. I relearned linear algebra for graphics.
If I were way better at websites and at advanced mathematics than I actually am, I'd make a site for learning math in a top down manner where you start with some result or application that interests you and then are taught just enough more elementary math to support that result or application.
The site would have a list of results and applications, and for each tell what math is necessary to understand it. You pick a result or application that interests you, either because it is interesting to you itself or because you see that it depends on some more elementary math that you wish to learn.
Once you pick, the site would show you a proof of the result or development of the application, at a level that one would find in a journal aimed at professionals in the relevant field. This of course will most likely be largely incomprehensible at this point.
You can select any part of the proof or development and ask the site for more information. There are two kinds of additional information you can ask for.
One is to ask for smaller steps. You use this when there is some step A -> B where you are comfortable with A and B but just don't see how it jumps from A to B. You understand what A means, what B means, just not why A -> B. The site fills in the intermediate steps.
The other is to ask what something means. This is for when the proof uses something you have not yet studies. For example if the proof uses integration and you have not yet studied it calculus that would be a great place to use a "what does this mean?" request. The site would then give you a short explanation of integration.
A key feature of the site would be that this is all recursive. If you use a "what does this mean?" request on an integral and get the short explanation of integration, you could use "smaller steps" requests and "what does this mean?" requests in that explanation.
Using "what does this mean?" requests recursively should let you go all the way down to things that can be explained with only high school algebra and precalculus.
Note that if you've never studied anything past high school algebra and precalculus and then use the site to learn something like say an analytic proof of the prime number theorem you will learn much elementary calculus but not all. You will learn just what is needed for the prime number theorem.
But there would be other interesting theorems and applications that use different parts of elementary calculus, so doing those would fill in more of your elementary calculus.
The site should have a planner that lets you pick areas of undergraduate or masters level math that you would like to learn and then shows you lists of interesting theorems and applications it has that will cover those areas.
I think this would be an interesting and effective way to learn. At all points everything you are learning goes directly toward supporting the top level proof you have chosen to learn, and you have an idea of why it is useful because you are there because you've already encountered something where you need it.
I think that for many people this will provide better motivation. In the conventional approach, where you do say a whole class in calculus or abstract algebra, then do a more advanced class that uses those results, and so on, a lot of time you are learning stuff with no idea of why it is useful.
I'll recommend Linear Algebra: A Modern Introduction by David Poole (which I picked up rather randomly in a library clearance sale for $2). It tackles most subjects from both algebraic and geometric perspectives, so from the visual aspect it might fit. What's particularly useful about it relative to HN is it leans into computational applications pretty heavily.
For example, if some particular method is computationally efficient relative to others, the text makes a note of it, and has lots of computational examples. Most of the examples could be set up fairly straightforwardly with something like a Python notebook and Numpy for matrices. It also covers things like computational errors wrt floating-point operations when doing vector and matrix calculations, efficient algorithms for approximating eigenvalues of a matrix, etc.
And!, the full text is available on archive.org with a free account:
I really liked Linear Algebra And Its Applications by David C Lay, although it seems that more people dislike it. I believe it's a pretty common book for college intro courses. It does illustrate everything pretty well if I remember correctly.
Perhaps a game development book is even more visual? I haven't read it (yet), but this book is getting recommendations: https://gamemath.com/book/
I liked Hirsch and Smale's old book called something like "Linear algebra, differential equations, and dynamical systems". It is now replaced by an expanded edition with a 3rd author added and a longer title, that I expect is also good, though I haven't looked at it.
I don't know if the H&S book is beginner friendly, but what I found good about it was studying linear algebra and differential equations at the same time, i.e. treating them as closely related topics rather than separate ones. So you could use your physical intuition about (say) a harmonic oscillator (mass on a spring, the archetypal second order ODE), then see how the 2nd order equation can be separated into a system of first order ODE's, and solved by finding matrix eigenvalues.
That worked better for me than the abstract linear algebra approach that was purely about vector spaces with nothing going on in them. It showed real sensible motivations of linear algebra.
3blue1brown is fantastic. Every once in a while I'll be like "what is the intuition behind determinants again?", and boom, there's a thoughtful and concise video on it.
However, there's no magic bullet that will let you learn linear algebra in a couple of hours. At some point you have to sit down and work to figure it out. The field has university departments researching it, so there's a lot more to it than just multiplying m×p by p×n matrices.
I don't know what you mean exactly by beginner, but assuming you have some level of mathematical maturity, UT Austin has an edx course you can audit for free, "Linear Algebra Foundations to Frontiers", and FastAI also have a pretty good free video series/course on it too.
It's definitely not the norm compared to many of the other listings in this thread but it definitely gave me a better understanding of many algebraic properties and helped build an intuition around spaces, vectors, products, etc.
It doesn't have a ton of graphics, to which you might snub your nose at it (you mentioned visual learning), but the graphics it does have are incredibly useful for building a geometric understanding of what linear algebra concepts map to. The subsection on quaternions and pseudoscalars is one of the best descriptions of such in my experience.
I've found the "_ for Dummies" series to be quite clear for math. I used Linear Algebra for Dummies to brush up on some concepts recently to solve a problem at work.
I don't remember how I found this guy but watching him feels more like learning from a friend who's extremely knowledgeable about linear algebra rather than sitting in a university course.
that book, i think, is fantastic. i was a TA for a graduate (and undergraduate) level course using it in Urbana-Champaign around 25 years ago. It's just a great book.
I’d ask a follow-up question of: what are the prerequisites for being able to successfully complete any of these courses/books? I’ve been thinking of doing something similar myself, and am 20 years removed from daily math exercises. Thanks in advance!
Algebra I and some trig, at least to get pretty deep into a first college course syllabus and get enough exposure to see where you want to go with it. That "10th grade" level of math, for instance, is actually enough to get you pretty far into the practical applications of linear algebra in cryptography, but it's not enough to get you all the way to machine learning.
I’d recommend Kahn Academy. They have a way of quickly reviewing what you know. You ought to refresh any gaps in high school math. Then take the Kahn courses in linear algebra.
For more and deeper, see the other recommendations here.
There's many good recommendations for books here. But if you're a developer without much mathematics experience, I think you can harness your dev experience to learn far more efficiently than a standard approach.
Basically, go through the exercises in the book, but use numpy, Julia, MATLAB, or similar. Learn the "Matrix API" back to front.
I've successfully taught a number of devs without a strong mathematics background with this method. The fast feedback loop this approach affords allows a good intuition to be built up very quickly.
Several commenters have mentioned 3B1B and other youtube videos. I'm curious about the suggested ordering: should one watch the videos to get an intuitive sense of things, then proceed with a textbook/practice questions? Or would it be better to struggle through a textbook/problems, and then watch videos to crystalize concepts after you've primed your brain a bit?
I realize the answers will differ for different people/situations, but I'd be curious to know what has or hasn't worked for others.
Videos and Books are Orthogonal. You switch as needed to reinforce understanding. That said, i always start with a book since they tend to be comprehensive and allow me to study at my own pace. They force me to imagine and think which i don't get from any video (which is spoon-feeding at the author's pace).
I used a combination of books, and all of them were mentioned in this thread. But the best resource that i've used as a self-learner werent books but udemy courses from a math professor
Did you take any college course on this? In class, I've found Linear Algebra to be confusing as there are many ways to say the same things and ideas are connected to each other like a giant web. I would not be able to follow along if I did not have my instructor who is willing to pause and explain it to me.
I recommend enrolling in a class, explain the instructor about your goal and s/he can find topics and walk you through them.
Lots of great suggestions here! The original question was probably answered already, but nevertheless I'd like to humbly leave a link to the article “So You Want to Study Mathematics…” by Susan Rigetti [1] — it covers not only linear algebra but other parts of math as well. Quite inspirational, I'd say.
I recommend The Manga Guide to Linear Algebra! I read it the summer before college and their visuals and analogies really helped me grasp basic concepts.
I disagree. I personally found that one to be a poorly written "Manga Guide". (Manga Guide to SQL was a good one, but there really weren't as many good analogies for Linear Algebra).
A lot of the "examples" were "This is complicated and abstract, so we'll just say it is and go to textbook form".
I am indeed here posting my original question after first trying the Manga Guide to Linear Algebra and finding it was not what I was looking for. Where I wanted visual explanation they went to textbook definitions, not helpful. A few illustrations in the book I did think were valuable so it wasn't a total loss.
LA is about vectors and rotations and stretches of vectors, which is what happens when you multiply a vector by a matrix. That’s what you will be visualizing.
Try the Kahn videos, then watch the 3B1B videos, which are very visual, but somewhat advanced. Or, watch both of them several times in parallel.
This is why I asked "what field are you learning Linear Algebra?".
Elsewhere, I've discovered that this poster is going into image processing, which is likely "signals and systems" linear algebra.
In signals and systems, your vectors can have infinite dimension, and these infinite-dimension vectors Fourier-transform into other infinite dimension vectors under a new basis.
Any field with more than "3 dimension" vectors / matricies is very difficult to visualize geometrically. Trying to do so is counter-productive to the understanding of the field. This geometric interpretation is really useful in graphics programming / 3d animation however.
---------
Or perhaps a more concrete example... your "visualize the matrix in X dimensions" advice just doesn't cut it if you're dealing with an 8x8 matrix JPEG DCT coefficient matrix (https://en.wikipedia.org/wiki/JPEG#Discrete_cosine_transform), unless you can imagine 8-diemsional space in your brain.
On the other hand, imagining the 8x8 matrix as 64 linearly-independent "Basis" to your 64-dimension discrete signal is... easier. (Well... for a definition of easier at least). And the transform from time domain into Fourier domain is a transformation in basis that contains the same information.
I believe I used this back in the day when preparing the LA course at uni, A First Course in Linear Algebra: http://linear.ups.edu/download.html .
Also I'm sure it's been mentioned here already, but the MIT Linear Algebra course by Gilbert Strang was absolutely phenomenal. Really made it click for me.
It's a great and very geometric presentation of linear algebra that takes it through to geometric algebra. A very nice presentation of a coherent set of geometric techniques.
I bought "Linear Algebra with Applications by Gareth Williams" 5th or 6th edition on ebay for 6 bucks. It's very simple and operative (manual matrix calc so you don't stay stuck in fuzzy terminology for too long). It's so easy I'm not even sure it's proper undergrad level. But it deeply unlocked my brain on the topic..
belfalas you probably feel you don't need another one but this is a great book that is friendly for visual learners (even though apparently they don't exist!)
Practical Linear Algebra - Gerald Farin & Dianne Hansford.
IIRC, the first edition was based on a work designed to educate design workers at a car firm who either didn't have the background or needed shoring up. So from the beginning it was aimed at the common person. It has since been developed over the years to it's current form.
All I know is this book has great illustrations / great analogies and was written in a way that worked _for me_ where other books did not and was a springboard to more advanced texts which as you can tell by the comments there are many. I highly reccomend.
The full name of the book is; Practical Linear Algebra: A Geometry Toolbox. The preface states The subtitle of this book is A Geometry Toolbox; this is meant
to emphasize that we approach linear algebra in a geometric and
algorithmic way (The first edition was named "The Geometry Toolbox for Graphics and Modeling").
Pair it with Introduction to Applied Linear Algebra – Vectors, Matrices, and Least Squares by Boyd and Vandenberghe (https://web.stanford.edu/~boyd/vmls/) and you are golden.
For my favourite math YouTuber besides 3B1B as is mentioned all over the comments, I really really like Steve Brunton at UW. I can't believe how much I've learned from his presentations on YouTube which led me to his book 'Data-Driven Science And Engineering' with J. Nathan Kurtz. Excellent book and excellent YouTube videos.
There's a lot of great responses, but I really like Linear Algebra by Harold Edwards. He's an excellent writer and takes an algorithmic approach in his approach. It's pretty short and designed for one-semester, and you could learn a lot from coding the various algorithms.
Try Singh's _Linear Algebra: Step by Step_, along with youtube.
Higher math tends to be abstract; you can't visualize higher-dimensional linear algebra concepts directly. The standard resources (Strang, Axler, etc) are worth the effort.
Not sure whether it's been said, but: there are not shortcuts. Sure, there are somewhat more and somewhat less intuitive texts, but there are no shortcuts. So I somewhat dislike the question.
YouTube. Khan academy. There are so many people trying to make a buck with a whiteboard online. Find one that you like ( gender, nationality, accent, whatever works for you ) and then stick to it
I remember Linear Algebra and its Applications by David Lay was very approachable and had good visuals. It’s a small book that sticks to the main points.
Scientific, replicable studies showing no correlation between "preferred learning styles" and effectiveness of learning, vs how the whole VARK theory which was built and promoted by an educator who never even tried to test the hypothesis?
For the graphical part, start with, say, (3,7). Regard that as the coordinates in the standard X-Y coordinate system of a point on the plane. So, the X coordinate is the 3 and the Y coordinate is the 7. You could get out some graph paper and plot the thing. So, more generally, given two numbers x and y, (x,y) is the coordinates of a point in the plane. We call (x,y) a vector and imagine that it is an arrow from the origin to an arrow head at point (x,y). Then we can imagine that we can slide the vector around on the plane, keeping its length and direction the same.
What we did for the plane and X-Y we could do for space with X-Y-Z. So, there a vector would have three coordinates. Ah, call them components.
Now in linear algebra, for a positive integer n, we could have a vector with n components. For geometric intuition, what we saw in X-Y or X-Y-Z is usually enough.
We can let R denote the set of real numbers. Then R^n denotes the set of all of the vectors with n components that are real numbers. Our R^n is the leading example of a vector space. Sometimes it is good to permit the components to be complex numbers, but that is a little advanced. And the components could be elements of some goofy finite field from abstract algebra, but that also is a bit advanced.
Here we just stay with the real numbers, elements of R.
Okay, suppose someone tells us
ax + by = s
cx + dy = t
where the a, b, c, d, s, t are real numbers.
We want to know what the x and y are. Right, we can think of the vector (x,y). Without too much work we can show that, depending on the coefficients a, ..., t, the set of all (x,y), that fits the two equations has none, one, or infinitely many (points, vectors) solutions.
So, that example is two equations in two unknowns, x and y. Well, for positive integers m and n, we could have m equations in n unknowns. Still, there are none, one, or infinitely many solutions.
C. F. Gauss gave us Gauss elimination that lets us know if none, one, or infinitely many, find the one, or generate as many as we wish of the infinite.
We can multiply one of the equations by a number and add the resulting equation to one of the other equations. We just did an elementary row operation, and you can convince yourself that the set of all solutions remains the same. So, Gauss elimination is to pick elementary row operations that make the pattern of coefficients have a lot of zeros so that we can by inspection read off the none, one, or infinitely many. Gauss elimination is not difficult or tricky and programs easily in C, Fortran, ....
Quite generally in math, if we have a function f, some numbers a and b, and some things, of high generality, e.g., our vectors, and it is true that for any a, b and things x and y
f(ax + by) = af(x) + bf(y)
then we say that function f is linear. Now you know why our subject is called linear algebra.
A case of a linear function is Schroedinger's equation in quantum mechanics, and linear algebra can be a good first step into some of the math of quantum mechanics.
Let's see why those equations were linear: Let
f(x,y) = ax + by
Then
f[ c(x,y) + d(u,v)]
= f[ (cx, cy) + (du,dv) ]
= f(cx + du, cy + dv)
= a(cx + du) + b(cy + dv)
= c(ax) + d(au) + c(by) + d(bv)
= c(ax + by) + d(au + bv)
= cf(x,y) + df(u,v)
Done!
This linearity is mostly what makes linear algebra get its mathematical theorems and its utility in applications.
We commonly regard the plane with coordinates X-Y as 2 dimensional and space with coordinates X-Y-Z as 3 dimensional. If we study dimension carefully, then the 2 and 3 are correct. Similarly R^n is n dimensional.
We can write
ax + by = s
cx + dy = t
as x(a,c) + y(b,d) = (s,t)
So, (a,c), (b,d), and (s,t) are vectors, and x and y are coefficients that let us write vector (s,t) as a linear combination of the two vectors (a,c) and (b,d).
Apparently the superposition in quantum mechanics is closely related to this linear combination.
Well suppose these two vectors (a,c) and (b,d) can be used in such a linear combination to get any vector (s,t). Then, omitting some details, (a,c) and (b,d) span all of R^2, are linearly independent, and form a basis for the vector space R^2.
Sure, the usual basis for R^2 is just
(1,0)
(0,1)
And that our basis has two vectors is because R^2 is 2 dimensional. Works the same in R^n -- n dimensional and a basis has n vectors that are linearly independent.
Now for some geometric intuition,
given vectors in R^3 (x,y,z) and (u,v,w), then for coefficients a and b, the set of all
a(x,y,z) + b(u,v,w)
forms, depending on the two vectors, a point, a line, or a plane through (0,0,0) -- usually a plane and, thus, a vector subspace of dimension 0, 1, 2, usually 2.
And this works in R^n: We can have vector subspaces of dimension 0, 1, ..., n. For a subspace V of dimension m, 1 <= m <= n, there will be a basis of m linearly independent vectors in subspace V.
Let's explain matrix notation: Back to
ax + by = s
cx + dy = t
On the left side, let's rip out the x and y and write the rest as
/a b\
| |
\c d/
So, this matrix has two rows and two columns. Let's call this matrix A. For positive integers m and n, we can have a matrix with m rows and n columns and call it an m x n (pronounced m by n) matrix.
The (x,y) we can now call a 1 x 2 matrix. But we really want its transpose, 2 x 1 as
/x\
| |
\y/
Let's call this matrix v.
We want to define the matrix product
Av
We define it to be just what we saw in
ax + by = s
cx + dy = t
That is, Av is the transpose of (s,t).
If we have a vector u and coefficients a and b and define matrix addition in the obvious way, we can have
that is, we have some (associativity), so that A acts like a linear function. Right, the subject is linear algebra.
And matrix multiplication is associative, and the usual proof is just an application of the interchange of summation signs for finitely many terms.
We can define the length of a vector and the angle between two vectors. then multiplying two vectors by an orthogonal matrix U does not change the length or angle of two vectors.
Then for any orthogonal matrix U, all it does is reflect and/or make a rigid rotation.
We can also have a symmetric, positive definite matrix S. What S does is stretch a sphere into an ellipsoid (the 3 dimensional case does provide good intuition). Then A can be written as SU. That is, all A can do is rotate and reflect and then move a sphere into an ellipsoid. That is the polar decomposition and is the key to much of the most advanced work in linear algebra. Turns out, once we know more about orthogonal and symmetric matrices, the proof is short.
It would be good to define a nullspace for the matrix and then show you can show the linear independence of the corresponding homogeneous system (of vectors if you wish) by having a non-zero determinant implying there is no nontrivial solution. That (L.I.) of two basis functions can just be shown by taking their sum to be zero and taking the derivative of this equation as another equation to form a like system.
I regret making this post, with no context or explanation, and I'd like to avoid making the same mistake in the future. There is always more room to grow.
I guess I'm going to have to call up that psychologist that gave my daughter that evaluation and give him a piece of your mind! But aside from the flat statement do you have anything to back it up?
I haven't watched the video, but, like your parent comment, I was already aware that "learning styles" was a research area supported almost exclusively by fraud. If you want more links, you can find them pretty easily through https://en.wikipedia.org/wiki/Learning_styles#Criticism .
I am no expert just a curious outsider, so take this with a large grain of salt, but it is my understanding that that’s one of the most pernicious misconceptions even in practicing psychologists, but that the current high quality research suggests the learning styles theory is flawed at best and wrong at worst. This article is ~8 years old but I don’t think anything has quantitatively changed the conclusions over the intervening years.
Veritasium has a very good video on the subject.[0] Sources are in the description but I might as well post them here.
Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: Concepts and evidence. Psychological science in the public interest, 9(3), 105-119. — https://ve42.co/Pashler2008
Willingham, D. T., Hughes, E. M., & Dobolyi, D. G. (2015). The scientific status of learning styles theories. Teaching of Psychology, 42(3), 266-271. — https://ve42.co/Willingham
Massa, L. J., & Mayer, R. E. (2006). Testing the ATI hypothesis: Should multimedia instruction accommodate verbalizer-visualizer cognitive style?. Learning and Individual Differences, 16(4), 321-335. — https://ve42.co/Massa2006
Riener, C., & Willingham, D. (2010). The myth of learning styles. Change: The magazine of higher learning, 42(5), 32-35.— https://ve42.co/Riener2010
Husmann, P. R., & O'Loughlin, V. D. (2019). Another nail in the coffin for learning styles? Disparities among undergraduate anatomy students’ study strategies, class performance, and reported VARK learning styles. Anatomical sciences education, 12(1), 6-19. — https://ve42.co/Husmann2019
Snider, V. E., & Roehl, R. (2007). Teachers’ beliefs about pedagogy and related issues. Psychology in the Schools, 44, 873–886. doi:10.1002/pits.20272 — https://ve42.co/Snider2007
Fleming, N., & Baume, D. (2006). Learning Styles Again: VARKing up the right tree!. Educational developments, 7(4), 4. — https://ve42.co/Fleming2006
Rogowsky, B. A., Calhoun, B. M., & Tallal, P. (2015). Matching learning style to instructional method: Effects on comprehension. Journal of educational psychology, 107(1), 64. — https://ve42.co/Rogowskyetal
Furey, W. (2020). THE STUBBORN MYTH OF LEARNING STYLES. Education Next, 20(3), 8-13. — https://ve42.co/Furey2020
Dunn, R., Beaudry, J. S., & Klavas, A. (2002). Survey of research on learning styles. California Journal of Science Education II (2). — https://ve42.co/Dunn2002
[0]: https://www.math.brown.edu/streil/papers/LADW/LADW_2017-09-0...
[1]: https://www.amazon.com/Linear-Algebra-Its-Applications-5th/d... — PDFs exist.
[2]: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...