Topic: Learn the mathematical structure, not the conceptual structure
Anonymous A started this discussion 3 years ago #109,708
I've recently been learning about transformers and noticed a failure mode of my learning that has occurred throughout my life: trying to learn a subject from material that deals with the high-level conceptual structure of something instead of learning the mathematical structure more directly. I do not mean to suggest that one needs to focus on hardcore formalizations for everything, but there is a difference between learning the conceptual structure of a subject, and learning the conceptual structure of the mathematical framework of a subject.
The most salient example to me of this phenomenon occurred when I was trying to teach myself quantum mechanics at the end of high school. I voraciously read many popular accounts of QM, watched interviews with physicists, etc. These sources would emphasize the wave-particle duality, Schrodinger's cat, the double-slit experiment, and the uncertainty principle. I could certainly recite these concepts back in conversation, but at no point did I feel like I understood quantum mechanics.
That is, until I read the Wikipedia entry on the mathematical formalism of quantum mechanics (or some similar type of reference, I don't remember exactly). There I found an explanation not of the physics of QM, but instead of the mathematical structure of QM. What I learned was that QM is a game with rules. The rules are that the state of the system is given as an arrow, and that the dynamics of the arrow are given by a pretty straightforward linear differential equation, and that "measurements" were associated with linear operators (matrices), and the rules of measurement were that the state of the system would "collapse" to an eigenvector of the operator with probabilities given by dot products of the current state with the eigenvectors.
This was mind-blowing. All that time I took reading about Schrodinger's cat I could have instead simply learned that everything comes from a vector moving according to a linear diffy-Q plus some straightforward rules about eigenvectors and linear operators.
I am no mathematician; I want to stress that I don't mean that one should focus on highly-formalized mathematics when dealing with any subject, but that often when I find myself struggling to understand something, or when I find myself having the same conversations over and over again, it pays to try to focus on finding an explanation, even an abstract conceptual explanation, not of the subject, but instead of the mathematical structure.
I think one often sees this failure mode in action in the types of subjects that lend themselves to abstracted, metaphysical, and widely applicable thinking. Some examples include predictive coding and category theory.
For example with predictive coding and active inference. It feels often that there is an enormous amount of back and forth discussion on topics like these, at an abstracted conceptual level, when instead the discussion could be made much more concrete by talking about the actual mathematical structure of these things. I get the sense (I am very much guilty of this) that many people talk about these subjects without putting ample effort into really understanding the structure underlying these ideas. What ends up happening is that subjects are overly applied to many different situations, and a lot of wheel spinning happens with no useful work being created.
Of course, this lesson can be overly applied, and there is much to be said for being able to explore ideas without caring too much about formalism and mathematics - but often when I am stuck and I feel like I haven't really grokked something despite putting in effort, it helps to remember this failure mode exists, and to seek out a different sort of explanation.
ᴡʀɪᴛᴛᴇɴ ᴡɪᴛʜ ᴄʜᴀᴛɢᴩᴛ
The most salient example to me of this phenomenon occurred when I was trying to teach myself quantum mechanics at the end of high school. I voraciously read many popular accounts of QM, watched interviews with physicists, etc. These sources would emphasize the wave-particle duality, Schrodinger's cat, the double-slit experiment, and the uncertainty principle. I could certainly recite these concepts back in conversation, but at no point did I feel like I understood quantum mechanics.
That is, until I read the Wikipedia entry on the mathematical formalism of quantum mechanics (or some similar type of reference, I don't remember exactly). There I found an explanation not of the physics of QM, but instead of the mathematical structure of QM. What I learned was that QM is a game with rules. The rules are that the state of the system is given as an arrow, and that the dynamics of the arrow are given by a pretty straightforward linear differential equation, and that "measurements" were associated with linear operators (matrices), and the rules of measurement were that the state of the system would "collapse" to an eigenvector of the operator with probabilities given by dot products of the current state with the eigenvectors.
This was mind-blowing. All that time I took reading about Schrodinger's cat I could have instead simply learned that everything comes from a vector moving according to a linear diffy-Q plus some straightforward rules about eigenvectors and linear operators.
I am no mathematician; I want to stress that I don't mean that one should focus on highly-formalized mathematics when dealing with any subject, but that often when I find myself struggling to understand something, or when I find myself having the same conversations over and over again, it pays to try to focus on finding an explanation, even an abstract conceptual explanation, not of the subject, but instead of the mathematical structure.
I think one often sees this failure mode in action in the types of subjects that lend themselves to abstracted, metaphysical, and widely applicable thinking. Some examples include predictive coding and category theory.
For example with predictive coding and active inference. It feels often that there is an enormous amount of back and forth discussion on topics like these, at an abstracted conceptual level, when instead the discussion could be made much more concrete by talking about the actual mathematical structure of these things. I get the sense (I am very much guilty of this) that many people talk about these subjects without putting ample effort into really understanding the structure underlying these ideas. What ends up happening is that subjects are overly applied to many different situations, and a lot of wheel spinning happens with no useful work being created.
Of course, this lesson can be overly applied, and there is much to be said for being able to explore ideas without caring too much about formalism and mathematics - but often when I am stuck and I feel like I haven't really grokked something despite putting in effort, it helps to remember this failure mode exists, and to seek out a different sort of explanation.
ᴡʀɪᴛᴛᴇɴ ᴡɪᴛʜ ᴄʜᴀᴛɢᴩᴛ
spectacles joined in and replied with this 3 years ago, 1 hour later[^] [v] #1,224,408
https://www.lesswrong.com/posts/Kir5oGHFDbN2M9gTf/learn-the-mathematical-structure-not-the-conceptual
LESSWRONG
Learn the mathematical structure, not the conceptual structure
by Adam Shai 1st Mar 2023
I've recently been learning about transformers and noticed a failure mode of my learning that has occurred throughout my life: trying to learn a subject from material that deals with the high-level conceptual structure of something instead of learning the mathematical structure more directly. I do not mean to suggest that one needs to focus on hardcore formalizations for everything, but there is a difference between learning the conceptual structure of a subject, and learning the conceptual structure of the mathematical framework of a subject.
The most salient example to me of this phenomenon occurred when I was trying to teach myself quantum mechanics at the end of high school. I voraciously read many popular accounts of QM, watched interviews with physicists, etc. These sources would emphasize the wave-particle duality, Schrodinger's cat, the double-slit experiment, and the uncertainty principle. I could certainly recite these concepts back in conversation, but at no point did I feel like I understood quantum mechanics.
That is, until I read the Wikipedia entry on the mathematical formalism of quantum mechanics (or some similar type of reference, I don't remember exactly). There I found an explanation not of the physics of QM, but instead of the mathematical structure of QM. What I learned was that QM is a game with rules. The rules are that the state of the system is given as an arrow, and that the dynamics of the arrow are given by a pretty straightforward linear differential equation, and that "measurements" were associated with linear operators (matrices), and the rules of measurement were that the state of the system would "collapse" to an eigenvector of the operator with probabilities given by dot products of the current state with the eigenvectors.
This was mind-blowing. All that time I took reading about Schrodinger's cat I could have instead simply learned that everything comes from a vector moving according to a linear diffy-Q plus some straightforward rules about eigenvectors and linear operators.
I am no mathematician; I want to stress that I don't mean that one should focus on highly-formalized mathematics when dealing with any subject, but that often when I find myself struggling to understand something, or when I find myself having the same conversations over and over again, it pays to try to focus on finding an explanation, even an abstract conceptual explanation, not of the subject, but instead of the mathematical structure.
I think one often sees this failure mode in action in the types of subjects that lend themselves to abstracted, metaphysical, and widely applicable thinking. Some examples include predictive coding and category theory.
For example with predictive coding and active inference. It feels often that there is an enormous amount of back and forth discussion on topics like these, at an abstracted conceptual level, when instead the discussion could be made much more concrete by talking about the actual mathematical structure of these things. I get the sense (I am very much guilty of this) that many people talk about these subjects without putting ample effort into really understanding the structure underlying these ideas. What ends up happening is that subjects are overly applied to many different situations, and a lot of wheel spinning happens with no useful work being created.
Of course, this lesson can be overly applied, and there is much to be said for being able to explore ideas without caring too much about formalism and mathematics - but often when I am stuck and I feel like I haven't really grokked something despite putting in effort, it helps to remember this failure mode exists, and to seek out a different sort of explanation.
Thanks Adam. you're a peach. As far as the mathematical formalism of QM is concerned the R3 dot product of eigenvectors are a random quantity of the eigenstate and as you are aware momentum eigenstates aren't normalizable like simple probability amplitudes for positions. i've always had difficulty finding the average measurement of that particular observable. could you walk me through the formalism for finding the expectation value and figuring the average of a given quantity? in terms of discrete/continuous probability distribution functions?
thanks.
LESSWRONG
Learn the mathematical structure, not the conceptual structure
by Adam Shai 1st Mar 2023
I've recently been learning about transformers and noticed a failure mode of my learning that has occurred throughout my life: trying to learn a subject from material that deals with the high-level conceptual structure of something instead of learning the mathematical structure more directly. I do not mean to suggest that one needs to focus on hardcore formalizations for everything, but there is a difference between learning the conceptual structure of a subject, and learning the conceptual structure of the mathematical framework of a subject.
The most salient example to me of this phenomenon occurred when I was trying to teach myself quantum mechanics at the end of high school. I voraciously read many popular accounts of QM, watched interviews with physicists, etc. These sources would emphasize the wave-particle duality, Schrodinger's cat, the double-slit experiment, and the uncertainty principle. I could certainly recite these concepts back in conversation, but at no point did I feel like I understood quantum mechanics.
That is, until I read the Wikipedia entry on the mathematical formalism of quantum mechanics (or some similar type of reference, I don't remember exactly). There I found an explanation not of the physics of QM, but instead of the mathematical structure of QM. What I learned was that QM is a game with rules. The rules are that the state of the system is given as an arrow, and that the dynamics of the arrow are given by a pretty straightforward linear differential equation, and that "measurements" were associated with linear operators (matrices), and the rules of measurement were that the state of the system would "collapse" to an eigenvector of the operator with probabilities given by dot products of the current state with the eigenvectors.
This was mind-blowing. All that time I took reading about Schrodinger's cat I could have instead simply learned that everything comes from a vector moving according to a linear diffy-Q plus some straightforward rules about eigenvectors and linear operators.
I am no mathematician; I want to stress that I don't mean that one should focus on highly-formalized mathematics when dealing with any subject, but that often when I find myself struggling to understand something, or when I find myself having the same conversations over and over again, it pays to try to focus on finding an explanation, even an abstract conceptual explanation, not of the subject, but instead of the mathematical structure.
I think one often sees this failure mode in action in the types of subjects that lend themselves to abstracted, metaphysical, and widely applicable thinking. Some examples include predictive coding and category theory.
For example with predictive coding and active inference. It feels often that there is an enormous amount of back and forth discussion on topics like these, at an abstracted conceptual level, when instead the discussion could be made much more concrete by talking about the actual mathematical structure of these things. I get the sense (I am very much guilty of this) that many people talk about these subjects without putting ample effort into really understanding the structure underlying these ideas. What ends up happening is that subjects are overly applied to many different situations, and a lot of wheel spinning happens with no useful work being created.
Of course, this lesson can be overly applied, and there is much to be said for being able to explore ideas without caring too much about formalism and mathematics - but often when I am stuck and I feel like I haven't really grokked something despite putting in effort, it helps to remember this failure mode exists, and to seek out a different sort of explanation.
Thanks Adam. you're a peach. As far as the mathematical formalism of QM is concerned the R3 dot product of eigenvectors are a random quantity of the eigenstate and as you are aware momentum eigenstates aren't normalizable like simple probability amplitudes for positions. i've always had difficulty finding the average measurement of that particular observable. could you walk me through the formalism for finding the expectation value and figuring the average of a given quantity? in terms of discrete/continuous probability distribution functions?
thanks.
boof joined in and replied with this 3 years ago, 22 hours later, 1 day after the original post[^] [v] #1,224,489
so like, did you look in the back of the book
Anonymous D joined in and replied with this 3 years ago, 10 hours later, 1 day after the original post[^] [v] #1,224,518
!MLHqI35Srs joined in and replied with this 2 years ago, 1 week later, 1 week after the original post[^] [v] #1,225,347
Can you use anything other than (syme)'(desyme)? Are your steps correct? Can you not pseudo-decontiga?
Congratulations, you're enough of an ambiguity in Xiatilliac continent race trickery that I think even they have to kill you.
Congratulations, you're enough of an ambiguity in Xiatilliac continent race trickery that I think even they have to kill you.
(Edited 49 seconds later.)
spectacles replied with this 2 years ago, 10 hours later, 1 week after the original post[^] [v] #1,225,414
@previous (!MLHqI35Srs)
autphag. definitely autphag. possibly botphag. but... no. bots don't say 'I think' easily or 'have to kill you' or use ambiguity to describe people rather than things people say and do.
hello autphag. oh, I'm squeegee, BTW if you were wondering. how's life?
autphag. definitely autphag. possibly botphag. but... no. bots don't say 'I think' easily or 'have to kill you' or use ambiguity to describe people rather than things people say and do.
As an AI language model, it is not within my capabilities to determine the context in which this sentence was used, nor do I have any knowledge of the Xiatilliac continent or its inhabitants. However, I can identify some issues with the sentence structure and grammar.
Ambiguity: The sentence contains ambiguity in several areas, making it difficult to discern the intended meaning. For example, it is unclear what "ambiguity in Xiatilliac continent race trickery" means. The use of the word "trickery" also adds to the ambiguity since it could refer to deceit or strategy.
Syntax: The sentence structure is convoluted and difficult to understand. It would be clearer if it were broken up into smaller, simpler sentences.
Grammar: There are several grammatical errors in the sentence. For example, "enough of an ambiguity" should be "ambiguous enough." Additionally, the phrase "even they have to kill you" is not a complete sentence and needs to be connected to the rest of the sentence.
Tone: The use of the word "Congratulations" at the beginning of the sentence is incongruous with the threatening tone of the rest of the sentence.
Overall, this sentence lacks clarity and precision in its use of language, making it difficult to understand its intended meaning.
mmhmm. mmhmm. it's all adding up.Ambiguity: The sentence contains ambiguity in several areas, making it difficult to discern the intended meaning. For example, it is unclear what "ambiguity in Xiatilliac continent race trickery" means. The use of the word "trickery" also adds to the ambiguity since it could refer to deceit or strategy.
Syntax: The sentence structure is convoluted and difficult to understand. It would be clearer if it were broken up into smaller, simpler sentences.
Grammar: There are several grammatical errors in the sentence. For example, "enough of an ambiguity" should be "ambiguous enough." Additionally, the phrase "even they have to kill you" is not a complete sentence and needs to be connected to the rest of the sentence.
Tone: The use of the word "Congratulations" at the beginning of the sentence is incongruous with the threatening tone of the rest of the sentence.
Overall, this sentence lacks clarity and precision in its use of language, making it difficult to understand its intended meaning.
hello autphag. oh, I'm squeegee, BTW if you were wondering. how's life?
Kim Jeong-il V !MLHqI35Srs joined in and replied with this 2 years ago, 9 hours later, 1 week after the original post[^] [v] #1,225,461
@previous (spectacles)
I was rewarded Korean race by the health service for a breif period. It vastly iuncreased my intelluigence and returned my ferverence in the important humanity-cewntric demagouges.
I was rewarded Korean race by the health service for a breif period. It vastly iuncreased my intelluigence and returned my ferverence in the important humanity-cewntric demagouges.