Asking the right questions
Asking the right questions
@Jacob has talked in some past about asking the right questions. I'd love to learn to think more like a researcher and how to ask the right questions. Are there any books on that? Advice? Wisdom?
Re: Asking the right questions
There appears to be a textbook on it and a tedx talk. Not sure if those are good are not.
I suspect you'd get some benefit from reading some nontechnical Feynman. For inspiration to ask *very* broad sorts of questions, give Maria Popova a try. Sub to her newsletter and I recommend her book Figuring.
I suspect you'd get some benefit from reading some nontechnical Feynman. For inspiration to ask *very* broad sorts of questions, give Maria Popova a try. Sub to her newsletter and I recommend her book Figuring.
-
jacob
- Site Admin
- Posts: 16001
- Joined: Fri Jun 28, 2013 8:38 pm
- Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
- Contact:
Re: Asking the right questions
There's a famous book ( https://www.amazon.com/How-Solve-Mathem ... 069111966X ) on closed-ended problems but IIRC it's about finding answers using the hints given in the question or the provided material. Some research is like that, but I presume you're asking about the open-ended research that lead to new perspectives, theories, or insights and not just a glorified word for looking things up on the interwebs 
The secret to asking the right questions is to first know all the answers. This requires becoming thoroughly familiar with the object of study. In CCCCCC-language, you need to copy, compare, compile, and master simple "computations" before the fun stuff starts. This can be done systematically---some use notebooks and have methods; it can be done by throwing mud on the wall and hoping for the muses to sing; or it can be done with lots of experience. "Lots of experience" goes by different names. In mathematics it's called "mathematical maturity" and in physics, it's called "physical intuition".
These are both a kind of "correctly informed intuition"(*) that spontaneously matches the patterns of the new object of research to the experience of lots of old objects of research. It's reasoning by analogy. The more analogies you've burned into your neurons, the easier and stronger you get. All these analogies group into something we could call "basic principles". These "basic principles" live under a paradigm. People are often unaware of their paradigm, but you see now how one's paradigm limits the kinds of analogies one can draw. This is why novel insight often comes about when different fields collide. E.g. physicist does personal finance. Philosopher does physics.
(*) It's basically a kind of mental chunking. To give an analogy, if you see 5x7, you immediately think 35. You don't think 7-14-21-28-35 (the addition level). You don't think 7-8-9-10-11-12-13-14, two, 15-16-... three... -34-35-five. That 5x7=35 is a pattern stored in your mind. The chunking allows you or rather your brain to operate at the 35 level rather than the 7-14-21... level. And there are patterns for the 35 group too. You actually use these patterns if I give you an IQ test like: 2 3 6 18 ? If you were doing multiplication by counting on your fingers, figuring out the next number would be impossible.
Journalists use a method called https://en.wikipedia.org/wiki/Five_Ws ... for the purpose of research, I suggest the following order: What, How, Why ... The other ones are not relevant for science. They are very relevant for history and the softer arts though.
Another good method that'll help you tremendously is being able to explain X in three different ways, whichever X is. Alternatively, be able to explain X to your grandmother. You'll find that rookies often concentrate on the technical details ("The answer is given by the following equation, ...") whereas old researchers are more likely to talk about what the equation actually means.
It probably bears saying but doing research requires a tremendous amount of base knowledge. Far more than the average person has on the subject. Figure 3000 hours to get to the starting line. This is more than most people bother to learn and this is why original research is uncommon.
The secret to asking the right questions is to first know all the answers. This requires becoming thoroughly familiar with the object of study. In CCCCCC-language, you need to copy, compare, compile, and master simple "computations" before the fun stuff starts. This can be done systematically---some use notebooks and have methods; it can be done by throwing mud on the wall and hoping for the muses to sing; or it can be done with lots of experience. "Lots of experience" goes by different names. In mathematics it's called "mathematical maturity" and in physics, it's called "physical intuition".
These are both a kind of "correctly informed intuition"(*) that spontaneously matches the patterns of the new object of research to the experience of lots of old objects of research. It's reasoning by analogy. The more analogies you've burned into your neurons, the easier and stronger you get. All these analogies group into something we could call "basic principles". These "basic principles" live under a paradigm. People are often unaware of their paradigm, but you see now how one's paradigm limits the kinds of analogies one can draw. This is why novel insight often comes about when different fields collide. E.g. physicist does personal finance. Philosopher does physics.
(*) It's basically a kind of mental chunking. To give an analogy, if you see 5x7, you immediately think 35. You don't think 7-14-21-28-35 (the addition level). You don't think 7-8-9-10-11-12-13-14, two, 15-16-... three... -34-35-five. That 5x7=35 is a pattern stored in your mind. The chunking allows you or rather your brain to operate at the 35 level rather than the 7-14-21... level. And there are patterns for the 35 group too. You actually use these patterns if I give you an IQ test like: 2 3 6 18 ? If you were doing multiplication by counting on your fingers, figuring out the next number would be impossible.
Journalists use a method called https://en.wikipedia.org/wiki/Five_Ws ... for the purpose of research, I suggest the following order: What, How, Why ... The other ones are not relevant for science. They are very relevant for history and the softer arts though.
Another good method that'll help you tremendously is being able to explain X in three different ways, whichever X is. Alternatively, be able to explain X to your grandmother. You'll find that rookies often concentrate on the technical details ("The answer is given by the following equation, ...") whereas old researchers are more likely to talk about what the equation actually means.
It probably bears saying but doing research requires a tremendous amount of base knowledge. Far more than the average person has on the subject. Figure 3000 hours to get to the starting line. This is more than most people bother to learn and this is why original research is uncommon.
Re: Asking the right questions
One way to look at it may be that asking questions and interpreting their answers requires a world model (or at least some kind of model). Good questions update the model by either (1) compressing it while retaining explanatory power or (2) expanding it to include more explanatory power (3) direct you towards new foundations.
Starting with some simple and robust models from STEM fields helps as there are many examples of these models being successfully stacked together to ask interesting questions and solve novel problems. Though, any starting point should work with enough iteration. It might even be said that western science and philosophy has been aimed at answering the questions proposed by Plato over two thousand years ago, so it doesn't hurt to familiarize yourself with the classics.
Sitting where you are at for a while may be your best bet for getting to rich questions quickly. What I mean by that is you are already likely using a pretty sophisticated framework or system from which you can generate ideas or questions. Explore this for a while by going through several cycles of (1) contemplation (2) questioning and (3) answering (or the attempt of such). Try to avoid going to external sources to alleviate your uncertainty, especially when that uncertainty is uncomfortable. Doing this you will start to map out the edges of your world model where it starts to break down. Use these limitations to motivate targeted exploration of external sources.
Starting with some simple and robust models from STEM fields helps as there are many examples of these models being successfully stacked together to ask interesting questions and solve novel problems. Though, any starting point should work with enough iteration. It might even be said that western science and philosophy has been aimed at answering the questions proposed by Plato over two thousand years ago, so it doesn't hurt to familiarize yourself with the classics.
Sitting where you are at for a while may be your best bet for getting to rich questions quickly. What I mean by that is you are already likely using a pretty sophisticated framework or system from which you can generate ideas or questions. Explore this for a while by going through several cycles of (1) contemplation (2) questioning and (3) answering (or the attempt of such). Try to avoid going to external sources to alleviate your uncertainty, especially when that uncertainty is uncomfortable. Doing this you will start to map out the edges of your world model where it starts to break down. Use these limitations to motivate targeted exploration of external sources.
-
jacob
- Site Admin
- Posts: 16001
- Joined: Fri Jun 28, 2013 8:38 pm
- Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
- Contact:
Re: Asking the right questions
To expound on what @daylen said, it is crucial for research to be operating with a world-model that is abstract, that is, one level above how one currently perceives the world. This means differentiating between theory and practice: between the map and the territory. Research is all about figuring out a better map. To do this requires "thinking about thinking". Humans are the only animals capable of that. Not all humans do it though. It's definitely possible to deal with the territory using but a memorized map w/o questioning whether there might be a better map.
Take a look at this: https://en.wikipedia.org/wiki/Model_of_ ... complexity
While one might claim that a baby does research on locomotion when they're figuring out how to walk (to the point where they just do it w/o thinking... just like learning the multiplication table), research in its professional vernacular is building the bridge between MHC11 to 12 and 13. Those we hail as geniuses (the Einsteins, the Newtons, the Kochs) operate at MHC14.
The ERE book sits at MHC12.5. To translate, the Renaissance concept with its telic WOGs, etc. is fundamentally about coordinating a multivariate system of systems (the different capitals). This also explains why ERE is not a quick&easy solution and why it takes the average person ~10 years to reach WL7.
Take a look at this: https://en.wikipedia.org/wiki/Model_of_ ... complexity
While one might claim that a baby does research on locomotion when they're figuring out how to walk (to the point where they just do it w/o thinking... just like learning the multiplication table), research in its professional vernacular is building the bridge between MHC11 to 12 and 13. Those we hail as geniuses (the Einsteins, the Newtons, the Kochs) operate at MHC14.
The ERE book sits at MHC12.5. To translate, the Renaissance concept with its telic WOGs, etc. is fundamentally about coordinating a multivariate system of systems (the different capitals). This also explains why ERE is not a quick&easy solution and why it takes the average person ~10 years to reach WL7.
Re: Asking the right questions
Question: I wonder if Michael Commons ever had the actual experience of spending hour after hour attempting to teach early algebra one-on-one to humans who have not yet memorized their multiplication tables?
ETA:
Answer; Irrelevant. Minor point I would make being that as with other hierarchal models, including ERE Wheaton levels, the specifics Commons assigns to each level do not necessarily correspond to the attainment of each level. Therefore, my observation that it is actually possible, depending on structure of course-work, to pass through almost all of high-school level math with only Concrete Level functioning. IOW, you can just keep memorizing algorithm after algorithm at higher levels without being able to conceptualize at higher levels. Therefore, for example, early algebra can actually be easier to accomplish for some learners than long-division, simply because the algorithms are shorter. OTOH, an otherwise competent adult who has been long out of school with no reason to perform the long-division algorithm for multiple decades, or a bright disadvantaged child who has not been well-educated/trained-in-algorithms-or-even-basic-facts, can still conceptualize division and sometimes come up with accurate result through any number of bodged methods.
For instance, I attempted to test one of my bright disadvantaged 10 year olds with Formal level questions using non-mathematical language and she did pretty well. Also, it's very commonplace to observe girls whose social functioning is apparently much higher level than their mathematical functioning, because they are more interested in solving complex social problems.
ETA:
Answer; Irrelevant. Minor point I would make being that as with other hierarchal models, including ERE Wheaton levels, the specifics Commons assigns to each level do not necessarily correspond to the attainment of each level. Therefore, my observation that it is actually possible, depending on structure of course-work, to pass through almost all of high-school level math with only Concrete Level functioning. IOW, you can just keep memorizing algorithm after algorithm at higher levels without being able to conceptualize at higher levels. Therefore, for example, early algebra can actually be easier to accomplish for some learners than long-division, simply because the algorithms are shorter. OTOH, an otherwise competent adult who has been long out of school with no reason to perform the long-division algorithm for multiple decades, or a bright disadvantaged child who has not been well-educated/trained-in-algorithms-or-even-basic-facts, can still conceptualize division and sometimes come up with accurate result through any number of bodged methods.
For instance, I attempted to test one of my bright disadvantaged 10 year olds with Formal level questions using non-mathematical language and she did pretty well. Also, it's very commonplace to observe girls whose social functioning is apparently much higher level than their mathematical functioning, because they are more interested in solving complex social problems.
Re: Asking the right questions
Did you come up with these 6 C's? I heard you mention it on a podcast. If not, where can I read more about it.jacob wrote: ↑Wed Feb 21, 2024 10:24 amThe secret to asking the right questions is to first know all the answers. This requires becoming thoroughly familiar with the object of study. In CCCCCC-language, you need to copy, compare, compile, and master simple "computations" before the fun stuff starts. This can be done systematically---some use notebooks and have methods; it can be done by throwing mud on the wall and hoping for the muses to sing; or it can be done with lots of experience. "Lots of experience" goes by different names. In mathematics it's called "mathematical maturity" and in physics, it's called "physical intuition".
I like that "research us all about finding a better map". So it seems like you are saying vast experience and internalizing fundamental concepts is the prerequisite for asking better questions. That is good to know for my career as a software engineer.
I was also curious how to ask original and thought-provoking questions with respect to my life.
Re: Asking the right questions
I wish I'd read Superthinking, before I started trying to have original questions. I burned a lot of energy, only to find that short book contains 90% of what I learned.
Not the most sophisticated answer, but it was a "you've got to be kidding me!!!" moment.
Not the most sophisticated answer, but it was a "you've got to be kidding me!!!" moment.
-
jacob
- Site Admin
- Posts: 16001
- Joined: Fri Jun 28, 2013 8:38 pm
- Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
- Contact:
Re: Asking the right questions
Yes, and here: https://www.amazon.com/Early-Retirement ... 145360121X (chapter 4 IIRC)
-
mountainFrugal
- Posts: 1144
- Joined: Fri May 07, 2021 2:26 pm
Re: Asking the right questions
I often recommend the following two books by MIT teaching prof Sanjoy Mahajan to grad students for getting better at ball parking and thinking about complex problems:
Street Fighting Mathematics: https://sept.mit.edu/sites/default/file ... matics.pdf
The Art of Insight: https://library.oapen.org/handle/20.500.12657/26055
Once you have the basics down you can start asking your own unique questions. However, another skill to develop is being able to tell when you might be wrong about something or when and what new information you would need to change your mind about something in light of new evidence. Julia Galef's book The Scout Mindset is a good introduction to meta-cognition and thinking about thinking.
https://www.penguinrandomhouse.com/book ... lia-galef/
Street Fighting Mathematics: https://sept.mit.edu/sites/default/file ... matics.pdf
The Art of Insight: https://library.oapen.org/handle/20.500.12657/26055
Once you have the basics down you can start asking your own unique questions. However, another skill to develop is being able to tell when you might be wrong about something or when and what new information you would need to change your mind about something in light of new evidence. Julia Galef's book The Scout Mindset is a good introduction to meta-cognition and thinking about thinking.
https://www.penguinrandomhouse.com/book ... lia-galef/
Re: Asking the right questions
I'm currently reading "No More Gold Stars: Regenerating Capacity to Think for Ourselves" by Carol Sanford. It is a more towards Green/Turquoise, anti-20th century behavioral science /organizational theory take on the topic. You can tell that it's going to be good right from the get-go, because Sanford chooses to have both Tom Peters and Tyson Yumkaporta write interesting forewords.
Re: Asking the right questions
Thanks Jacob. I am embarrassed I forgot. I read your book at least 4 times.jacob wrote: ↑Thu Feb 22, 2024 7:07 amYes, and here: https://www.amazon.com/Early-Retirement ... 145360121X (chapter 4 IIRC)
Those books look awesome - look forward to reading the second one. I heard you interview on an obscured youtube podcast and it was great. I learned a lot from you and you got me into process design and applying better processes to my own life.mountainFrugal wrote:
Re: Asking the right questions
Damn, those look great. Thanks for the share.mountainFrugal wrote: ↑Fri Feb 23, 2024 4:01 pmI often recommend the following two books by MIT teaching prof Sanjoy Mahajan to grad students for getting better at ball parking and thinking about complex problems:
Street Fighting Mathematics: https://sept.mit.edu/sites/default/file ... matics.pdf
The Art of Insight: https://library.oapen.org/handle/20.500.12657/26055
Also worth pointing out those are both under the creative common attribution (and share-alike) license, so free to download... Everyone, call to action: download them.
Re: Asking the right questions
@mF:
Yes, thank you for the links! Downloaded the two and added the third to my library list.
Yes, thank you for the links! Downloaded the two and added the third to my library list.