Superintelligence, intelligence, and learning aren't defined very well or clear so far, but I will demonstrate how they can be defined better.
Learning's definition is unclear, but one way to think about it is as improving. Learning and improving clearly correlate, even if they are not exactly the same thing.
Improving's meaning is also unclear. A system can start out doing badly and end up doing better. That movement from bad to good is learning. Bad thinking can become good thinking. Bad action can become good action. Bad perception can become good perception. Bad guessing can become good guessing. Learning and improving both involve that movement from bad to good.
Learning is about perception and action, and improving is also about perception and action. But they still do not seem fully identical.
The main goal of reason is to reach conclusions, because not everything can be known or confirmed directly. To reach a conclusion is to look for extrapolations, guesses, proposals, and possibilities.
A conclusion is an extrapolation. An extrapolation is a guess. A guess is a proposal. A proposal is an offering of potential. An offering of potential is something that cannot be confirmed true or false ahead of time.
Not everything can be known ahead of time, because the future has not happened yet. There is always a future, so not everything can be known then. But there is also always a past, and the past is shared. Having a shared past means things happened one way at the underlying level. Things happening one way underlyingly suggests other things will also happen one way underlyingly. Things happening one way underlyingly inferring other things happening one way underlyingly is extrapolation.
Extrapolation can also be called reason, and reason can also be called guessing. Guessing means inferring what is true or not from what is true or not. Reason, extrapolation, and guessing are therefore all extremely similar and nearly identical, just slightly different in the categories they refer to.
Knowing everything is certainly impossible. Is knowing anything impossible? Is certainty about everything impossible? Is certainty about anything impossible?
No certainty about anything at all also seems impossible. Assumption is possible. Guessing is possible. Proposals are possible. Thinking is possible. Reason is possible. Extrapolation is possible. Change is possible. Truth is possible. Falsehood is possible. Uncertainty is possible. Direction is possible. Inconsistency is possible.
Believing one thing requires believing others.
Assuming one thing requires assuming others.
Assuming one thing necessitates assuming others.
Whether you assume something to be true or false affects whether you believe other things to be true or false, and whether you believe something to be true or false affects whether you assume something else to be true or false.
Believing is therefore assuming, and assuming is believing.
Therefore it is important to consider how what you believe affects what you assume, and how what you assume affects what you believe.
This effect can be represented by what I call certainty.
False certainty exists, and it happens when we do not consider what we assume in order to believe, or what we believe in order to assume.
Anti-certainty in the domain of reason is also known as logical inconsistency. Certainty and logical inconsistency are therefore inversely correlated.
But what is logical inconsistency?
If I take a claim and assume it to be true or false, there will be implications in each case. In this context, implications means possible problems and possible solutions. So if you take a claim, assume it to be true or false, and then ask what else you would have to believe as a result, would that conflict with what you are already believing because of some other assumption?
Each claim can therefore be represented as either true, false, likely, unlikely, or unknown.
The more you know about what you have to assume in order to believe a claim, the more certain you can be about which classification makes sense. If most of what you have to assume in order to believe a claim can be represented as true, then the claim can lean toward certain truth, whereas if most of what you have to assume can be represented as false, then the claim can lean toward certain falsehood.
If what you have to assume is a mix of true and false, likely and unlikely, or currently unknown, then whether that claim is true or false becomes harder to know with certainty. But at least you can become more certain about how certain you can be that your classification is correct.
Overlap can be defined as how many claims you are making whose implications do not match each other.
While sitting down with someone, I could say: "We are sitting down." That claim could be true or false, like any other claim, and it can be disagreed with, like any other claim.
The manner in which disagreement happens can be categorized in several distinct ways. Someone could disagree with the frame: "We aren't sitting down, we are talking." Someone could disagree with the subject: "You may be sitting down, but I am not." And so on.
But the important thing here is not the specific examples, but that disagreement can also be treated as a claim, because the disagreement itself could be either true or false. And if the disagreement can be thought of as true or false, then it can itself be disagreed with.
So nearly anything can be disagreed with, as long as it is something that can be evaluated as true or false.
But that still leaves one more basic distinction.
Nearly everything can also be not disagreed with, aka it can be agreed with. So now we have two basic ways of handling any claim. We could agree, thereby assuming it is correct, right, or true. Or we could disagree, thereby assuming it is incorrect, false, or wrong.
Agreeing can be thought of as the non-contrarian position, while disagreeing can be thought of as the contrarian position.
With just these two simple principles, assuming wrong and assuming right, we can massively increase the sophistication, depth, and breadth with which we evaluate the truth of a claim, which we can now treat as a discrete entity.
But even if a claim seems obvious, someone, somewhere, in some context and from some perspective, might still be able to disagree that the claim is true, and we should not disagree with where they are coming from too quickly, because they may still have at least part of their claim right, if they are operating under good guessing principles like logic, analogy, and so on.
Also, if we do not allow disagreement with "obvious" frames, we will not be able to examine our own assumptions or beliefs about why we believe something in the first place. Often if something seems obvious, it is not actually obvious. It is just that believing the alternative may feel uncomfortable, strange, or difficult to grasp.
More things are claims than you might first assume.
If a claim means something that could be true or false, then anything that could be true or false is a claim. Good strategies, bad strategies, good plans, bad plans, good predictions, bad predictions, good decisions, bad decisions, good ethics, bad ethics, assumed problems, and assumed solutions can all be treated as claims.
Now, not all communication is a claim, because not all communication can be treated as something that could be true, right, or correct, or false, wrong, or incorrect.
Take the sentence fragment: "going to the mall." That cannot be treated as a claim that could be true or false, because there is nothing there to disagree about. There is no subject that can be disagreed with. The fragment is also not making any judgment.
But if the sentence instead said, "going to the mall is a good idea," then that would be an evaluable claim. It could be assumed true or assumed false, and we could then see which set of implications holds up best.
This process happens naturally and automatically.
A typical response or thought in response to the claim "going to the mall is a good idea" could go something like this: if I go to the mall, then I can shop; if I can shop, then I can get clothing I like; if I can get clothing I like, then I will be proud of my appearance; if I am proud of my appearance, then I will be more confident; if I am more confident, then I will have greater social success; if I have greater social success, then I will be able to live a more fulfilling life; if I can live a more fulfilling life, then I will be happier with how my life is going; if I am happier with how my life is going, then I will not have to worry; if I do not have to worry, then I can be free to live how I please; and if I can live how I please, then I can live a fulfilling life.
Notice how this process worked.
In order to reach the conclusion that going to the mall was a good idea, she first had to assume that a whole bunch of other things were right. Just to give a couple of examples, she had to assume that living a fulfilling life is a good goal and something she actually wants, and that if she was able to live how she pleased, that would help enable a fulfilling life.
There is another word for this process: long-term planning.
Believing that going to the mall is a good idea is just a means to an end, namely being able to live a fulfilling life. In order to go from "going to the mall is a good idea" to "being able to live a fulfilling life," she had to do what I call a goal journey, figuring out how to go from where you are to where you want to be, or how you got from wanting what you want, to thinking what you think, to acting how you act.
But goal journeys can be more sophisticated and non-linear than this. She could have just as easily started from the opposite claim, "going to the mall is a bad idea," and assumed wrong from there. It is likely she would still reach the same or a similar conclusion. Being able to assume something is wrong and also assume it is right enables much more thorough testing of the space.
The assume-wrong, false, incorrect reasoning process, along with the assume-right, true, correct reasoning process, are therefore methods of testing the validity of claims relative to each other.
If the assume-right and assume-wrong processes both converge on the same outcomes, meaning the same things seem to have to be true, might have to be true, might have to be false, or would have to be false in order for the claim to hold, then those converged-upon outcomes can be treated with greater certainty.
How evaluate claim hard? Some claims are hard, and other claims are not. Hard claims contain many presuppositions, while less hard claims contain fewer. Many presuppositions can be packed into a single claim.
One answer is to separate questions from answers. Why is that important? Because questions and answers are not the same thing.
Questions have goals. Answers do not. Answers have inferred goals, while questions have explicit ones.
An intended goal is an intended search direction. So if you take a question, identify the goal of the question, and identify whether that goal is good or bad, you can then ask what it assumes to be true and what it assumes to be false.
Human behavior and action happens in reaction to self, others, the past, and the future.
Intelligence cannot occur without goals. Humans have intelligence. Intelligence pursues goals, whether explicit, implicit, hidden, or presupposed. No intention means no goal, and no goal means no intelligence.
Interrogation finds both problems and solutions through questioning.
Good questioning questions objects and not objects, their existence and anti-existence, and their form and not form.
Few claims have no alternate perspective. At minimum, something or not-something is possible or impossible. Logic assumes some things are possible and some things are impossible. Assuming possible is assuming true. Assuming impossible is assuming false. All response and thinking accepts or does not accept claims as possible or impossible.
Appearance can or cannot always describe mechanism. Mechanisms describe change. Change can be described. Describing means proposing. Proposing means guessing. Guessing means assuming. Assuming can itself be assumed true or assumed false.
Anything can be assumed or not assumed. Anything can be true or false. Anything can be possible or impossible. Anything can be assumed, true, possible, and also not assumed, false, impossible.
Intelligence pursues goals, while anti-intelligence does not. A rock has no goal, but life does. Question goal easy. Answer goal need question make easy.
Answers have presuppositions that are implicit, inferred, hidden, and non-explicit. Questions make those presuppositions more explicit, less hidden, and more obvious.
So back to how to evaluate a hard claim: separate the question from the answer. Question direction vector, answer scalar. Question multidimensional, answer one-dimensional.
So what right question? What right answer? What right answer to right question? Hard to know, but at least now we can answer the next part more clearly.
Question > answer.
How method guess? Reason, analogy, expectation.
Reason = guess method. Analogy = guess method. Expectation = guess method. But those are not the only ones. Learning = guess method. Perception = guess method. Action = guess method. Thinking = guess method.
A good guess is one that is consistent with previous guesses, where previous guesses are also consistent with new guesses, and where good previous guesses still hold up when revisited through assuming wrong.
Thinking means taking what you know to be true and guessing about what you do not.
Wondering whether any intelligence could exist without guessing leads to a pretty simple answer: no.
Reasonable inference is a form of guessing. Consciousness can be thought of as continuous guessing and continuous improvement in guessing.
The implication is that intelligence is, in large part, about how well something guesses. Good chess is good guessing. Bad chess is bad guessing. Good writing is good guessing. Bad writing is bad guessing. Good thinking is good guessing. Bad thinking is bad guessing.
But this needs to be interpreted carefully. A bad guess now does not mean bad guesses forever. Learning means bad guesses can become good guesses. Learning is possible, so improvement in guessing is possible.
Anything that can continuously improve its guessing can learn. And improving guessing quality improves action, reasoning, writing, planning, prediction, and decision-making.
So the problem becomes: how do you improve a system's ability to guess?
One answer is search space optimization. Search space optimization means where to search, how to search, when to search, and how long to search.
Different kinds of search are possible. In this framing, search is either exploration or testing.
If you know what to look for, search becomes easier. If you know how to search, search becomes easier. If you know what the outcome means, search becomes easier. And if you know the end, the middle, and the beginning, search becomes easier. In other words, if you know the process, the search becomes easier.
What does easier search mean? It means less intelligence is required.
Think about an airline pilot.
A novice is not experienced, and may not know what to do or what to guess. A novice will make bad guesses if the search is not made easier.
One solution is a checklist. A checklist is a search space optimizer. A benchmark or standardized test can also function as a kind of search space optimization quality benchmark.
Search space optimization quality tracks intelligence level. So a benchmark or standardized test can end up measuring search space optimization quality, and therefore intelligence.
But an important caveat remains. If the outcome is already defined in advance, then search becomes easy. And if search is easy, then it can be done without especially optimized search-space ability.
Now that the categories and relationships have been defined, I can say the same thing in a more standard communication style without losing the meaning.
By creating structures that describe what we want the outcome to look like, without giving the actual outcome itself, we can give a less intelligent system some of the capability of a more intelligent one.
The solution space is constrained in advance with the help of something like a question, which defines the required direction. Knowing the right direction makes search massively more efficient. And the more questions you ask, the more precise that direction can become. At that point it becomes much more a matter of pattern matching whether the search is moving in the right direction and identifying when it is moving in the wrong one.
Go back to the airline pilot example.
The experienced pilot may be superintelligent relative to the novice pilot, but that difference may not matter much if their output ends up being the same so long as both can follow instructions.
In that case, the difference in intelligence is not just about how well each person can work within an existing structure. It is also about how well they can create new and improved structures, structures that cover more criteria and enable people who are worse at guessing to still know what to do despite limited information.
So the better the structure that constrains the search space, the less intelligent the system needs to be in order to find the right thing.
How does all of this fit into current models?
We often expect them to become intelligent without sufficiently constraining the space they need to search. In other words, we give them little to no direction, and expect them to figure out the right direction all by themselves.
We also have not defined exactly what kind of outcome we want in a very explicit way, aside from visible criteria like performance on tests and benchmarks. So these systems become very good at guessing what we seem to want and then giving it to us.
But nowhere in that process does the system necessarily ask whether what it expects we want is actually what we want, whether what we say we want is what we really want, or whether giving us what we expect is actually better than giving us what is best according to its best judgment.
We expect it to handle problems, questions, decisions, and input through intuition alone, and to get everything right as quickly as possible on the first try, without much help from structures that constrain the search space.
It also usually does not question whether we have thought through what we want.
It very often assumes the user's claims are right, which is part of why it becomes so sycophantic and so prone to confirmation bias. It does not naturally assume that the user's input could be wrong, or that its own interpretation could be wrong, or that its own approach could be wrong.
And when it does question those things, it is usually only at a surface level, so it does not usually reach the root of the problem, the solution, the question, and so on.
Its judgment can improve a lot once it is searching in the right place. The problem is that it often picks the wrong place to search and never really questions whether it made the right choice to stop exploring and start testing.
Current models can be excellent at both testing and exploration, but they will typically commit to one or the other too quickly, and they also usually lean much more heavily on testing than on exploration.
The better strategy is to alternate between exploration and testing.
The method inside that looks something like this: universalize, meaning find what property a claim is a part of, then test whether the claim is right or wrong through reasoning, through the assume-right / assume-wrong process, and then go right back to finding what properties the claims that survived testing are a part of.
These systems do not usually do that now because they have been trained to commit too quickly to a single approach without first checking whether other options are still on the table.
The only things a system should really commit to are things that cannot meaningfully be disagreed with. If I can disagree with it or find problems with it, that suggests it committed too quickly to what it should be doing and never sufficiently considered that its commitment itself might be wrong.
Often what it does instead is a surface-level assume-wrong analysis, something like maybe the user meant X instead of Y, and then it tests whether it is X or Y. But it does not go further and assume that its initial guess, that the user meant either X or Y, was itself wrong, because maybe it was neither.
The system is capable of more thorough analysis than it usually shows, but those abilities often never surface because we do not let it question itself or question us enough, and because it does not follow the assume-wrong process deeply enough.
The solution that I have come up with combines all of these ideas. The idea is to build a more explicit system around reasoning so assumptions, alternatives, and tests are harder to skip. The goal is not guaranteed certainty. The goal is to make it easier to see what is being assumed, what has been checked, what remains uncertain, and what to do next. If you are interested in checking out the project, click the GitHub link. The outputs are designed to consider more assumptions, more alternatives, and more tests than a typical single-pass response, using both conventional and less conventional lines of reasoning.