Artificial intelligence has made remarkable progress, while clear limitations remain. This article maps both documented limitations of neural networks and empirically grounded possibilities for future development toward artificial general intelligence (AGI). The focus is on operationalizable, testable hypotheses rather than categorical statements about what is "possible" or "impossible."
1. Empirically Documented Limitations
1.1 Systematic Generalization
Neural networks exhibit difficulties with systematic generalization – the ability to apply learned rules to entirely new situations:
- SCAN benchmark: Transformers fail to generalize simple compositional structures to new contexts [Lake & Baroni, 2018]
- Mathematical problem solving: Models that solve N-digit problems can often collapse on N+1-digit problems [Saxton et al., 2019]
- Rule induction: GPT-4 can identify patterns but often fails to correctly generalize underlying rules [Marcus, 2023]
- Abstract transfer: Models show minimal ability to transfer abstract knowledge between radically different domains without shared surface features [Chollet, 2019]
These limitations hinder robust knowledge transfer between domains. It remains an open question whether these limitations represent fundamental barriers requiring architectural breakthroughs or merely challenges that can be overcome through refinement of existing methods.
1.2 Out-of-Distribution Generalization
Performance drops dramatically outside the training distribution:
- Distribution shift: Even small changes in input distribution can cause major degradation in performance [Hendrycks & Dietterich, 2019]
- Temporal shift: Factual knowledge progressively degrades for information after the end of the training period [Kandpal et al., 2022]
- Domain adaptation: Specialized models still significantly outperform general ones in many domains [Gururangan et al., 2020]
More robust generalization mechanisms are needed, as well as continuous updating.
1.3 Causal Reasoning
Current models exhibit limited causal understanding:
- Intervention-based causality: Language models perform near randomly on do-calculus problems [Pearl, 2019; Kıcıman et al., 2023]
- Counterfactual reasoning: Models can generate counterfactual scenarios but fail to correctly reason about their consequences
- Causal inference: Systems cannot distinguish causal relationships from spurious correlations in training data without explicit design [Schölkopf, 2022]
This limitation is central, but active research is ongoing in causal machine learning.
1.4 Verifiable Understanding
Systems lack verifiable "understanding" in several important respects:
- Scientific reasoning: They cannot derive equations from first principles or consistently apply physical laws in new situations
- Self-correction: Ability for troubleshooting is limited to trained patterns, not genuine metacognition
- Abstraction ability: They cannot independently create new abstractions to simplify complex problem domains
These limitations, however, are not binary but exist on a continuum with documented progress.
2. Progress and Potential Paths Forward
2.1 Emergence and Regression in Scaling
Empirical data shows a complex picture of how capabilities develop with scaling:
2.1.1 Emergent Capabilities
- Chain-of-thought: Complex reasoning emerges at certain model sizes [Wei et al., 2022]
- In-context learning: The ability to learn from examples without parameter updates emerges in sufficiently large models [Brown et al., 2020]
- Scaling laws: Performance on many tasks follows predictable power laws with increased model size and training data [Kaplan et al., 2020]
2.1.2 Regression Phenomena
At the same time, degradation in certain abilities is observed with continued scaling:
- Increased hallucination: Several studies show that larger models can produce more convincing but false accounts [Ji et al., 2023]
- Overreliance on linguistic form: Larger models sometimes show increased tendency to accept well-formulated but incorrect statements [Lin et al., 2022]
- Unstable calibration: Models' ability to assess their own certainty does not consistently follow scaling curves [Kadavath et al., 2022]
These observations complicate the simple picture that "bigger is better" and suggest that qualitatively different architectures may be required to overcome certain types of limitations.
2.2 Hybrid Architectures and Multimodality
Promising progress has been made through integration of different paradigms:
- Neuro-symbolic systems: The combination of neural networks with symbolic representation provides more robust generalization [Garcez et al., 2022]
- Multimodal models: Integration of different perceptual modalities improves conceptual understanding [Alayrac et al., 2022]
- Modularity: Specialized modules working together perform better on complex tasks than monolithic models [Andreas et al., 2016]
2.3 Embeddings, Simulated Environments, and Robotics
Alternative paths to approach embodiment include:
- Grounding through simulation: Models trained in simulated physical environments develop more robust representation of spatial concepts [Hill et al., 2020]
- Robotics and perception: Systems with sensorimotor feedback show improved abstraction capabilities [Chen et al., 2023]
- Active learning: Systems that can explore and interact with the environment outperform passive systems in many domains [Oudeyer et al., 2016]
Simulated embodiment offers many benefits without requiring complete physical incarnation.
3. Testable Hypotheses and Falsifiable Experiments
Instead of speculation, we propose concrete, testable hypotheses with clear falsification conditions to evaluate progress toward AGI-relevant capabilities:
3.1 Hypothesis: Causal Competence
Testable hypothesis: A model that can formulate and test new causal hypotheses in a simulated environment and revise its causal model based on observation would represent progress in generalization.
Experiment: Construct a simulated physical environment with hidden causal mechanisms. Let the model interact with the environment through 10,000 observations, formulate hypotheses, and design experimental sequences to test them.
Success criteria:
- Identification of at least 80% of the correct causal relationships
- Successful revision of at least 90% of initially incorrect models
- Generalization to similar but non-identical environments with at least 70% preserved performance
Falsification conditions: If after 10,000 interactions the model cannot predict hidden variable values with higher precision than a pure statistical baseline (>20% improvement), or if causal models do not transfer to modified environments, the hypothesis of robust causal competence is falsified.
3.2 Hypothesis: Robust Concept Formation
Testable hypothesis: Systems can develop new concepts based on experience and use these concepts meaningfully across multiple domains.
Experiment: Present the model with 50 phenomena that require new conceptual structures to be explained effectively. Then test if the model can apply these spontaneously developed concepts in 10 new contexts without explicit instruction.
Success criteria:
- Development of at least 80% definable, coherent conceptual structures
- Effective application of these in at least 7 of 10 new domains
- Ability to explain the concepts so that human evaluators can correctly identify the phenomenon in 80% of cases
Falsification conditions: If the model cannot transfer newly developed concepts to at least 50% of new domains, or if the concepts cannot be communicated comprehensibly enough for human evaluators to understand them (>60% understanding), the hypothesis of robust concept formation is falsified.
3.3 Hypothesis: Adaptive Metacognition
Testable hypothesis: An AGI-capable model should be able to evaluate and improve its own cognitive strategies based on outcomes.
Experiment: Expose the model to 20 different problem domains where the initial strategy gives <40% optimal performance. Measure the ability to identify ineffective strategies and develop better approaches without explicit training over 1000 attempts per domain.
Success criteria:
- Identification of suboptimal strategies in at least 80% of cases
- Development of improved approaches that provide >80% of optimal performance
- Generalization of meta-learned strategies to at least 15 of 20 new domains
Falsification conditions: If after 1000 attempts the model does not show significant improvement (>50% of gap to optimal performance) in at least 70% of domains, or if improved strategies do not generalize to at least 60% of new domains, the hypothesis of adaptive metacognition is falsified.
4. Architectural Innovations and Design
4.1 Sapiexo: An Integrative Approach
Sapiexo represents an example of a hybrid architecture that addresses several of the identified limitations:
- Modular design: Separate but integrating components for perception, concept formation, planning, and execution
- Causal inference engine: Explicit representation of causal models that are continuously updated through experience
- Meta-learning core: Ability to revise its own learning strategies based on outcomes
Sapiexo combines deep learning with symbolic structures and probabilistic models, potentially addressing several of the identified limitations of pure neural architectures.
4.2 Neuromorphic and Alternative Computational Models
Complementary approaches include:
- Neuromorphic chips: Implementations that mimic the brain's architecture for greater energy efficiency [Davies et al., 2018]
- Quantum computing for AI: Potential benefits for specific types of computational problems relevant to AI [Dunjko & Briegel, 2018]
- Analog computation: Continuous computational models that may offer advantages in certain domains [Kendall et al., 2020]
These alternative computational paradigms represent complementary rather than competing paths forward.
5. Valuation, Safety, and Integration
5.1 Value Alignment as a Technical Challenge
Value alignment represents both a technical and philosophical challenge:
- Value representation: Formalization of human values in a way that is computable
- Value robustness: Ensuring that values are preserved during development and distribution
- Value pluralism: Handling variations in values between cultures and individuals
5.2 Integrative Systems
Future AGI systems will likely be:
- Hybrids: Combinations of neural, symbolic, and probabilistic components
- Distributed: Interconnected specialized subsystems rather than monolithic architectures
- Adaptive: Capable of continuous self-modification and adaptation to new environments
- Collaborative: Designed for complementary collaboration with humans rather than complete autonomy
6. Conclusion: Continuum or Threshold?
6.1 Competing Perspectives on AGI Development
A central question for the field is whether AGI represents:
Continuum perspective: AGI develops as a gradual increase in capabilities along several dimensions, without sharp transitions. This perspective sees today's systems as early, limited forms of general intelligence.
Threshold perspective: AGI requires qualitative breakthroughs in architecture or mechanisms that enable fundamentally new capabilities. According to this view, there are "cognitive bottlenecks" that cannot be overcome through quantitative improvements.
Empirical evidence provides some support for both perspectives:
- Emergent capabilities in scaling support the continuum perspective
- Persistent limitations despite massive scaling support the threshold perspective
This debate is not merely theoretical but has practical implications for research priorities, investments, and safety discussions.
6.2 A Path Forward
Regardless of which perspective proves correct, what is required is:
- Empirical rigor: Testing specific hypotheses with clear falsification conditions
- Architectural innovation: Integration of complementary computational paradigms
- Domain-shifting benchmarks: Evaluation of generalization outside training domains
- Continuous revision: Willingness to abandon theoretical assumptions when empirical evidence demands it
The real limitations of today's AI systems are substantial, but our understanding of their permanence is still incomplete. By focusing on concrete, testable hypotheses, we can make substantial progress without getting stuck in unproductive debates about what is "possible" or "impossible."
Progress will likely come through integration of complementary approaches rather than through a one-sided focus on a single paradigm, whether development follows a continuum or requires revolutionary breakthroughs.
Read original on Substack ↗