Melanie Mitchell

What's New:

Blog

• AI: A Guide for Thinking Humans

Podcast

• Complexity, Season 2: The Nature of Intelligence

Social Media

• Twitter: @melmitchell1


• Bluesky Social: @melaniemitchell.bsky.social


• LinkedIn: Melanie Mitchell

ConceptARC Dataset

• Download dataset here.


• Paper describing dataset and our experiments.

Science Expert Voices Columns

• The Turing Test and Our Shifting Conceptions of Intelligence Science, August 15, 2024


• Debates on the Nature of Artificial General Intelligence, Science, March 21, 2024.


• AI's Challenge of Understanding the World. Science, November 10, 2023.


• How Do We Know How Smart AI Systems Are? Science, July 13, 2023

Recent Papers

• Imagining and building wise machines: The centrality of AI metacognition arXiv:2411.02478, 2024


• Can large language models generalize analogy solving like people can? arXiv:2411.02348, 2024


• Large language models. In M. C. Frank & A. Majid (Eds.), Open Encyclopedia of Cognitive Science. MIT Press, 2024.


• Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Models. To appear in Proceedings of the Cognitive Science Society Conference, 2024.


• Perspectives on the State and Future of Deep Learning, 2023. arXiv:2312.09323, 2023


• Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks. Proceedings of the LLM-CP Workshop, AAAI-24.


• The ConceptARC benchmark: Evaluating understanding and generalization in the ARC domain. Transactions on Machine Learning Research, 2023.


• Rethink reporting of evaluation results in AI. Science, 2023. Non-paywalled preprint.


• The debate over understanding in AI’s large language models. Proceedings of the National Academy of Sciences, 2023. Non-paywalled preprint.


• Probing the psychology of AI models. Proceedings of the National Academy of Sciences, 2023.

Biographical Sketch:

Melanie Mitchell is Professor at the Santa Fe Institute. Her current research focuses on conceptual abstraction and analogy-making in artificial intelligence systems.

Melanie is the author or editor of six books and numerous scholarly papers in the fields of artificial intelligence, cognitive science, and complex systems. Her 2009 book Complexity: A Guided Tour (Oxford University Press) won the 2010 Phi Beta Kappa Science Book Award, and her 2019 book Artificial Intelligence: A Guide for Thinking Humans (Farrar, Straus, and Giroux) was shortlisted for the 2023 Cosmos Prize for Scientific Writing. Melanie is the recipient of the Senior Scientific Award from the Complex Systems Society, the Distinguished Cognitive Scientist Award from UC Merced, and the Herbert A. Simon Award of the International Conference on Complex Systems.

Melanie originated the Santa Fe Institute's Complexity Explorer platform, which offers online courses and other educational resources related to the field of complex systems. Her online course Introduction to Complexity was named one of Class Central’s "Best Free Online Courses of All Time."

Curriculum Vitae

Books:

Academic Papers:

Mitchell, M. (2024). Large language models. In M. C. Frank & A. Majid (Eds.), Open Encyclopedia of Cognitive Science. MIT Press.

Lewis, M. and Mitchell, M. (2024). Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Models. To appear in Proceedings of the Cognitive Science Society Conference, 2024.

Mitchell, M., Palmarini, A. B., and Moskvichev, A. (2023). Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks. In Proceedings of the LLM-CP Workshop, AAAI-24.

Moskvichev, A., Odouard, V. V., and Mitchell, M. (2023). The ConceptARC benchmark: Evaluating understanding and generalization in the ARC domain. Transactions on Machine Learning Research.

Burnell, R., Schellaert, W., Burden, J., Ullman, T. D., Martinez-Plumed, F., Tenenbaum, J. B., Rutar, D., Cheke, L. G., Sohl-Dickstein, J., Mitchell, M., Kiela, D., Shanahan, M. Voorhees, E. M., Cohn, A. G., Leibo, J. Z., and Hernandez-Orallo, J. (2023). Rethink reporting of evaluation results in AI. Science, 380 (6641), 136-138. Non-paywalled preprint.

Mitchell, M. and Krakauer, D. C. (2023). The debate over understanding in AI's large language models. Proceedings of the National Academy of Sciences, 120 (13).

Shiffren, R. and Mitchell, M. (2023) Probing the psychology of AI models. Proceedings of the National Academy of Sciences, 120 (10).

Odouard, V. V. and Mitchell, M. (2022). Evaluating understanding on conceptual abstraction benchmarks. In Proceedings of the AI Evaluation Beyond Metrics Workshop (IJCAI-2022).

Shanahan, M. and Mitchell, M. (2022). Abstraction for Deep Reinforcement Learning. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-2022).

Springer, J. M., Mitchell, M., and Kenyon, G. T. (2021). A little robustness goes a long way: Leveraging robust features for targeted transfer attacks. In Proceedings of 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

Springer, J. M., Mitchell, M., and Kenyon, G. T. (2021). Uncovering universal features: How adversarial training improves adversarial transferability. Workshop on Adversarial Machine Learning, International Conference on Machine Learning (ICML).

Mitchell, M. (2021). Why AI is Harder Than We Think, arXiv: 2104.12871. To appear in Mind Design 3.

Mitchell, M. (2021). Abstraction and Analogy-Making in Artificial Intelligence. Annals of the New York Academy of Sciences, 1505 (1), 79–101.

Springer, J. M., Mitchell, M., and Kenyon, G. T. (2021). Adversarial perturbations are not so weird: Entanglement of robust and non-robust features in neural network classifiers, arXiv:2102.05110.

Jenkins, O. C., Lopresti, D., and Mitchell, M. (2020). Next Wave Artificial Intelligence: Robust, Explainable, Adaptable, Ethical, and Accountable. CCC Quadrennial Paper. November 2020.

Mitchell, M. (2020). On crashing the barrier of meaning in AI. AI Magazine, 41(2), 86-92.

Conser, E., Hahn, K., Watson, C. M., and Mitchell, M. (2019). Revisiting visual grounding. To appear in Proceedings of the Workshop on Shortcomings in Vision and Language, NAACL-2019, ACL.

Quinn, M. H., Conser, E., Witte, J. M., and Mitchell, M. (2018). Semantic image retrieval via active grounding of visual situations . In Proceedings of the 12th International Conference on Semantic Computing. IEEE.

Lundquist, S. Y., Mitchell, M., and Kenyon, G. T. (2017). Sparse coding on stereo video for object detection. In Proceedings of the NIPS Workshop on Learning with Limited Labeled Data: Weak Supervision and Beyond.

Rhodes, A. D., Witte, J., Mitchell, M., and Jedynak, B. (2017). Bayesian optimization for refining object proposals. In Proceedings of the Seventh International Conference on Image Processing Theory, Tools, and Applications (IPTA 2017). IEEE.

Rhodes, A. D., Quinn, M. H., and Mitchell, M. (2016). Fast on-line kernel density estimation for active object localization. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), 2017.

Quinn, M. H., Rhodes, A. D., and Mitchell, M. (2016). Active object localization in visual situations. arXiv 1607.00548, 2017.

Ghosh, P., Mitchell, M., Tanyi, J. A., and Hung, A. Y. (2015). Incorporating priors for medical image segmentation using a genetic algorithm. To appear in Neurocomputing Journal.

Thomure, M. D., Mitchell, M., and Kenyon, G. T. (2013). On the role of shape prototypes in hierarchical models of vision. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), 2013.

Landecker, W., Thomure, M. D., Bettencourt, L. M. A., Mitchell, M., Kenyon, G. T., and Brumby, S. P. (2013). Interpreting individual classifications of hierarchical networks. In Proceedings of the 2013 Conference on Computational Intelligence and Data Mining (CIDM 2013).

Ghosh, P., Mitchell, M., and Gold, J. (2010). LSGA: Combining level-sets and genetic algorithms for segmentation. Evolutionary Intelligence, 3, 1-11.

Ghosh, P., Mitchell, M., and Gold, J. (2010). Segmentation of thermographic images of hands using a genetic algortithm. In Proceedings of SPIE, Vol. 7538, 75380D (2010).

Ghosh, P., Mitchell, M., Tanyi, J. A., and Hung, A. (2009). A genetic algorithm-based level-set curve evolution for prostate segmentation on pelvic CT and MRI images. E. Romero and F. Gonzalez (editors), Biomedical Image Analysis and Machine Learning Technologies: Applications and Techniques, ICI Global.

Ghosh, P. and Mitchell, M. (2008). Prostate segmentation on pelvic CT images using a genetic algorithm. In Proceedings of the International Society for Optical Engineering (SPIE), Conference on Medical Imaging, February, 2008. SPIE Press.

Marques-Pita, M., Mitchell, M., and Rocha, L. (2008). The role of conceptual structure in designing cellular automata to perform collective computation. In Proceedings of the Conference on Unconventional Computation, UC 2008, Springer (Lecture Notes in Computer Science).

Juengling, R. and Mitchell, M. (2007). Combinatorial shape decomposition. In Proceedings of the Third International Symposium on Visual Computing (ISVC07). Springer (Lecture Notes in Computer Science).

Mitchell, M. (2006) Complex systems: Network thinking. Artificial Intelligence, 170(18), 1194-1212.

Ghosh, P. and Mitchell, M. (2006). Segmentation of medical images using a genetic algorithm. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO-2006, pp. 1171-1178.

Mitchell, M (2006). Coevolutionary learning with spatially distributed populations. In G. Y. Yen and D. B. Fogel (editors), Computational Intelligence: Principles and Practice . New York: IEEE Computational Intelligence Society.

Mitchell, M., Thomure, M. D., and Williams, N. L. (2006). The role of space in the success of coevolutionary learning. In Proceedings of Artificial Life X: Tenth Annual Conference on the Simulation and Synthesis of Living Systems . Cambridge, MA: MIT Press.

Williams, N. L. and Mitchell, M. (2005) Investigating the success of spatial coevolutionary learning. In H. G. Beyer et al. (editors), Proceedings of the 2005 Genetic and Evolutionary Computation Conference, GECCO-2005 , New York: ACM Press, 523-530.

Mitchell, M. (2005). Self-awareness and control in decentralized systems. In Working Papers of the AAAI 2005 Spring Symposium on Metacognition in Computation. Menlo Park, CA: AAAI Press.

Crutchfield, J. P., Mitchell, M., and Das, R. (2003). The evolutionary design of collective computation in cellular automata. In J. P. Crutchfield and P. K. Schuster (editors), Evolutionary Dynamics---Exploring the Interplay of Selection, Neutrality, Accident, and Function, pp. 361--411. New York: Oxford University Press.

Mitchell, M. and Newman, M. (2002). Complex systems theory and evolution. In M. Pagel (editor), Encyclopedia of Evolution , New York: Oxford University Press.

Jimenez-Morales, F., Mitchell, M., and Crutchfield, J. P. (2002). Evolving one dimensional cellular automata to perform a non-trivial collective behavior task: One case study. In P. M. A. Sloot, C. J. K. Tan, J. J. Dongarra and A. G. Hoekstra (editors), Computational Science-ICCS 2002, Part I, Proceedings 2329, 793-802. Berlin: Springer-Verlag.

Pagie, L. and Mitchell, M. (2002). A comparison of evolutionary and coevolutionary search. International Journal of Computational Intelligence and Applications, 2(1), 53--69.

Jimenez-Morales, F., Crutchfield, J. P., and Mitchell, M. (2001). Evolving two-dimensional cellular automata to perform density classification: A report on work in progress. Parallel Computing, 27 (5), 571--585.

Mitchell, M. (2001). Life and evolution in computers. History and Philosophy of the Life Sciences, 23, 361-383.

Mitchell, M. (2001). Analogy-making as a complex adaptive system. In L. Segel and I. Cohen (editors), Design Principles for the Immune System and Other Distributed Autonomous Systems . New York: Oxford University Press.

Werfel, J., Mitchell, M., and Crutchfield, J. P. (2000). Resource sharing and coevolution in evolving cellular automata. IEEE Transactions on Evolutionary Computation, 4(4), 388--393.

Brumby, S. P., Perkins, S. J., Theiler, J., Szymanski, J. J., Bloch, J. J., and Mitchell, M. (1999). Investigation of image feature extraction by a genetic algorithm. In Proceedings of the International Society for Optical Engineering, Proceedings of SPIE 3812, 24-31. Bellingham, WA: SPIE Press.

van Nimwegen, E., Crutchfield, J. P., and Mitchell, M. (1999). Statistical dynamics of the Royal Road genetic algorithm. Theoretical Computer Science, 229 (1), 41-102.

Mitchell, M. (1999). Evolutionary computation. In R. Wilson and F. Keil (editors), The MIT Encyclopedia of the Cognitive Sciences . Cambridge, MA: MIT Press.

Mitchell, M. and Taylor, C. E. (1999) Evolutionary computation: An overview. Annual Review of Ecology and Systematics, 30, 593-616

Mitchell, M. and Forrest, S. (1998). Royal Road functions. In T. Back, D. Fogel, and Z. Michalewicz (editors), Handbook of Evolutionary Computation . Oxford: Oxford University Press.

Hordijk, W., Crutchfield, J. P., and Mitchell, M. (1998). Mechanisms of emergent computation in cellular automata. In A. E. Eiben (ed.), Proceedings of the Fifth International Conference on Parallel Problem Solving From Nature---PPSN V. New York: Springer.

Mitchell, M. (1998). A complex-systems perspective on the ``computation vs. dynamics'' debate in cognitive science. In M. A. Gernsbacher and S. J. Derry (eds.), Proceedings of the 20th Annual Conference of the Cognitive Science Society---Cogsci98, 710-715.

Mitchell, M. (1998). Theories of structure versus theories of change. (Commentary on ``The dynamical hypothesis in cognitive science'', by T. van Gelder.) Behavioral and Brain Sciences, 21, 645-646 .

Mitchell, M. (1998). Computation in cellular automata: A selected review. In T. Gramss, S. Bornholdt, M. Gross, M. Mitchell, and T. Pellizzari, Nonstandard Computation , pp. 95--140. Weinheim: VCH Verlagsgesellschaft.

Mitchell, M., Crutchfield, J. P., and Das, R. (1998). Evolving cellular automata to perform computations. In T. Back, D. Fogel, and Z. Michalewicz (editors), Handbook of Evolutionary Computation. Oxford: Oxford University Press.

van Nimwegen, E., Crutchfield, J. P., and Mitchell, M. (1997). Finite populations induce metastability in evolutionary search. Physics Letters A, 229 (2), 144-150.

Belew, R. K., Mitchell, M., and Ackley, D. H. (1996). Computation and the natural sciences. In R. K. Belew and M. Mitchell (editors), Adaptive Individuals in Evolving Populations: Models and Algorithms. Reading, MA: Addison-Wesley.

Mitchell, M., Crutchfield, J. P., and Das, R. (1996). Evolving cellular automata to perform computations: A review of recent work. In Proceedings of the First International Conference on Evolutionary Computation and its Applications (EvCA '96). Moscow, Russia: Russian Academy of Sciences.

Hordijk, W., Crutchfield, J. P., and Mitchell, M. (1996). Embedded particle computation in evolved cellular automata. In Proceedings of the Conference on Physics and Computation---PhysComp96, Boston, MA.

Crutchfield, J. P. and Mitchell, M. (1995). The evolution of emergent computation. Proceedings of the National Academy of Sciences, USA, 92 (23): 10742.

Das, R., Crutchfield, J. P., Mitchell, M., and Hanson, J. E. (1995). Evolving globally synchronized cellular automata. In L. J. Eshelman (editor), Proceedings of the Sixth International Conference on Genetic Algorithms. San Mateo, CA: Morgan Kaufmann.

Das, R., Mitchell, M., and Crutchfield, J. P. (1994). A genetic algorithm discovers particle-based computation in cellular automata. In Y. Davidor, H.-P. Schwefel, and R. Manner (editors), Parallel Problem Solving from Nature---PPSN III. Berlin: Springer-Verlag.

Mitchell, M., Crutchfield, J. P., and Hraber, P. (1994). Evolving cellular automata to perform computations: Mechanisms and impediments. Physica D, 75, 361--391.

Mitchell, M., Crutchfield, J. P., and Hraber, P. T. (1994). Dynamics, computation, and the ``edge of chaos'': A re-examination. In G. Cowan, D. Pines, and D. Melzner (editors), Complexity: Metaphors, Models, and Reality. Reading, MA: Addison-Wesley.

Mitchell, M., Holland, J. H., and Forrest, S. (1994). When will a genetic algorithm outperform hill climbing? In J. D. Cowan, G. Tesauro, and J. Alspector (editors), Advances in Neural Information Processing Systems 6, 51-58, San Mateo, CA: Morgan Kaufmann.

Mitchell, M. (1995). Genetic algorithms: An overview. Complexity, 1 (1) 31--39.

Hofstadter, D. R. and Mitchell, M. (1995). The The Copycat project: A model of mental fluidity and analogy-making Chapter 5 in D. R. Hofstadter, Fluid Concepts and Creative Analogies. Basic Books.

Mitchell M., and Forrest, S. (1994). Genetic algorithms and artificial life. Artificial Life, 1 (3), 267-289.

Mitchell, M., Hraber, P. T., and Crutchfield, J. P. (1993). Revisiting the edge of chaos: Evolving cellular automata to perform computations. Complex Systems, 7, 89-130.

Forrest, S. and Mitchell, M. (1993). What makes a problem hard for a genetic algorithm? Some anomalous results and their explanation. Machine Learning, 13, 285-319.

Forrest, S. and Mitchell, M. (1993). Relative building-block fitness and the building-block hypothesis. In D. Whitley (editor), Foundations of Genetic Algorithms 2, San Mateo, CA: Morgan Kaufmann.

Mitchell, M. (1993). Genetic algorithms. In L. Nadel and D. L. Stein (editors), 1992 Lectures in Complex Systems. Reading, MA: Addison-Wesley.

Mitchell, M., Forrest, S., and Holland, J. H. (1992). The royal road for genetic algorithms: Fitness landscapes and GA performance. In F. J. Varela and P. Bourgine (editors), Proceedings of the First European Conference on Artificial Life. Cambridge, MA: MIT Press.

Forrest, S. and Mitchell, M. (1991). The performance of genetic algorithms on Walsh polynomials: Some anomalous results and their explanation. In R. Belew and L. Booker (editors), Proceedings of the Fourth International Conference on Genetic Algorithms. San Mateo, CA: Morgan Kaufmann.

Mitchell, M. and Hofstadter, D. R. (1990). The emergence of understanding in a computer model of concepts and analogy-making. Physica D, 42, 322-334.

Mitchell, M. and Hofstadter, D. R. (1990). The right concept at the right time: How concepts emerge as relevant in response to context-dependent pressures. In Proceedings of the Twelfth Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Lawrence Erlbaum Associates.

Mitchell, M. and Hofstadter, D. R. (1989). The role of computational temperature in a computer model of concepts and analogy-making. In Proceedings of the Eleventh Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Lawrence Erlbaum Associates.

Hofstadter, D. R. and Mitchell, M. (1988). Conceptual slippage and analogy-making: A report on the Copycat project. In Proceedings of the Tenth Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Lawrence Erlbaum Associates.

Hofstadter, D. R. and Mitchell, M. (1988). Concepts, analogies, and creativity. In Proceedings of the Canadian Society for Computational Studies of Intelligence. Edmonton: University of Alberta.

Seward, F. D. and Mitchell, M. (1981). An X-ray survey of the Small Magellanic Cloud. Astrophysical Journal, 243, 736.

Mitchell, M. (1979). Period changes in two W Virginis variables. Journal of the American Association of Variable Star Observers, 8 (2).

Essays and Reviews:

Mitchell, M. (2024). The Turing Test and Our Shifting Conceptions of Intelligence Science, August 15, 2024

Mitchell, M. (2024). Debates on the Nature of Artificial General Intelligence, Science, March 21, 2024.

Mitchell, M. (2023). AI's Challenge of Understanding the World, Science, November 10, 2023.

Mitchell, M. (2023). How Do We Know How Smart AI Systems Are?, Science, July 13, 2023.

Mitchell, M. (2022). What Does It Mean to Align AI With Human Values? Quanta, December, 2022.

Mitchell, M. (2021). What Does It Mean for AI to Understand? Quanta, December, 2021.

Flack, J. and Mitchell, M. (2020). Uncertain Times, Aeon, August, 2020.

Mitchell, M. (2020). Can GPT-3 Make Analogies? Medium, August, 2020.

Mitchell, M. (2020). How the Analogies We Live By Shape Our Thoughts, Transmissions, Santa Fe Institute, April, 2020.

Mitchell, M. (2019). Can a Computer Ever Learn to Talk?, OneZero, November, 2019.

Mitchell, M. (2019). We Shouldn’t be Scared by 'Superintelligent A.I.', New York Times, November, 2019.

Mitchell, M. (2019). Blade Runner is set in November 2019, but what does it say about our future?, The Big Issue, November, 2019.

Mitchell, M. (2019). AI Can Pass Standardized Tests --- But It Would Fail Preschool, Wired.

Mitchell, M. (2019). How do you teach a car that a snowman won’t walk across the road?, Aeon.

Mitchell, M. (2018). Artificial Intelligence Hits the Barrier of Meaning, New York Times, November, 2018.

Forrest, S. and Mitchell,M. (2016) Adaptive computation: The multidisciplinary legacy of John H. Holland. Communications of the ACM, 59(8), 58-63.

Mitchell, M. (2014). How Can the Study of Complexity Transform Our Understanding of the World?. In Big Questions Online.

Mitchell, M. (2010) Biological computation. In ACM Ubiquity Symposium on "What is Computation?"

Mitchell, M. (2008) Five questions. In C. Gershenson (editor), Complexity: 5 Questions. Automatic Press

Mitchell, M. (2003). Review of "Conceptual Coordination: How the Mind Orders Experience in Time" by William J. Clancey. Contemporary Psychology, 48 (3).

Mitchell, M. (2002). Review of "A New Kind of Science" by Stephen Wolfram. Science, 298, 65--68.

Mitchell, M. (1999). Can Evolution Explain How the Mind Works? A Review of the Evolutionary Psychology Debates. Complexity , 3 (3), 17-24.

Mitchell, M. (1998). Review of "Handbook of Genetic Algorithms" by Lawrence Davis. Artificial Intelligence, 100 (1-2), 325-330.

Mitchell, M. (1997). Review of "Figments of Reality" by Ian Stewart and Jack Cohen. New Scientist, August 11, 1997.

Mitchell, M. (1997). Review of "Darwin's Dangerous Idea" by Daniel Dennett. Complexity, 2 (1), 32--26.

Mitchell, M. (1995). Review of "Out of Control: The Rise of Neo-Biological Civilization" by Kevin Kelly. Technology Review, October, 1995.

Mitchell, M. (1993). Computer Models of Adaptive Complex Systems. New Scientist, February 13, 1993.

Mitchell, M. (1991). Review of "The Dreams of Reason: The Computer and the Rise of the Sciences of Complexity" by Heinz Pagels. In Bulletin of the Santa Fe Institute, 6 (1).

Mitchell, M. (1985). Artificial Intelligence and the Popular Press. Popular Computing, January, 1985.

Online Courses:

Current Online Courses

• Introduction to Complexity, on Complexity Explorer, Santa Fe Institute. Free and accessible to all. Always available.

Speaking:

Upcoming Talks

November 14-16: Bloomington Symposium on Intelligence, Indiana University, Bloomington, IN (in person)

November 22, 2024: Annual Lecture Series, Center for Philosophy of Science at the University of Pittsburgh (in person)

December 14, 2024: Pluralistic Alignment, NeurIPS Workshop, Vancouver, BC, Canada (in person)

December 15, 2024: System 2 Reasoning At Scale, NeurIPS Workshop, Vancouver, BC, Canada (in person)

March 28, 2025: Psychology Department, Princeton University, Princeton, NJ (in person)

April 1, 2025: Engineering Department, Brown University, Providence, RI (in person)

April 10-13, 2025: Virgin Islands Literary Festival, St. Croix, US Virgin Islands (in person)

May 7, 2025: NASA Goddard Space Flight Center (online)

May 12, 2025: Science and Cocktails Copenhagen, Copenhagen, Denmark

May 15, 2025: DIEP Workshop on General AI, University of Amsterdam, the Netherlands

May 16, 2025: Workshop on AI and Logical Reasoning, University of Amsterdam, the Netherlands

May 20, 2025: Science and Cocktails Amsterdam, Amsterdam, the Netherlands

May 28, 2025: Johns Hopkins Mathematical Institute for Data Science, Johns Hopkins University, Baltimore, MD (in person)

May 29-31, 2025: Natural Philosophy Symposium, Johns Hopkins University, Baltimore, MD (in person)

Recent Talks (2023-2024)

Panel discussion on AI. Workshop on Language, Mind, and Ethics in AI, University of Texas, Austin, TX, October 30, 2024 (in person)

"AI's Challenge of Understanding the World". Spotlight Seminars on AI, Italian Association for Artificial Intelligence, October 25, 2024 (online)

"AI's Challenge of Understanding the World". Center for Accelerating Operational Efficiency, Arizona State University, October 23, 2024 (online)

"AI's Challenge of Understanding the World". Philosophy SIG Annual Conference, Royal College of Psychiatrists, October 2, 2024 (online)

"AI's Challenge of Understanding the World". EADM Summer School on Large Language Models in Behavioural Science, Max Planck Institute for Human Development, September 12, 2024 (online)

September 18, 2024: Panel Discussion, A Complex Universe: Diverse Perspectives, Santa Fe Institute Public Event, Santa Fe, NM (in person)

"The Debate Over Understanding in AI's Large Language Models". SFI CounterBalance Seminar, September 10, 2024 (online)

"AI's Challenge of Understanding the World". Robophilosophy 2024, August 22, 2024 (online)

"Evaluating the Robustness of LLMs on Abstract Reasoning and Analogy". ICML Workshop on Large Language Models and Cognition, July 27, 2024 (online)

"AI's Challenge of Understanding the World". Diverse Intelligences Summer Institute, St. Andrews University, St. Andrews, Scotland, July 8, 2024 (online)

"AI's Challenge of Understanding the World". SFI Complex Systems Summer School, Santa Fe, NM, June 17, 2024 (in person)

"AI's Challenge of Understanding the World". SFI ACtioN meeting, Austin, TX, June 13, 2024 (in person)

"AI's Challenge of Understanding the World". Large Language Models: Science and Stakes, Summer Institute, Université du Québec à Montréal, June 6, 2024 (online)

"The Past, Present, and Uncertain Futures of AI". Los Jardineros de Placitas, Placitas Community Library, June 5, 2024 (in person)

"The Past, Present, and Uncertain Futures of AI". Southwest Seminars, Santa Fe, NM, June 3, 2024 (in person)

"Can AI Understand the World?" Center for Literary Studies, Berlin, Germany, May 30, 2024 (online)

"The Debate Over Understanding in AI's Large Language Models". Brain Science and Large Language Models, Leopoldina and Max Planck Institute for Brain Research, Frankfurt, Germany, May 13, 2024 (online)

"The Past, Present, and Uncertain Future of Artificial Intelligence." Symposium on the New Wave of AI in Healthcare, New York Academy of Sciences and Icahn School of Medicine at Mount Sinai, New York, NY, May 2, 2024: (in person)

"The Debate over Understanding in AI's Large Language Models." Distinguished Speaker Series, IBM Research, Yorktown Heights, NY, April 30, 2024: (in person)

"The Debate Over Understanding in AI's Large Language Models". Ukrainian IT Society, April 24, 2024 (online)

"The Debate Over Understanding in AI's Large Language Models". Center for Brains, Minds, and Machines, MIT, Cambridge, MA, April 2, 2024 (in person)

"Can AI Understand the World?" Weinberg Symposium, University of Michigan, Ann Arbor, MI, March 29, 2024 (in person)

"The Past, Present, and Uncertain Future of Artificial Intelligence". Dean's Distinguished Speaker Series, UC Davis, Davis, CA, March 7, 2024 (in person)

"Can AI Understand the World?" AAAS Annual Meeting, Denver, CO, February 17, 2024 (in person)

"The Debate Over Understanding in AI's Large Language Models", Mitsubishi Electric Research Labs, February 13, 2024 (online)

"The Debate Over Understanding in AI's Large Language Models". Workshop in Law, Philosophy, and Political Theory, UC Berkeley, Berkeley, CA, February 2, 2024 (in person)

"The Future of Artificial Intelligence". Distinctive Voices, National Academies of Science, Engineering, and Medicine, Irvine, CA, January 31, 2024 (in person)

"Abstraction and Analogy are the Keys to Robust, Open-Ended AI". Workshop on Agent Learning in Open-Endedness, NeurIPS 2023, December 15, 2023 (online)

"Adaptive Computation: Past, Present, and Future". Workshop on Emergent Computation: From Viruses and Immune Systems to Software and Security, Santa Fe Institute, Santa Fe, NM, November 29, 2023 (in person)

"The Past, Present, and Uncertain Futures of AI". CIO Forum, Bank of International Settlements, November 29, 2023 (online)

"The Future of Artificial Intelligence". SFI Community Lecture, Santa Fe, NM, November 15, 2023 (in person)

"The Past, Present, and Uncertain Futures of AI". Procter & Gamble AI Community of Practice, November 15, 2023 (online)

National Academy of Sciences Panel on Generative AI and the Implications for Science Communication. November 7, 2023 (online)

"The Debate Over Understanding in AI's Large Language Models". Computational Social Science Society Conference, Santa Fe, NM, November 3, 2023 (in person)

"The Debate Over Understanding in AI's Large Language Models". Complex Systems Society Conference, October 20, 2023 (online)

"The Debate Over Understanding in AI's Large Language Models". Cognitive Informatics Seminar, University of Quebec, October 19, 2023 (online)

"The Debate Over Understanding in AI's Large Language Models". LANL P/T and AI Forum Colloquium, Los Alamos, NM, October 19, 2023 (in person)

"The Debate Over Understanding in AI's Large Language Models". Complexity Interactive, Santa Fe Institute, October 17, 2023 (online)

SFI ACtioN Risk Meeting, New York City, NY, October 12, 2023 (in person)

Premio Cosmos Finalist Lecture, Reggio Calabria, Italy, October 6, 2023 (in person)

Being Human series, Sarah Lawrence College, Bronxville, NY, October 3, 2023 (in person)

"The Debate over Understanding in AI's Large Language Models". Institute of Neuroinformatics, UZH and ETH, Zurich, September 22, 2023. (online)

Computer Science Department, University of New Mexico, Albuquerque, NM, September 20, 2023 (in person)

"The Debate over Understanding in AI's Large Language Models". Cybernetic Seminar, Max Planck Institutes, Tuebingen, Germany, September 15, 2023 (online)

"Abstraction and Analogy: The Keys to Robust Artificial Intelligence." Margaret Boden Lecture, Leverhulme Centre for the Future of Intelligence, University of Cambridge, UK, August 18, 2023 (in person)

SFI International Summer Institute on Intelligence and Representation, University of Cambridge, UK, August 14, 15, 2023 (in person)

"The Debate Over Understanding in AI's Large Language Models", Microsoft Research, New York City, NY, August 1, 2023 (online)

"Concepts, Abstraction, and Understanding in Artificial Intelligence", Complex Systems Summer School, Santa Fe Institute, Santa Fe, NM, June 28, 2023 (in person)

"The Meaning of Intelligence and Understanding in AI", Collective Intelligence: Foundations + Radical Ideas, Santa Fe, NM, June 21, 2023: (in person)

Round table on AI and Higher Education: Myth and Reality", Texas A&M University, Commerce, June 15, 2023 (online)

"The Debate Over Understanding in AI's Large Language Models", Soar Workshop, University of Michigan, June 14, 2023 (online)

"Abstraction and Analogy are the Keys to Robust AI", Generalization in Mind and Machine seminar, University of Bristol, UK, April 6, 2023 (online)

"Abstraction and Analogy are the Keys to Robust AI", Cognitive Forum, UCLA, March 17, 2023 (online)

"Conceptual Abstraction and Analogy in Humans and AI Systems", CIFAR Workshop on Consciousness and AI, Mila, Montreal, March 8, 2023 (in person)

"AI Beyond Deep Learning", AAAS Meeting, Washington, DC, March 5, 2023 (in person)

The Debate Over Intelligence (and Stupidity) in AI's Large Language Models, SFI ACtioN meeting, February 28, 2023 (online)

Students:

Current Research Interns

Alessandro Palmarini, Santa Fe Institute

Former Postdocs

Arseny Moskvichev, AMD

Tyler Millhouse, University of Arizona

Efsun Sarioglu Kayi, Johns Hopkins Applied Physics Lab

Ludo Pagie, Annogen B.V.

Manuel Marques-Pita , Instituto Universitário de Lisboa

Former Ph.D. Students

Max Quinn (Ph.D. 2021, Portland State University), Machine Learning Scientist, Cambia Health Solutions. Dissertation

Sheng Lundquist (Ph.D. 2020, Portland State University), Senior Research Engineer, DELV. Dissertation

Anthony Rhodes (Ph.D. 2020, Portland State University), Senior Research Scientist, Intel Corp. Dissertation

Will Landecker (Ph.D. 2014, Portland State Univeristy), Owner, Accountable Algorithm. Dissertation

Ralf Juengling (Ph.D. 2013, Portland State University), Senior Software Engineer, HeartFlow Inc. Dissertation

Mick Thomure (Ph.D. 2013, Portland State University), Senior AI Engineer, Sony AI. Dissertation

Martin Cenek (Ph.D. 2011, Portland State University), Associate Professor, University of Portland. Dissertation

Payel Ghosh (Ph.D. 2010, Portland State University), Answer Product Lead, VisuRay. Dissertation

Wim Hordijk (Ph.D. 1999, University of New Mexico), Consultant. Dissertation

Rajarshi Das (Ph.D. 1996, Colorado State University), Co-Founder & Chief Scientific Officer, MQube Cognition. Dissertation

Former M.S. (Thesis) Students

Sharad Kumar (M.S., Portland State University): Thesis

Andrea Cleland (M.S., Portland State University): Thesis

Erik Conser (M.S., Portland State University): Thesis

Nome Dickerson (M.S., Portland State University): Thesis

James Lewis (M.S., Portland State University): Thesis

Callista Bee (M.S., Portland State University): Thesis

Clinton Olson (M.S., Portland State University): Thesis

Joanna Solmon (M.S., Portland State University): Thesis

George Dittmar (M.S., Portland State University): Thesis

Karan Sharma (M.S., Portland State University): Thesis

Davy Stevenson (M.S., Portland State University): Thesis

Lanfranco Muzi (M.S., Portland State University): Thesis

Dan Coates (M.S., Portland State Univeristy): Thesis

Nathan Williams (M.S., Oregon Graduate Institute): Thesis

Former Undergraduate and Post-Bac Interns

Victor Odouard, Santa Fe Institute (2022-2023)

Ky-Vinh Mai, UC Irvine, 2023

Andrew Geyko, University of New Mexico, 2023

Yutaro Shimizu, Minerva University, 2022

Daniel Cotayo, Florida International University, 2022

Lee Beckwith, Scripps College, 2020

Kennedy Hahn, Portland State University, 2018-2019

Katherine Watson, Stanford University, 2018

Evan Roche, Lewis & Clark College, 2016

Rory Soiffer, University of Washington, 2016

Favian Rahman, Carnegie Mellon University, 2014

Max Boddy, Reed College, 2013

Eben Wood, Portland State University, 2012-2013

Max Orhai, Portland State University, 2012-2013

Jennifer Meneghin, Portland State University, 2006

Michael Thomure, Portland State University, 2004–2005

Jonathan Carlson, Dartmouth College, 2003

Justin Werfel, Princeton University, 1998

Alex Wo, Harvard University, 1997

Elizabeth Ayer, Duke University, 1995

Adam Messinger, Willamette University, 1994

Peter Hraber, Santa Fe Institute, 1993

Former High School Interns

Shrey Poshiya, Santa Fe Prep, Santa Fe, NM 2021

Joaquin Bas, Santa Fe Prep, Santa Fe, NM 2020

Robin Tan, Jesuit High School, Portland, OR, 2016

Bryan Lee, Westview High School, Beaverton, OR, 2016

Katherine Watson, Oregon Episcopal School, Portland, OR 2014-2015

Vicki Niu, Lincoln High School, Portland, OR, 2013

Preetha Velu, Jesuit High School, Portland, OR, 2013

Explorations:

• ConceptARC Dataset
  • Download dataset here.
  • Paper describing dataset and our experiments.


• Current and upcoming free courses and tutorials from Complexity Explorer


• AI/ML fellowship/residency programs at companies and labs:
  
  • Hugging Face AI Residency
  • Salesforce AI Residency
  • OpenAI Residency
  • Allen Institute Young Investigators
  • Google Brain AI Residency
  • Facebook AI Residency
  • Microsoft AI Residency
  • Nvidia AI Research Residency
  • Intel AI Research Residency
  • IBM AI Residency
  • Apple AI/ML Residency
  
  
• Advice for people who want to get into the field of complex systems.


• Sample Copycat letter-string analogy problems


• Copycat project source code


• Jim Marshall's Metacat project source code


• Source code for "Robby the Robot" genetic algorithm from Complexity: A Guided Tour


• Harry Foundalis' index of Bongard problems.

Contact:

Santa Fe Institute
1399 Hyde Park Road
Santa Fe, NM 87501

Email: mm@santafe.edu