Space Foundation Model (SpaceFM)

Foundation models (FMs), which are machine learning models that are trained on large amounts of general data, have the potential to revolutionize the operation, decision-making, and mission design of distributed space systems. Their strengths in high-level reasoning, adaptability to anomalous behaviors, and their ability to leverage multi-modal data sources, such as vision, language, and numeric, make FMs a powerful tool for tasks ranging from image classification to code curation. Leveraging the full potential of FMs for the application of distributed space systems relies on large amounts of data and environments in which they may operate, explore, and design. The Space Rendezvous Laboratory is particularly poised to provide this data and these environments given its extensive work in developing and designing image data sets (SPEED, SPEED+, SHIRT), high-fidelity simulation and GN&C frameworks (S3, RPOKit), and several space missions (Starling, SWARM-Ex, VISORS). The main challenges in adapting FMs for space-based tasks involve their formal verification, runtime operation, and sparsity of problem-specific space data. Overcoming these challenges is a multi-disciplinary effort that involves adapting state-of-the-art techniques in machine learning and artificial intelligence, development of large-scale, highly realistic simulation frameworks, and the integration of reliable and heritage algorithms for safe use.

Related Publications

Foutter, M., Gammelli, D., Kruger, J., Foss, E., Bhoj, P., Guffanti, T., D'Amico, S., Pavone, M.;
Space-LLaVA: a Vision-Language Model Adapted to Extraterrestrial Applications;
IEEE Aerospace Conference 2025, Big Sky, MT, 2025