Within the space engineering field there is an ever-growing demand for more efficient and cost-effective ways to design, build, and operate spacecraft. Generative AI, specifically large language models (LLMs) like GPT-3, have shown the potential to address these challenges by automating design processes, freeing human expertise to spend time on value-added tasks, and improving the overall performance of space systems.
In the past two years, generative AI models have seen an explosion in popularity with many models being published. Text-to-Text models such as ChatGPT3, LaMDA, and PEER as well as Text-to-Code models such as Codex and Alphacode have demonstrated abilities to propose high quality solutions to technical tasks.
In simple terms, LLMs are machine learning models that can understand and generate text. They take as an input some text (a paragraph, lines of code, etc.) and predict what text is most likely to come next, based on the patterns it has learned from its training data.
The most successful LLM of late utilise a self-attention mechanism. Self-attention is a mechanism that allows a model to weigh the importance of different parts of the input when making predictions. It works by looking at each word in the input and determining how much attention it should pay to each other word when processing the input. This allows the model to focus on the most relevant information and ignore irrelevant information when making its predictions. To be applied to a particular task base LLMs are often fine-tuned on a specific dataset.
Some research has already been conducted looking at applying transformer language models to space engineering:
The innovation that arguably led to the success of ChatGPT is the use of Reinforcement Learning from Human Feedback (RLHF). With RLHF, the original language model is fine tuned with a separate reward model which was trained using feedback from human labellers.
Figure 1: RLHF Process Diagram (Source: Hugging Face)
One use case where LLM could add significant value is within Model Based Systems Engineering (MBSE) processes , why MBSE is the key. LLMs offer a potential avenue to reduce both the manual work involved in modelling complex systems and reduce the risk or modelling error. Such applications include:
Of course, issues remain with these models. LLMs have been shown to generate false information or hallucinate. Ultimately, current models cannot be trusted to generate text that is always correct, and a human verification and validation step is essential to safe applications of these models.