Language agents are emerging AI systems that use large language models (LLMs) to interact with the world. While various methods and demos have been developed, it is often hard to systematically understand or evaluate them.
For both humans and machines, the essence of learning is to pinpoint which components in its information processing pipeline are responsible for an error in its output – a challenge that is known as credit assignment. It has long been assumed that credit assignment is best solved by backpropagation, which is also the foundation of modern machine learning.
Symmetries play a paramount role in the mathematical sciences. For example, they describe the crystal structure of atoms in materials, and understanding symmetries often goes a long way towards the solution of a problem.
Attention-based neural network sequence models such as transformers have the capacity to act as supervised learning algorithms: They can take as input a sequence of labeled examples and output predictions for unlabeled test examples.
Battery companies want to know the relationship between their manufacturing parameters and the performance of the resulting cells, so that they can optimise their products for particular applications, reduce costs, and improve yield. The literature contains many examples of physics-based models of the various manufacturing processes (including mixing, coating, drying and calendaring), but these systems are hugely complex, and as a result they are expensive to simulate and hard to validate.
Post the release of Meta’s Llama weights, the open source development of large language models (LLMs) are seeing rapid progress almost every day. This talk will share our experience with serving and evaluating 20+ LLM-based Chatbots, including Vicuna, within the Chatbot Arena I will start by briefly explaining Vicuna, an open source chatbot we finetuned from Llama, and the Chatbot Arena platform we developed to evaluate the quality of such models in the wild.
