Hello Everyone...Welcome to my blog of MINERVA-A breakthrough for math...There are so many articles and blogs about MINERVA LANGUAGE MODEL then why you need to come here. Well.. here is the place for you to learn about MINERVA in a simple manner as I tried to make you understand in an easy manner with less use of any creepy technical terms.Let's just dive in...
WHAT IS MINERVA :
Google developed their new model named " MINERVA" ,an AI that can handle complex computation of maths and physics. It is based on their other language model named pathways language model which is also known as PaLM . This came into existence in April 2022. PaLM model can perform various natural language processing tasks like
-Generating code from text.
-answering math word problems.
-explaining a joke.
-common sense reasoning.
-translations.
WHY MINERVA:
QUANTITATIVE REASONING is one such area in which all language models still fall short of human level performances. This solving mathematical problems and solving scientific questions requires a combination of skills. For eg, if you want to solve any math or scientific question, you need to understand the question first, then we think of a relevant formulae which are required for the problem and we generate step by step solutions to solve that problem. These kind of combination of skills are required when you want to work on quantitative reasoning,
Minerva which is built on paLM is capable of solving mathematical and scientific problems using step by step procedure. They are transformer based models in which they accept inputs as tokens(roughly words/symbols/numbers) and generate output tokens based on the previous tokens.
Let me show you one problem which Minerva solved.
 |
This above question needs a multi -step solution and the model write down the question, simplified it, substitute a variable and solved it. These math question is from Joint Entrance Examination taken each year by almost 2 million Indian high school Student. As you can see Minerva has done it. To the note, MINERVA doesn't use calculator or python interpreter.
Now the questions is how minerva is able to do such a complex mathematical and scientific task. The answer is, AI scientists has further trained their paLM model with 118 GB datasets of scientific papers taken from
ArXiV preprint server
Webpages (that contains natural language and mathematical expressions).
They have done text cleaning in a very careful manner unlike standard text cleaning.
In Standard text cleaning:
In Minerva text cleaning :
Minerva do Careful Data processing which preserves all mathematical symbols that adds moral to the question and makes the question reasonable.
ACCURACY OF MINERVA:
Minerva model is evaluated based on STEM benchmark where STEM stands for Science, Technology, Engineering and Mathematics. They evaluate accuracy based on the overall performance on the science and math. Here it's is ranging in difficulty from grade school level problems to graduate level coursework.
 |
In all cases, Minerva obtains state-of-the-art results, sometimes by a wide margin.
MATH: High school math competition level problems
MMLU-STEM: A subset of the Massive Multitask Language Understanding benchmark focused on STEM, covering topics such as engineering, chemistry, math, and physics at high school and college level.
They also evaluated Minerva on OCWCourses, a collection of college and graduate level problems covering a variety of STEM topics such as solid state chemistry, astronomy, differential equations, and special relativity that they collected from MIT OpenCourseWare.
As you can see ,It gives 50.83% accuracy in Math which was previously 6.9%.
The professional forecasters predicted these % results for 2025. Now you can see they achieved those 50% in 2022 itself. Google achieved that accuracy three years prior. This is amazing right?
TECHNIQUES USED IN MINERVA:
Few shot prompting: (ability to learn task with limited example.)
Fine tuning on target domain: (tuning PaLM model with 118 GB of datasets as we said earlier in this blog)
Chain of thought prompting: Model is prompted to produce intermediate results.(i.e, several questions with step by step solutions are prompted to Minerva before asking a new question).NOTE: PaLM is also trained with this inference technique.
Majority voting : It generates different possible solutions and assign probability to each of them and give the solution which is having high probability.
MINERVA OVER GPT:
You can ask me what is the use of minerva when chat GPT-3 is playing good. Well,, the answer is GPT is good at natural languages, coding etc.. but still lagging in quantitative reasoning skills where minerva take advantage of. Moreover both are having different architecture and different training data sets.
Below is one math question that is answered by GPT-3.
This is how MINERVA answered for that same question:
As you can see the question on your right hand side, MINERVA solved it like how we exactly need it. It is like how a math teacher will solve .
LIMITATION:
Don't close your math book and don't take your mobile thinking all your math problem is resolved. wait... there are some limitations that comes along with minerva.
Google has pointed out that the model is not good in formal mathematics.
To better identify areas where the model can be improved, they analyzed a sample of questions the model gets wrong, and found that most mistakes are easily interpretable. About half are calculation mistakes, and the other half are reasoning errors, where the solution steps do not follow a logical chain of thought.
Below is one such example mistake the model makes.
 |
FUTURE WITH MINERVA:
The researchers hope their model can develop quantitative logic to help experimenters and scientists learn new openings.
This could change the rules of the quantitative modeling game in the long run.
This article paves the way by helping readers understand the nuances of this incredible sandwich of technology and experimentation from Google brainchild Minerva.
To wrap up...students in future will not need to worry about their math and physics problems. They will have a virtual teacher with them whenever they want....
You can explore Minerva’s output with their interactive sample explorer!
If you want to know more about MINERVA, you can hear it from GUY GUR-ARI, one of the research scientists in Google who made MINERVA:https://www.youtube.com/watch?v=CXrBr0RpGQg&t=310s