Llama 3 70B TESTED: FAILS Logical Reasoning
code_your_own_AI code_your_own_AI
32.8K subscribers
1,659 views
0

 Published On Apr 19, 2024

Llama 3 70B Instruct: First logical and causal reasoning test of the new Llama 3 70B Instruct by Meta - with a minimal context length of 8K on a mono-modal LLM. Causal mathematical reasoning is important also within 8K token length, therefore two simple tests on video from my test suite for LLMs.

Please note this is just a snapshot of my personal testing suite for causal and logical reasoning. It is not a representative test for anybody else and has no statistical significant relevance, it provides me with answers to my standardized real-world test tasks, that I use for all my LLM testing (based on Stanford Univ, see video    • LLM (w/ RAG) need a new Logic Layer (...   ).

You have to test this LLM on your specific domain for your specific tasks for a relevant answer for your business. If you use LLMs just for poetry, or cooking or writing e-mails, your results may vary significantly.

Thanks for HuggingFace for providing the compute infrastructure.

#airesearch
#newtechnology

show more

Share/Embed