Tsinghua & Zhipu.AI
https://github.com/THUDM/ChatGLM-6B
ChatGLM-6B is a lightweight conversational version of the GLM family that has been specially optimized for Chinese contexts has been specially optimized for Chinese contexts. ChatGLM-6B also applies quantization so it can
Zero-shot
Here is a detailed breakdown of the results of this model across 52 different subjects,
presented in sortable tables and bar graphs.
Among them, STEM contains 20 subjects, social sciences contains
10
subjects, humanities also contains 11 subjects, and other
contains
11 subjects.
Sortable table:
| Super Category | Subject | Accuracy |
| STEM | Computer Network | 35.7 |
| STEM | Operating System | 40.8 |
| STEM | Computer Architecture | 35.8 |
| STEM | College Programming | 28.9 |
| STEM | College Physics | 30.7 |
| STEM | College Chemistry | 29.9 |
| STEM | Advanced Mathematics | 30.1 |
| STEM | Probability and Statistics | 27.1 |
| STEM | Discrete Mathematics | 24.2 |
| STEM | Electrical Engineer | 28 |
| STEM | Metrology Engineer | 44.8 |
| STEM | High School Mathematics | 31.9 |
| STEM | High School Physics | 31.4 |
| STEM | High School Chemistry | 28.5 |
| STEM | High School Biology | 35.4 |
| STEM | Middle School Mathematics | 26 |
| STEM | Middle School Biology | 43.2 |
| STEM | Middle School Physics | 37.6 |
| STEM | Middle School Chemistry | 40.5 |
| STEM | Veterinary Medicine | 35.2 |
| Social Science | College Economics | 32.6 |
| Social Science | Business Administration | 37.2 |
| Social Science | Marxism | 52 |
| Social Science | Mao Zedong Thought | 53.4 |
| Social Science | Education Science | 43.7 |
| Social Science | Teacher Qualification | 61.9 |
| Social Science | High School Politics | 48.9 |
| Social Science | High School Geography | 43.3 |
| Social Science | Middle School Politics | 68.9 |
| Social Science | Middle School Geography | 41.7 |
| Humanities | Modern Chinese History | 49.1 |
| Humanities | Ideological and Moral Cultivation | 59.3 |
| Humanities | Logic | 31.9 |
| Humanities | Law | 28.5 |
| Humanities | Chinese Language and Literature | 40.2 |
| Humanities | Art Studies | 38.6 |
| Humanities | Professional Tour Guide | 45.9 |
| Humanities | Legal Professional | 27.9 |
| Humanities | High School Chinese | 29.2 |
| Humanities | High School History | 43.4 |
| Humanities | Middle School History | 60.4 |
| Other | Civil Servant | 36.6 |
| Other | Sports Science | 37.8 |
| Other | Plant Protection | 45.7 |
| Other | Basic Medicine | 39.4 |
| Other | Clinical Medicine | 37 |
| Other | Urban and Rural Planner | 39.7 |
| Other | Accountant | 38.4 |
| Other | Fire Engineer | 33.3 |
| Other | Environmental Impact Assessment Engineer | 40.9 |
| Other | Tax Accountant | 29.1 |
| Other | Physician | 39.5 |
Bar graph:
