How much of that is the fault of colleges? All that shit about requiring science majors to take liberal arts classes or art majors taking calculus to make them “well rounded.” A bachelor’s degree is supposed to be a mark that you’re just all around better educated than someone with a mere high school diploma, to the point that “It doesn’t matter what you major in, just get a degree” is somehow valid advice. But a doctorate is awarded for a significant work of original research; a Ph. D. means you’re the world’s foremost expert in some tiny corner of a sub-discipline, kind of the opposite of being “well rounded.”
You missed what they meant. It means gpt5 is really good at one arbitrary and extremely specific topic. Anything else it’s comparable with a random person on the street.
Reality is the opposite though. GPT5 is expert in a pretty wide amount of trivia. It’s better than the average uneducated person in every subject, but worse than an expert in every subject.
All the AI are wrong at least 1 in 20 tokens, probably more like 3 in 20, to their sample data which is also wrong a considerable amount of the time, so it is an expert in literally nothing and never will be.
That 1 in 20 number comes from the 2020 paper by OpenAI and the 2022 paper correcting it by Deepmind about AI scaling laws where the AI inverse error rate would never reach 94% with infnite training and power.
I do that and I have a BS in mathematics. and in 4th grade I literally used to write “I hate math” at the top of my math homework. as much as primary education systems want it to be, computation speed is not mathematical aptitude. you can memorize multiplication tables up to 20, that’s not gonna help you understand Cantor’s theorem
At one point I had an audio book version of Surely You’re Joking, Mr. Feynman. One of Richard Feynman’s memoirs, he avoids talking about his work for the most part and tells stories in oddly low-level English about the shenanigans he’d get up to in his off hours. The entire book sounds something like this:
“One day I decided to go for a walk. I passed a bar. There was music playing in the bar, and people were dancing. It sounded great. I went inside to look at the girls. Their dancing looked great. I noticed one of the musicians was playing a little drum. I asked if I could try. He let me try the drum. It made a really interesting sound.”
At one point, he was in a bar, and was approached by an abacus salesman, who challenged him to a math race. The abacus easily bested Feynman’s mental math in addition, subtraction, multiplication and division, managed to outpace him in exponents and logarithms, and then it just so happened that as the math problems got harder, it just so happened that Feynman had the answers to the exact problems asked memorized, so it appeared he did them instantly in his head. Like by coincidence they asked the exact problem he’d spent the previous week calculating.
Expertise is about knowing what you don’t know, just as much about what you know. The terms like “PhD level intelligence” are meant to mislead people. LLMs cannot understand simply because they are just statistical parrots. Only fools blindly trust output of LLMs.
There’s a lot of a disagreement to be had here…but AI is a topic where we all pretty much seem to agree.
I’ve experimented with AI (so-called chatbots) and the best that can be said about it is it’s a tool…not even a good tool…for starting a project when you’re stuck. It’s absolutely unreliable for anything approaching a final project. I guess…it’s really great if you enjoy being pissed off that it can both mine valid obscure information, while at the same time lie to your face over and over…but then do a 180 on something that’s true or a lie just because you tell it to.
The only value I got out of AI was it made me laugh out loud once…genuinely. I realized it had created this entire fake universe when it was supposed to be researching something real…and like a crazy person I was scolding it…when it was apologing it told me “I’m sorry…I understand that I’m a bit like a toaster that was advertised as a chef…but really all I do is try to burn your house down”. I thought it was hilarious and appropriate.
I understand that I’m a bit like a toaster that was advertised as a chef…but really all I do is try to burn your house down
This is a beautiful metaphor and I’m pissed that it came out of an LLM.
Either this is a hilarious consequence of the word “toaster” often being followed by “burn your house”, or someone else on the Internet came up with it and the LLM just regurgitated.
I mean…I’m sure it came from something I said to it…all LLMs are good for is being agreeable and reflecting what you want back to you…provided what you ask for is simple.
Am I supposed to be impressed? I have a PhD level intelligence and I am not exactly impressive.
LMAO right.
This PhD = Genius trope needs to end.
How much of that is the fault of colleges? All that shit about requiring science majors to take liberal arts classes or art majors taking calculus to make them “well rounded.” A bachelor’s degree is supposed to be a mark that you’re just all around better educated than someone with a mere high school diploma, to the point that “It doesn’t matter what you major in, just get a degree” is somehow valid advice. But a doctorate is awarded for a significant work of original research; a Ph. D. means you’re the world’s foremost expert in some tiny corner of a sub-discipline, kind of the opposite of being “well rounded.”
You missed what they meant. It means gpt5 is really good at one arbitrary and extremely specific topic. Anything else it’s comparable with a random person on the street.
I’d say we’re actually worse than the average person at everything else. Too much of our brain is allocated to our research.
It does seem like there’s an inverse correlation of general intelligence/common sense and specialized study.
Reality is the opposite though. GPT5 is expert in a pretty wide amount of trivia. It’s better than the average uneducated person in every subject, but worse than an expert in every subject.
And the average person has (usually) no idea if what they’re being fed is even correct. But as long as it sounds correct to someone who has no idea…
Plausible bullshit machines.
That’s my favorite Nine Inch Nails album!
All the AI are wrong at least 1 in 20 tokens, probably more like 3 in 20, to their sample data which is also wrong a considerable amount of the time, so it is an expert in literally nothing and never will be.
That 1 in 20 number comes from the 2020 paper by OpenAI and the 2022 paper correcting it by Deepmind about AI scaling laws where the AI inverse error rate would never reach 94% with infnite training and power.
deleted by creator
GIGO
It can also have good in and garbage out, as I mentioned it will never achieve human input accuracy much less superhuman input.
I still count and do math with my fingers and still fuck it up. I guess they’re just like us. 🥲
I would like to see you count 1.33 + 3.25 on your fingers. 🙂
Hold my beer while I sum up each decimal placement and carry the number over.
Nah, it’ll be more impressive if you do it while holding your beer.
I need full access to my piggies. :(
I do that and I have a BS in mathematics. and in 4th grade I literally used to write “I hate math” at the top of my math homework. as much as primary education systems want it to be, computation speed is not mathematical aptitude. you can memorize multiplication tables up to 20, that’s not gonna help you understand Cantor’s theorem
At one point I had an audio book version of Surely You’re Joking, Mr. Feynman. One of Richard Feynman’s memoirs, he avoids talking about his work for the most part and tells stories in oddly low-level English about the shenanigans he’d get up to in his off hours. The entire book sounds something like this:
“One day I decided to go for a walk. I passed a bar. There was music playing in the bar, and people were dancing. It sounded great. I went inside to look at the girls. Their dancing looked great. I noticed one of the musicians was playing a little drum. I asked if I could try. He let me try the drum. It made a really interesting sound.”
At one point, he was in a bar, and was approached by an abacus salesman, who challenged him to a math race. The abacus easily bested Feynman’s mental math in addition, subtraction, multiplication and division, managed to outpace him in exponents and logarithms, and then it just so happened that as the math problems got harder, it just so happened that Feynman had the answers to the exact problems asked memorized, so it appeared he did them instantly in his head. Like by coincidence they asked the exact problem he’d spent the previous week calculating.
Right? I know more shit than some people on some topics…and less shit than other people on other topics.
Expertise might be what they were going for…but they can’t say that because AI can’t have expertise on anything.
Expertise is about knowing what you don’t know, just as much about what you know. The terms like “PhD level intelligence” are meant to mislead people. LLMs cannot understand simply because they are just statistical parrots. Only fools blindly trust output of LLMs.
There’s a lot of a disagreement to be had here…but AI is a topic where we all pretty much seem to agree.
I’ve experimented with AI (so-called chatbots) and the best that can be said about it is it’s a tool…not even a good tool…for starting a project when you’re stuck. It’s absolutely unreliable for anything approaching a final project. I guess…it’s really great if you enjoy being pissed off that it can both mine valid obscure information, while at the same time lie to your face over and over…but then do a 180 on something that’s true or a lie just because you tell it to.
The only value I got out of AI was it made me laugh out loud once…genuinely. I realized it had created this entire fake universe when it was supposed to be researching something real…and like a crazy person I was scolding it…when it was apologing it told me “I’m sorry…I understand that I’m a bit like a toaster that was advertised as a chef…but really all I do is try to burn your house down”. I thought it was hilarious and appropriate.
This is a beautiful metaphor and I’m pissed that it came out of an LLM. Either this is a hilarious consequence of the word “toaster” often being followed by “burn your house”, or someone else on the Internet came up with it and the LLM just regurgitated.
I mean…I’m sure it came from something I said to it…all LLMs are good for is being agreeable and reflecting what you want back to you…provided what you ask for is simple.