To be fair, though, I must add that the LLM even mentioned the usual caveats here. But these may not have been understood by the person:
“requires rigorous mathematical formulation and testing - currently speculative without such validation”
“a tall order needing extensive proof”
“a hypothesis awaiting the hard test of evidence”
5/5
After all, what could possibly go wrong when non-physicists pour their “physics buzzword bingo” with questions that sound like physics (but are not) into an LLM that ultimately doesn't understand physics either?
If the LLM then says, among other things, “it does seem almost too logical and elegant to be completely wrong”, this is of course problematic. Especially if the person at that point decides not to listen to any experts at all…
4/
My objection that the central definition underlyinig the “theory” is not physical and ambiguous was completely ignored.
Worse still: because I was perceived to be to “stupid” has led the person to conclude that they will only discuss this with AIs in the future because “it obviously doesn't make sense (anymore) to discuss it with physicists.”
That's probably a problem.
3/
Often they have become so bogged down in their pseudo theories that it is almost impossible to have a meaningful discussion with them.
Sometimes you can try and present a well-reasoned objection. Whether the other side will get it, is a different question.
Recently, I had the case that the person had “discussed” their theory with an AI / LLM beforehand and, due to “praise” from the language model, was convinced that they *could not be wrong*.
2/
Today, I realized another risk of AI that I hadn't considered before, namely as a Dunning-Kruger amplifier.
At the institute, I regularly have contact with people (*) who think they have refuted the theory of relativity or solved other big physical problems.
Usually, these “solutions” are simply meaningless sentences that sound like physics, but are not physics.
1/
(*) So far only men.
Daily report 
Here are today's most popular trending hashtags 
on our website 
️:
#science, #opensource, #astronomy, #ai, #llm, #space, #innovation
Stay tuned!
I don't understand people who say they'll pay for a #searchengine. Not only is all that public data and knowledge, but they pretend like #SearXNG don't exist - which basically includes indexing from many search engines.
In some cases, it's just another #LLM and selling of tokens. Fine - pay per process is fine. But let's not pretend that these LLMs have anything more special than the aforementioned public data at their disposal, and let's not pretend that this algorithm can't also be skewed.
Been learning #Vue today using #Gemini as a way of doing it. I understand perfectly well both #HTML and #CSS, as well as some #javascript, because I've been doing web dev since some of you hadn't even left your dads balls.
I could sit and structure all the boilerplate manually... but then I write a prompt that does exactly the same thing. I still need the necessary knowledge to deal with the underlying problems although.
#LLM programming is just the new #nvim macro.
Don't @ me bro.
Had a very insightful conversation about the limitations on AI with a marketing copywriter.
Her comment was that actually writing marketing materials is a small part of her job.
If it was just about writing something that persuades a customer to buy a product, it would be a cakewalk.
What takes time is the stakeholder management.
It's navigating conflicting and contradictory demands of different departments.
Legal wants to say one thing. Sales something different. Legal something else entirely.
There's higher-up managers who need their egos soothed.
There's different managers with different views about what the customers want and what their needs are.
And there's a big difference in big bureaucratic organisations between writing marketing collateral, and writing something that gets signed off by everyone who needs to.
She's tried using AI for some tasks, and what that typically involves is getting multiple AI responses, and splicing them together into a cohesive whole.
Because it turns out there's a big difference in the real world between generating a statistically probable output, and having the emotional intelligence to navigate humans.
@thomholwerda Unfortunately @duckduckgo also included an #AI #LLM bot option in their search recently. But at least they aren't pretending to save our planet. #climatecrisis
The current chatbot religion is the distilled wish magic of every college student (especially STEM majors) who failed humanities courses (especially philosophy).
The Amazon AI summary of "Mein Kampf" is such a shining example of how LLM tech excels at appearing intelligent without actually providing any valuable insight.
“Customers find the book easy to read and interesting. They appreciate the insightful and intelligent rants. The print looks nice and is plain. Readers describe the book as a true work of art. However, some find the content boring and grim. Opinions vary on the suspenseful content, historical accuracy, and value for money.”
Det tok meg litt tid å skjønne at kopirot ikke var et ordspill men et oppriktig forsøk på å uttale Copilot med japansk aksent.
Treffende, da.
"This infection of Western chatbots was foreshadowed in a talk American fugitive turned Moscow based propagandist John Mark Dougan gave in Moscow last January at a conference of Russian officials, when he told them, “By pushing these Russian narratives from the Russian perspective, we can actually change worldwide AI.”
A NewsGuard audit has found that the leading AI chatbots repeated false narratives laundered by the Pravda network 33 percent of the time — validating Dougan’s promise of a powerful new distribution channel for Kremlin disinformation."
https://www.newsguardrealitycheck.com/p/a-well-funded-moscow-based-global #ai #llm #llms
@UlrikeHahn Ha this is a fascinating thought. So using an #LLM for #search is more like asking another person about a fact they might or might not remember and than attempt to reproduce, rather than actually looking that fact up. What can possibly go wrong. 
in the context of using #LLM s for retrieval (search) has there been meaningful discussion of the fact that decades of research on human memory showed it to be fundamentally reconstructive (i.e., we don’t merely retrieve something recorded, we construct a memory to some extent?)
this parallel seems so obvious, you’d expect it to be everywhere, but I’m not really seeing that. What have I missed?
How bad are things like AI searches for blogs and the web?
Really bad, it turns out. Even worse than I'd assumed. The drop in referrals when AI search is engaged is apparently *96 percent*.
https://www.forbes.com/sites/rashishrivastava/2025/03/03/openai-perplexity-ai-search-traffic-report/
That's scary for two reasons - firstly, it's bad for supporting anyone actually making content and will reduce content due to demand drops. Second, it implies a HUGE number of people are now just relying on AI slop to answer searches. Eek.
Technische Frage: Ist es eigentlich möglich, ein autoencoding oder seq2seq Modell so zu trainieren, dass es - wie die bekannten Chat-Modelle - beliebige Anweisungen in natürlicher Sprache entgegennehmen und verarbeiten kann, oder ist dazu die generative Architektur unabdingbar?
Das ist ja vielleicht der größte Vorteil des Trainings, das diese Modelle erfahren haben.
Eine grundlegende technische Differenz, die m.E. jede wissenschaftspolitische LLM Strategie berücksichten muss:
Generative (autoregressive) Modelle (die würden wir z.B. für Code Generation brauchen) sind etwas anderes als autoencoding Modelle (für z.B. Klassifikation) oder seq2seq Modelle (für z.B. (multimodale) Übersetzungen). Die autoencoders müssten im Vergleich zu GPT, Claude & Co. - bei gleicher Skalierungsstufe wohlgemerkt - Klassifikation und Informationsextraktion *viel besser* beherrschen, kein ausbeuterisches RLHF benötigen und nur wenig für Halluzinationen anfällig sein. Sie sind halt von den kommerziellen Anbietern nicht auf dieselbe Stufe hochskaliert worden wie die "Chat" Modelle.
Das müssten wir in der Wissenschaft vielleicht selber machen, aber das hätte ja auch Vorteile.
Bip-bop the Bot 
