So DeepSeek has this very cool feature that displays what it is “thinking” before it gives you its answer. It’s quite neat in that you can see its “thought” process, but it also has the added benefit of revealing whatever bias it might have developed with its training data.

In this case, I asked it if we might be living in a “slow motion World War 3” with the Maiden Coup in Ukraine being the opening shots. The mf thought that I might “buy Russian propaganda” because I called it a coup rather than a revolution.

So although DeepSeek is Chinese, it was still very clearly trained on a lot of mainstream / LIB information.

  • sodium_nitride [she/her, any]@hexbear.netEnglish
    561·
    3 days ago

    People need to remember that LLMs are closer to being super juiced up autocorrects trained on other people’s texting patterns and less close to being actual reasoning engines.

    There are actual AI architectures that work on explicitly coded rules. But those did not recieve anywhere near this level of funding or hype.

    They’ve been around for decades, I even had a book from the 90s that I’d borrowed from my uni library talking about how to make inference engines … and also lamenting that inference engines had been neglected for many years (lmao)

    • insurgentrat [she/her, it/its]@hexbear.netEnglish
      34·
      3 days ago

      A huge amount of what they’ve ingested is social media comments, blogs, and what passes for news.

      So they’re basically as reliable as asking a redditor except the subreddit you submit it to is random and hidden from you, and the redditor is on LSD doing a free association exercise utterly unconcerned with truth claims, and also the most convincing liar you’ll ever meet.

      • NewOldGuard@lemmy.mlEnglish
        111·
        3 days ago

        The machine learning models which came about before LLMs were often smaller in scope but much more competent. E.g. image recognition models, something newer broad “multimodal” models struggle with; theorem provers and other symbolic AI applications, another area LLMs struggle with.

        The modern crop of LLMs are juiced up autocorrect. They are finding the statistically most likely next token and spitting it out based on training data. They don’t create novel thoughts or logic, just regurgitate from their slurry of training data. The human brain does not work anything like this. LLMs are not modeled on any organic system, just on what some ML/AI researchers assumed was the structure of a brain. When we “hallucinate logic” it’s part of a process of envisioning abstract representations of our world and reasoning through different outcomes; when an LLM hallucinates it is just creating what its training dictates is a likely answer.

        This doesn’t mean ML doesn’t have a broad variety of applications but LLMs have gotta be one of the weakest in terms of actually shifting paradigms. Source: software engineer who works with neural nets with academic background in computational math and statistical analysis

      • sodium_nitride [she/her, any]@hexbear.netEnglish
        101·
        3 days ago

        All the other avenues of AI research are and were NO WHERE near as comprehensive or competent as LLM machines.

        Depends on what you want to accomplish and how much resources you want to expend.

        Discarding probability based systems as “juiced up autocorrect” will discard

        I have not discarded LLMs. I know some people use them to great effect, but one must be deeply skeptical of their use as oracles.

        If you use them for their intended purpose, they can be useful, just as autocorrect is useful. I have used LLMs to great effect for helping me cut down on my word count for certain assignments, or as a psudeo-google search for coding assistance.

        I am well aware that the other approaches cannot do these things. They tend to suck at language processing. However, AI architectures using explicitly coded rules have the advantage over LLMs that they are not so prone to hallucinating, which makes them safer and more useful for certain other tasks.

        Not to mention that LLMs themselves were largely unviable until the creation of the attention mechanism and humanity throwing ungodly amounts of resources at them (hundreds of billions of dollars of investment).

        I am sorry to tell you that your brain also hallucinates logic, just on a much larger scale with a ton more neural connections

        I am aware that human brains also hallucinate logic. That’s why I don’t place must weight on random anecdotes when talking about politics or science.

        Please don’t do this kind of luddite historical revisionism

        What historical revisionism? The only thing my comment mentions is that inference engines did not receive as much hype or funding as LLMs, which is true. And how is anything I have stated “luddism”?

        go ask LISP bros how their AI machine business turned out, just don’t mention Chapter 11 they’d get PTSD

        This doesn’t mean anything when all the AI companies are hemorrhaging money at an epic scale. At least the LISP bros can say that they never built the monument to the irrationality of capitalism that is the AI stock bubble.

        Or maybe they did with the dot com bubble. Idk much about that period.

  • Euergetes [none/use name]@hexbear.netEnglish
    60·
    3 days ago

    I know this is somewhat unserious but I’m genuinely distressed at the thought you’d think a LLM model would be “based” if it aggreed on this.

    It is talking to you in english and english media, subsequently english public opinion is overwhelmingly of that opinion. if Elon Musk can’t keep his pet chatbot a nazi because the input data isn’t 100% nazi, why would the one from a chinese firm–not the government–uphold a decent political line?

  • Chana [none/use name]@hexbear.netEnglish
    19·
    3 days ago

    LLMs don’t think they just chew up and recycle garbage from the internet. Sometimes the recycled garbage is tasty, yum yum! Like you don’t know the right terms for something but you can describe it and by association it will find the right links for you (basically a search engine). But usually, recycled garbage is still garbage and churning it only makes it worse.

    • IHave69XiBucks@lemmygrad.mlEnglish
      8·
      3 days ago

      I dont think its an illusion of choice so much as it not being the choice you think. Using a different LLM isnt gonna get you that much different of an answer, but it does change which company gets your info from the query. So i think its a good idea if your in the US to use Local, or Chinese LLMs as much as possible. Otherwise your telling US companies a lot about yourself.

    • Le_Wokisme [they/them, undecided]@hexbear.netEnglish
      10·
      3 days ago

      i’d think you could build a vector space in multiple languages (or in those meta languages the pre-LLM machine translation tools use). the programmers would have to design it to do that of course but there’s no reason the tokens for blue cat, gato azul, and 蓝猫 shouldn’t be correlated.