ChatGPT generates cancer treatment plans that are full of errors — Study finds that ChatGPT provided false information when asked to design cancer treatment plans::Researchers at Brigham and Women’s Hospital found that cancer treatment plans generated by OpenAI’s revolutionary chatbot were full of errors.

  • zeppo@lemmy.world
    link
    fedilink
    English
    arrow-up
    222
    ·
    1 year ago

    I’m still confused that people don’t realize this. It’s not an oracle. It’s a program that generates sentences word by word based on statistical analysis, with no concept of fact checking. It’s even worse that someone actually did a study instead of simply acknowledging or realizing that ChatGPT is happy to just make stuff up.

    • net00@lemm.ee
      link
      fedilink
      English
      arrow-up
      21
      ·
      1 year ago

      Yeah this stuff was always marketed to automate simple and repetitive things we do daily. it’s mostly the media I guess who started misleading everyone into thinking this was AI like skynet. It’s still useful, not just as a all knowing AI god

    • inspxtr@lemmy.world
      link
      fedilink
      English
      arrow-up
      18
      arrow-down
      2
      ·
      1 year ago

      while I agree it has become more of a common knowledge that they’re unreliable, this can add on to the myriad of examples for corporations, big organizations and government to abstain from using them, or at least be informed about these various cases with their nuances to know how to integrate them.

      Why? I think partly because many of these organizations are racing to adopt them, for cost-cutting purposes, to chase the hype, or too slow to regulate them, … and there are/could still be very good uses that justify it in the first place.

      I don’t think it’s good enough to have a blanket conception to not trust them completely. I think we need multiple examples of the good, the bad and the questionable in different domains to inform the people in charge, the people using them, and the people who might be affected by their use.

      Kinda like the recent event at DefCon trying to exploit LLMs, it’s not enough we have some intuition about their harms, the people at the event aim to demonstrate the extremes of such harms AFAIK. These efforts can help inform developers/researchers to mitigate them, as well as showing concretely to anyone trying to adopt them how harmful they could be.

      Regulators also need these examples in specific domains so they may be informed on how to create policies on them, sometimes building or modifying already existing policies of such domains.

      • zeppo@lemmy.world
        link
        fedilink
        English
        arrow-up
        10
        ·
        1 year ago

        This is true and well-stated. Mainly what I wish people would understand is there are current appropriate uses, like ‘rewrite my marketing email’, but generating information that could result in great harm if inaccurate is an inappropriate use. It’s all about the specific model, though - if you had a ChatGPT system trained extensively on medical information, it would result in greater accuracy, but still the information would need expert human review before any decision were made. Mainly I wish the media had been more responsible and accurate in portraying these systems to the public.

      • jvisick@programming.dev
        link
        fedilink
        English
        arrow-up
        7
        ·
        1 year ago

        I don’t think it’s good enough to have a blanket conception to not trust them completely.

        On the other hand, I actually think we should, as a rule, not trust the output of an LLM.

        They’re great for generative purposes, but I don’t think there’s a single valid case where the accuracy of their response should be outright trusted. Any information you get from an AI model should be validated outright.

        There are many cases where a simple once-over from a human is good enough, but any time it tells you something you didn’t already know you should not trust it and, if you want to rely on that information, you should validate that it’s accurate.

    • iforgotmyinstance@lemmy.world
      link
      fedilink
      English
      arrow-up
      12
      arrow-down
      1
      ·
      1 year ago

      I know university professors struggling with this concept. They are so convinced using an LLM is plagiarism.

      It can lead to plagiarism if you use it poorly, which is why you control the information you feed it. Then proofread and edit.

      • zeppo@lemmy.world
        link
        fedilink
        English
        arrow-up
        12
        ·
        1 year ago

        Another related confusion in academia recently is the ‘AI detector’. It could easily be defeated with minor rewrites, if they were even accurate in the first place. My favorite misconception is there was a story of a professor who told students “I asked ChatGPT if it wrote this, and it said yes” which is just really not how it works.

      • ZodiacSF1969@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        5
        arrow-down
        1
        ·
        1 year ago

        I can understand the plagiarism argument, though you have to extend the definition of it. If I am expected to write an essay, but I use ChatGPT instead, then I am fraudulently presenting the work as my own. Plagiarism might not be the right word, or maybe it’s a case where language is going to evolve so that plagiarism includes passing off AI generated work as your own. Either way it’s cheating unless I was specifically allowed to use AI.

        • iforgotmyinstance@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          arrow-down
          1
          ·
          1 year ago

          If the argument and the sources are incongruous, that isn’t the fault of the LLM/AI. That’s the authors fault for not proofreading and editing.

          You assume an inherent morality of LLMs but they are amoral constructs. They are tools, and you limit yourself by not learning them.

          • ZodiacSF1969@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            1
            arrow-down
            1
            ·
            1 year ago

            I didn’t say anything about the sources being incongruent? That’s a completely separate issue. We were talking about plagiarism.

            I don’t understand the morality comment either, I didn’t ascribe any morality to AI, I was talking about whether using them fits the definition of plagiarism or not.

            If you are expected to write it yourself, and you use an LLM to generate it, then that’s cheating in my opinion. Yes, of course we shoukd learn to use AI, but if you are told to do something and you get a person or LLM to do it for you, then you didn’t complete the task as you were told. And at university that can have consequences.

    • fubo@lemmy.world
      link
      fedilink
      English
      arrow-up
      23
      arrow-down
      15
      ·
      1 year ago

      It’s even worse that someone actually did a study instead of simply acknowledging or realizing that ChatGPT is happy to just make stuff up.

      Sure, the world should just trust preconceptions instead of doing science to check our beliefs. That worked great for tens of thousands of years of prehistory.

      • zeppo@lemmy.world
        link
        fedilink
        English
        arrow-up
        29
        arrow-down
        1
        ·
        edit-2
        1 year ago

        It’s not merely a preconception. It’s a rather obvious and well-known limitation of these systems. What I am decrying is that some people, from apparent ignorance, think things like “ChatGPT can give a reliable cancer treatment plan!” or “here, I’ll have it write a legal brief and not even check it for accuracy”. But sure, I agree with you, minus the needless sarcasm. It’s useful to prove or disprove even absurd hypotheses. And clearly people need to be definitely told that ChatGPT is not always factual, so hopefully this helps.

        • adeoxymus@lemmy.world
          link
          fedilink
          English
          arrow-up
          8
          ·
          1 year ago

          I’d say that a measurement always trumps arguments. At least you know how accurate they are, this statement cannot follow from reason:

          The JAMA study found that 12.5% of ChatGPT’s responses were “hallucinated,” and that the chatbot was most likely to present incorrect information when asked about localized treatment for advanced diseases or immunotherapy.

          • zeppo@lemmy.world
            link
            fedilink
            English
            arrow-up
            5
            ·
            1 year ago

            That’s useful. It’s also good to note that the information the agent can relay depends heavily on the data used to train the model, so it could change.

      • PetDinosaurs@lemmy.world
        link
        fedilink
        English
        arrow-up
        10
        arrow-down
        1
        ·
        1 year ago

        Why the hell are people down voting you?

        This is absolutely correct. We need to do the science. Always. Doesn’t matter what the theory says. Doesn’t matter that our guess is probably correct.

        Plus, all these studies tell us much more than just the conclusion.

      • Takumidesh@lemmy.world
        link
        fedilink
        English
        arrow-up
        7
        ·
        1 year ago

        It’s not even a preconception, it’s willful ignorance, the website itself tells you multiple times that it is not accurate.

        The bottom of every chat has this text: “Free Research Preview. ChatGPT may produce inaccurate information about people, places, or facts. ChatGPT August 3 Version”

        And when you first use it, a modal pops up explaining the same thing.

      • yiliu@informis.land
        link
        fedilink
        English
        arrow-up
        9
        arrow-down
        2
        ·
        1 year ago

        “After an extensive three-year study, I have discovered that touching a hot element with one’s bare hand does, in fact, hurt.”

        “That seems like it was unnecessary…”

        “Do U even science bro?!”

        Not everything automatically deserves a study. Were there any non-rando people out there claiming that ChatGPT could totally generate legit cancer treatment plans that people could then follow?

      • Windex007@lemmy.world
        link
        fedilink
        English
        arrow-up
        5
        arrow-down
        2
        ·
        1 year ago

        ChatGPT isn’t some newly discovered sentient species.

        It’s a machine designed and built by human engineers.

        This is like suggesting that we study fortune cookies to see if they can accurately forecast the future. The manufacturer can simply tell you the limitation of their product… Being that they can not divine the future.

    • dual_sport_dork 🐧🗡️@lemmy.world
      link
      fedilink
      English
      arrow-up
      7
      arrow-down
      2
      ·
      edit-2
      1 year ago

      This is why without some hitherto unknown or so far undeveloped capability of these sorts of LLM models, they’ll never actually be useful for performing any kind of mission critical work. The catch-22 is this: You can’t trust the AI to produce correct work without some kind of potentially dangerous, showstopping, or embarassing error. This isn’t a problem if you’re just, say, having it paint pictures. Or maybe even helping you twiddle the CSS on your web site. If there is a failure here, no one dies.

      But what if your application is critical to life or safety? Like prescribing medical care, or designing a building that won’t fall down, or deciding which building the drone should bomb. Well, you have to get a trained or accredited professional in whatever field we’re talking about to check all of its work. And how much effort does that entail? As it turns out, pretty much exactly as much as having said trained or accredited professional do the work in the first place.

    • nfsu2@feddit.cl
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 year ago

      true, I tried to explain this to my parents because they were scared of it and they seemed skeptical.

    • PreviouslyAmused@lemmy.ml
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      2
      ·
      1 year ago

      But it’s supposed to be the future! I want the full talking spaceship like in Star Trek, not this … “initial learning steps” BS!

      I was raised on SciFi and am now mad that I don’t have all the cool made up things from those shows/movies!

    • xkforce@lemmy.world
      link
      fedilink
      English
      arrow-up
      36
      arrow-down
      3
      ·
      edit-2
      1 year ago

      Part of the reason for studies like this is to debunk peoples’ expectations of AI’s capabilities. A lot of people are under the impression that cgatGPT can do ANYTHING and can think and reason when in reality it is a bullshitter that does nothing more than mimic what it thinks a suitable answer looks like. Just like a parrot.

    • PeleSpirit@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      arrow-down
      13
      ·
      1 year ago

      Because if it’s able to crawl all of the science pubs, then it would be able to try different combos until it works. Isn’t that how it could/is being used, to test stuff?

      • Ranessin@feddit.de
        link
        fedilink
        English
        arrow-up
        16
        ·
        1 year ago

        It doesn’t check the stuff it generates other than on grammatical and orthographical errors. It’s not intelligent or has knowledge outside of how to create text. The text looks useful, but it doesn’t know what it contains in a way something intelligent would.

        • PeleSpirit@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          arrow-down
          6
          ·
          1 year ago

          It seems like it could check for that though, which is what chatgpt doesn’t do but we all assumed would. I’m sure there are ai programs that could and do check for possibilities on only information we know to be true.

          • ZodiacSF1969@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            1
            ·
            1 year ago

            People who understand the technology did not assume that, but yes the general public has a lot of misconceptions about it.

      • stephen01king@lemmy.zip
        link
        fedilink
        English
        arrow-up
        6
        ·
        1 year ago

        If you want an AI that can create cancer treatment, you need to train it on creating cancer treatment, and not just use one that is trained on general knowledge. Even if you train it on science publications, all it can now reliably do is mimic a science journal since it has not been trained on how to parse the knowledge in the journal itself.

        • amki@feddit.de
          link
          fedilink
          English
          arrow-up
          4
          ·
          edit-2
          1 year ago

          Which is exactly the problem people think has been solved but isn’t anywhere near being solved. It cannot comprehend semantics, the meaning of things is completely beyond it and all other AIs.

          Unfortunately saying I made a thing that creates vaguely human looking speech with little content isn’t astonishing to most people hence they are looking for something useful this breakthrough machine must be able to do and then they don’t find anything leading to these articles.

        • PeleSpirit@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          Right, but can’t they tell it to also try thousands and thousands of combos that humans could never do? I think ChatGPT is both super amazing and as stupid as a rock at the same time. I thought the vaccine used an AI to do that. I’m obviously clueless, I’m seriously asking.

          • ZodiacSF1969@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            1
            ·
            1 year ago

            I don’t know about AI, but there are already computer programs that try many different combinations of, for example, chemical structures with known pharmacological properties and then output new drugs that could possibly be used to treat something. Of course you have to verify with research and studies.

            I’m sure there will be AI’s or machine learning programs, if not already, that can do this as well and perhaps improve upon the process. But they would need to be specifically trained for that purpose. ChatGPT is a LLM, it’s made to generate language that fits a given prompt, I would not expect it to be great at creating cancer treatments and I’m not sure why we needed a study to learn that. OpenAI tells you already that the results can be inaccurate or outright wrong.

            • PeleSpirit@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              ·
              1 year ago

              I’m in Seattle and surrounded by people who are techy while not being techy myself, so the innovations they talk about are mind blowing. I thought ChatGPT at first was like all the other tech I heard about. But when you think about it, they would never release that for free first of all, and it would be too powerful for evil people. I was just letting people know what a non-techy thought.

  • Uncaged_Jay@lemmy.world
    link
    fedilink
    English
    arrow-up
    51
    arrow-down
    1
    ·
    1 year ago

    “Hey, program that is basically just regurgitating information, how do we do this incredibly complex things that even we don’t understand yet?”

    “Here ya go.”

    “Wow, this is wrong.”

    “No shit.”

    • JackbyDev@programming.dev
      link
      fedilink
      English
      arrow-up
      21
      arrow-down
      1
      ·
      edit-2
      1 year ago

      “Be aware that ChatGPT may produce wrong or inaccurate results, what is your question?”

      How beat cancer

      wrong, inaccurate information

      😱

    • lolcatnip@reddthat.com
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      1
      ·
      1 year ago

      It reminds me of some particularly badly written episodes of Star Trek where they use the holodeck to simulate some weird scenario involving exotic physics nobody understands, and it works perfectly.

    • 5BC2E7@lemmy.world
      link
      fedilink
      English
      arrow-up
      12
      arrow-down
      2
      ·
      1 year ago

      Because it’s been hyped. They had announced it could pass the medical licensing exam with good scores. The belief that it can replace a doctor has already been put forward

        • 5BC2E7@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          edit-2
          1 year ago

          If my previous response didn’t answer your question. I’m not sure if it helps to tell you that no one said it can replace doctors.

    • solstice@lemmy.world
      link
      fedilink
      English
      arrow-up
      9
      ·
      edit-2
      1 year ago

      On two occasions I have been asked, ‘Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?’ I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.

      Charles Babbage

      Better tech, same stupid end users lmao

  • elboyoloco@lemmy.world
    link
    fedilink
    English
    arrow-up
    43
    arrow-down
    1
    ·
    1 year ago

    Scientist: Askes question to magic conch about cancer.

    Conch: “Trying shoving bees up your ass.”

    Scientists: 😡

    • PreviouslyAmused@lemmy.ml
      link
      fedilink
      English
      arrow-up
      4
      ·
      1 year ago

      Scientist to world: “magic conch is useless in trying to cure cancer.”

      News media: “Magic Conch is completely useless!”

    • solstice@lemmy.world
      link
      fedilink
      English
      arrow-up
      11
      ·
      1 year ago

      It’s hilarious to me that people need to be told word for word that chat gpt is NOT literally the cure for cancer.

    • playerwhoplayyes@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 year ago

      Probably people who want to check AI accuracy or people who don’t want to search or go to the doctor and ask it to ChatGPT, even if I ask a cure, I will use other AI such as the bing AI, but still I go to the doctor, I will never ask an AI or search on the internet cures to cancer, never self-medicated.

    • Agent641@lemmy.world
      link
      fedilink
      English
      arrow-up
      7
      ·
      1 year ago

      Chatgpt fails at basic math, and lies ablut the existence of technical documentation.

      I mostly use it for recipe inspuration and discussing books Ive read recently. Just banter, you know? Nothing mission-critical.

      • IDontHavePantsOn@lemm.ee
        link
        fedilink
        English
        arrow-up
        5
        ·
        1 year ago

        Just a couple days ago it continually told me it was possible to re-tile part of my shower that is broken without cutting tiles, but none of the math added up. (18.5H x 21.5w area) “Place a 9” tile vertically. Place another 9“ tile vertically on top on the same side. Place another 9" tile on top vertically to cover the remainder of the area."

        I told chatgpt it was wrong, which it admitted, and spit out another wrong answer. I tried specifying a few more times before I started a new chat and dumbed it down to just a simple math algorithm problem. The first part of the chat said it was possible, layed out the steps, and then said it wasn’t possible in the last sentence.

        I surely wouldn’t trust chatgpt to advise my healthcare, but after seeing it spit out very wrong answers to a basic math question, I’m just wondering why anyone would try to have it advise anyone’s health are.

        • Brandon658@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          People want to trust it as a source of quick knowledge. It is easier to be told 9 goes into 81 a total of 8 times trusting that the computer is always right, because it had access to everything, than to work out the answer given was wrong and is actually 9.

          Think of WebMD. People love to self diagnose despite it commonly being known as a bad practice. But they do so because they can do it with less effort, faster, and cheaper than making an appointment to drive to an office so you can speak with a doctor that runs a few tests and gets back to you in a week saying they aren’t sure and need to do that process all over again.

          • raptir@lemm.ee
            link
            fedilink
            English
            arrow-up
            1
            ·
            1 year ago

            The healthcare issue is that I’m usually checking WebMD to see if what I’m experiencing is an actual issue that I need to go to the doctor for since it’s so expensive to go to a doctor.

  • Obinice@lemmy.world
    link
    fedilink
    English
    arrow-up
    32
    ·
    1 year ago

    Well, it’s a good thing absolutely no clinician is using it to figure out how to treat their patient’s cancer… then?

    I imagine it also struggles when asked to go to the kitchen and make a cup of tea. Thankfully, nobody asks this, because it’s outside of the scope of the application.

    • clutch@lemmy.ml
      link
      fedilink
      English
      arrow-up
      12
      arrow-down
      1
      ·
      1 year ago

      The fear is that hospital administrators equipped with their MBA degrees will think about using it to replace expensive, experienced physicians and diagnosticians

      • whoisearth@lemmy.ca
        link
        fedilink
        English
        arrow-up
        11
        ·
        1 year ago

        They’ve been trying this shit for decades already with established AI like Big Blue. This isn’t a new pattern. Those in charge need to keep driving costs down and profit up.

        Race to the bottom.

      • Obinice@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 year ago

        If that were legal, I’d absolutely be worried, you make a good point.

        Even Doctor need special additional qualifications to do things like diagnose illnesses via radiographic imagery, etc. Specialised AI is making good progress in aiding these sorts of things, but a generalised and very poor AI like ChatGPT will never be legally certified to do this sort of thing.

        Once we have a much more effective generalised AI, things will get more interesting. It’ll have to prove itself thoroughly though, before being certified, so it’ll still be a few years after it appears before we see it used in clinical applications.

  • Rexios@lemm.ee
    link
    fedilink
    English
    arrow-up
    27
    ·
    1 year ago

    Okay and? GPT lies how is this news every other day? Lazy ass journalists.

  • Sanctus@lemmy.world
    link
    fedilink
    English
    arrow-up
    19
    arrow-down
    1
    ·
    1 year ago

    These studies are for the people out there who think ChatGPT thinks. Its a really good email assistant, and it can even get basic programming questions right if you are detailed with your prompt. Now everyone stop trying to make this thing like Finn’s mom in adventure time and just use it to helo you write a long email in a few seconds. Jfc.

    • lonke@feddit.nu
      link
      fedilink
      English
      arrow-up
      5
      arrow-down
      2
      ·
      1 year ago

      I use ChatGPT primarily for programming, and it’s particularly well suited for programming.

      “Even get basic programming questions right if you are detailed with your prompt”

      is underselling its capabilities in that regard. Especially GPT-4 has been able to help me with everything from obscure adobe ExtendScript scripts to infrequently seen ‘unsafe’ C# OpenGL perspective matrix math. All with prompts of a sentence maximum.

      • Sanctus@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        I’m specifically referring to ChatGPT. GPT-4 is a different beast that I’m sure is quite adept.

        • lonke@feddit.nu
          link
          fedilink
          English
          arrow-up
          3
          ·
          1 year ago

          ChatGPT is GPT 3.5 & GPT 4, as far as I’m aware.

          3.5 is also very capable when it comes to programming, for any well known framework or language. It’s not as capable, but it is still very capable.

          • Sanctus@lemmy.world
            link
            fedilink
            English
            arrow-up
            2
            ·
            edit-2
            1 year ago

            I found it was okay with Unity libraries but really good with things like Excel.Interop and business libraries, as well as general programming concepts like linked lists.

            For instance, I made a random dungeon generator using Unity’s visual scripting. It seemed to be unaware of the visual scripting library. But I’m automating excel processes right now and its on point with those.

            • lonke@feddit.nu
              link
              fedilink
              English
              arrow-up
              1
              ·
              1 year ago

              Well, the data it was trained on had a cutoff point in 2021 which would explain that.

              I’ve used it (GPT 3) a fair amount for Unity, and I’m fairly pleased with the results, it’s saved me a fair amount of time. Implementing object pooling and editor window dialogues for scene translation management for example.

              Of course, programming knowledge is required for it to be of consistent use, which, on second thought, may not be at all obvious.

              • Sanctus@lemmy.world
                link
                fedilink
                English
                arrow-up
                1
                ·
                1 year ago

                Maybe Visual Scripting is on the cusp of its knowledge. I thought it released in 2021. It has replaced my rubber ducky in corporate environments thats for sure. I plan on using it again for game development after this discussion. My visual scripting use case was off the beaten path which was probably why it had a hard time.

    • EssentialCoffee@midwest.social
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      1
      ·
      1 year ago

      I use it for D&D. It’s fantastic at coming up with adventures, NPCs, story hooks, taverns, etc.

      All of those things are made up.

      • Sanctus@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 year ago

        Its fantastic at that. I had it help me with a Dark Heresy session. Its not bad at generating names, places, and even personalities for jobbers.

    • Corngood@lemmy.ml
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      I’m going to need it to turn those emails back into the bullet points used to create them, so I don’t have to read the filler.

  • LazyBane@lemmy.world
    link
    fedilink
    English
    arrow-up
    17
    ·
    1 year ago

    People really need to get in their heads that AI can “hallucinate” random information and that any implementation on an AI needs a qualified human overseeing it.

    • grabyourmotherskeys@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      Exactly, it’s stringing together information in a series of iterations, each time adding a new inference consistent with what came before. It has no way to know if that inference is correct.

  • NigelFrobisher@aussie.zone
    link
    fedilink
    English
    arrow-up
    16
    ·
    1 year ago

    People really need to understand what LLMs are, and also what they are not. None of the messianic hype or even use of the term “AI” helps with this, and most of the ridiculous claims made in the space make me expect Peter Molyneux to be involved somehow.

    • dx1@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      edit-2
      1 year ago

      LLMs fit in the “weak AI” category. I’d be inclined to not call them “AI” at all, since there is no intelligence, just the illusion of intelligence (if I could just redefine the term “AI”). It’s possible to build intelligent AI, but probabilistic text construction isn’t even close.

      • fsmacolyte@lemmy.world
        link
        fedilink
        English
        arrow-up
        5
        ·
        1 year ago

        It’s possible to build intelligent AI

        What does intelligent AI that we can currently build look like?

        • dx1@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          ·
          edit-2
          1 year ago

          There’s “can build” and “have built”. The basic idea is about continuously aggregating data and performing pattern analysis and basically cognitive schema assimilation/accommodation in the same way humans do. It’s absolutely doable, at least I think so.

          • fsmacolyte@lemmy.world
            link
            fedilink
            English
            arrow-up
            1
            ·
            1 year ago

            I haven’t heard of cognitive schema assimilation. That sounds interesting. It sounds like it might fall prey to challenges we’ve had with symbolic AI in the past though.

            • dx1@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              ·
              edit-2
              1 year ago

              It’s a concept from psychology. Instead of just a model of linguistic construction, the model has to actually be a comprehensive, data-forged model of reality as far as human observation goes/we care about. In poorly tuned, low-information scenarios, it would fall mostly into the same traps human do (e.g. falling for propaganda or pseudoscientific theories) but, if finely tuned, should emulate accurate theories and even predictive results with an expansive enough domain.

  • SirGolan@lemmy.sdf.org
    link
    fedilink
    English
    arrow-up
    20
    arrow-down
    6
    ·
    1 year ago

    What’s with all the hit jobs on ChatGPT?

    Prompts were input to the GPT-3.5-turbo-0301 model via the ChatGPT (OpenAI) interface.

    This is the second paper I’ve seen recently to complain ChatGPT is crap and be using GPT3.5. There is a world of difference between 3.5 and 4. Unfortunately news sites aren’t savvy enough to pick up on that and just run with “ChatGPT sucks!” Also it’s not even ChatGPT if they’re using that model. The paper is wrong (or it’s old) because there’s no way to use that model in the ChatGPT interface. I don’t think there ever was either. It was probably ChatGPT 0301 or something which is (afaik) slightly different.

    Anyway, tldr, paper is similar to “I tried running Diablo 4 on my Windows 95 computer and it didn’t work. Surprised Pikachu!”

    • eggymachus@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      9
      arrow-down
      14
      ·
      1 year ago

      And this tech community is being weirdly luddite over it as well, saying stuff like “it’s only a bunch of statistics predicting what’s best to say next”. Guess what, so are you, sunshine.

      • PreviouslyAmused@lemmy.ml
        link
        fedilink
        English
        arrow-up
        6
        ·
        1 year ago

        I mean, people are slightly more complicated than that. But sure, at their most basic, people simply communicate with statistical models.

        • eggymachus@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          2
          ·
          1 year ago

          Ok, maybe slightly :) but it surprises me that the ability to emulate a basic human is dismissed as “just statistics”, since until a year ago it seemed like an impossible task…

          • markr@lemmy.world
            link
            fedilink
            English
            arrow-up
            2
            ·
            1 year ago

            The dismissal is coming from the class of people most threatened by these systems.

      • dukk@programming.dev
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        1
        ·
        1 year ago

        IMO for AI to reach a useful point it needs to be able to learn. Now I’m no expert on neural networks, but if it can’t learn anything new once it’s been trained, it’s never really going to reach its true potential. It can imitate a human, but that’s about it. Once AI can really learn, it’ll become an order of magnitude more useful. Don’t get me wrong: all this AI work is a step in the right direction, but we’ll only be able to go so far with pre-trained models.

      • amki@feddit.de
        link
        fedilink
        English
        arrow-up
        4
        arrow-down
        2
        ·
        1 year ago

        Might be true for you but most people do have a concept of true and false and don’t just dream up stuff to say.

        • markr@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          Actually we ‘dream up’ things to say quite a lot. As in our unconscious functions are far more important to our mental processes than we like to admit. Also we are basically not very good at evaluating the truth value of complex expressions.

        • eggymachus@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          Yeah, I was probably a bit too caustic, and there’s more to (A)GI than an LLM can achieve on its own, but I do believe that some, and perhaps a large, part of human consciousness works in a similar manner.

          I also think that LLMs can have models of concepts, otherwise they couldn’t do what they do. Probably also of truth and falsity, but perhaps with a lack of external grounding?

      • SirGolan@lemmy.sdf.org
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        2
        ·
        1 year ago

        Hah! That’s the response I always give! I’m not saying our brains work the exact same way because they don’t and there’s still a lot missing from current AI but I’ve definitely noticed that at least for myself, I do just predict the next word when I’m talking or writing (with some extra constraints). But even with LLMs there’s more going on then that since the attention mechanism allows it to consider parts of the prompt and what it’s already written as it’s trying to come up with the next word. On the other hand, I can go back and correct mistakes I make while writing and LLMs can’t do that…it’s just a linear stream.

        • eggymachus@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          2
          ·
          1 year ago

          Agree, I have definitely fallen for the temptation to say what sounds better, rather than what’s exactly true… Less so in writing, possibly because it’s less of a linear stream.