I am once again thinking about programming in natural language. For most programming languages, you can parse them according to strict grammars that eliminate their ambiguity. These can get ugly, especially for languages like C++ where one symbol can mean many different things depending on its context, but the code is, in theory, able to be parsed. One of the biggest barriers for programming in natural languages, however, are their inherent ambiguity. Natural languages were optimized for efficiency and convenience in communicating between people. They were not designed for clearly communicating with machines. This means that computer programs that try to understand natural language must do so stochastically. It is likely that the text means this. It is possible that a text means that. It's impossible to work with certainty, even if we had a language model with perfect knowledge of a language and the surrounding cultures. One answer is to embrace the probabilistic nature of language and use Large Language Models (LLMs) for natural language programming. After all, they can already write code that seems statistically likely based on natural language prompts. This approach embraces the chaos. The other answer is to describe a strict subset of a language and use that. Instead of trying to let the computer understand the entirety of what a user is saying, make the user explain it in language the computer will understand. This is still a computer language, but something more like the user's native language. This approach clings to rigidity. Neither of these options are ideal, in my opinion. LLMs require a large amount of resources to use, shackling access to natural language programming to those who can pay to rent from large corporations or can afford an expensive GPU while costs are rising. Another computer language might ease new programmers into the concepts of programming, but they are still forced to describe their problems in a rigid language that punishes straying from its parser. I'd like to propose a combination of the two. A language model that parses natural language but checks back in with the user to see if it has understood. This can be done by converting high-ambiguity sentences to low-ambiguity ones when appropriate and storing the latter as source. When the interpretation program is confused, it can check back in with the user to ask what they meant. There are a few important differences between this and programming with an LLM. First, the source is unambiguous natural language, not prompts. This means we can retain deterministic pipelines and reproducible builds. Second, the language model needs to parse, not generate. This should mean that we can use models that are less expensive to run, opening it up to be used more wildly. The goal is understandability. A system that produces consistent output based on given input is more understandable than one that doesn't. In summary, there would be three main layers: Natural language (parsed by a language model) Computer language (subset of a natural language) Language-agnostic IR The computer language is still considered the "source code". It is a common-ground understanding between both the user and the machine. It is expected that such a programming environment convert between the natural language to the computer language live to provide immediate feedback. Once this is done, the computer language can be compiled into a language-agnostic IR that can then be compiled or interpreted. I say language-agnostic because the top-level natural language should be any language the user desires. I haven't made a proof of concept yet, because this is outside of the realm of my current expertise. The current state of Natural Language Processing (NLP) is focused on LLMs and generating text, not interpreting it. I don't know if there are robust solutions for machines to understand human text beyond simply transforming it directly. If anyone reading this has any suggestions or knows of any existing research in this area, please contact me! I'd love to hear your thoughts.
  • 0t79JeIfK01RHyzo@lemmy.mlEnglish
    5·
    20 days ago
    • When does 4 become 5?

    4.5 >=? Lots of programming languages define casting as the largest integer value, not the the rounded value. Outside of programming, there are scenarios where 4 cannot be rounded to 5 either… Like a height requirement for a ride at an amusement park. Though, it might seem natural to round it to humans.

    • When does green become black in a gradient?

    With computer languages, we define colors with red, blue, and green. The above is a gradient on horizontal x-axis, with

    (r = 0, g = x, b = 0)

    This is more complicated. If we surveyed 1 million people, maybe we’d get a plot that defined the start of green as something like the following.

    And then maybe we could take the the average and state that as green.

    Though, is that really good enough description of green to serve as a legal definition of green? Maybe lawyers would select significantly different results than the general population when surveyed for the value of green. Maybe the results should be restricted to lawyers?

    • And when does 2 become 7?

    If we restricted it pixels to a 3x5 grid…

    We could possibly define 2 and 7.

    But then is this a 2 or a 7?

    • jacksilver
      7·
      20 days ago

      These are the kinds of things I bring up when talking to people about using LLMs. Sure for short simple logic the room for ambiguity is small, but for more complex systems, the benefit of programming it is to be explicit. Prompts != programming as they don’t produce deterministic outputs and can lead to faulty outcomes (even if they run).

  • chicken@lemmy.dbzer0.com
    2·
    19 days ago

    I’d like to propose a combination of the two. A language model that parses natural language but checks back in with the user to see if it has understood. This can be done by converting high-ambiguity sentences to low-ambiguity ones when appropriate and storing the latter as source. When the interpretation program is confused, it can check back in with the user to ask what they meant.

    This is something I’ve been thinking about lately. It’s a huge problem that looking into how a given software project works or specifically what it does is normally beyond the reach of most people, or in the case of software that is very elaborate or wasn’t written to be read, beyond the reach of almost everyone. It could help a lot to have some kind of tiered specification/documentation going from more concise to more detailed that can at least be independently confirmed in an automated way to have been derived from each other.

  • nark3d@thelemmy.club
    1·
    18 days ago

    The ambiguity is exactly why a spec has to be runnable. Natural language always leaves room, so what carries the weight is the acceptance check the agent can run and fail against, not the prose. If nothing can fail the output, the ambiguity just moves downstream into the code.