• @[email protected]
    link
    fedilink
    46
    edit-2
    1 year ago

    well 0.9999… is actually 1 because

    x = 0.9999...
    
    10x = 9.9999...
    
    10x (9.9999...) - x (0.9999...) = 9
    
    9x = 9
    
    x = 1
    
    so 0.9999... is 1
    
  • @nogrub
    link
    411 year ago

    it’s almost like computers are not that accurate when calculating floating point numbers

    • @afraid_of_zombies
      link
      61 year ago

      About a year ago I ended up with a floating point value that was something like 1.0000000000078 when it should have been 1. Tore my hair out for hours trying to get the piece of crap embedded vendor locked device to just make it 1.

    • @macrocephalic
      link
      51 year ago

      It’s almost like some useless person created a variable with a distinct set unlikely to be higher than the hundreds as a floating point - when it obviously should have been an int.

      • @WhiskyTangoFoxtrot
        link
        121 year ago

        Nah, it makes sense to use a floating point number here, since unless the test is marked out of a factor of 100 then there will likely be a fractional value as the final percentage. The mistake was not rounding the final displayed value.

  • GigaWerts
    link
    fedilink
    16
    edit-2
    1 year ago

    The issue here lies in how it calculates each correct answer value, which is set at 1/15. This approach introduces an approximation error. When you sum all these values together, the total doesn’t quite reach 1.

    edit: It’s actually 1/19 for each question

    (1/19)*19 = 0,9999999991

  • @xpinchx
    link
    91 year ago

    Heh, our WMS does this. Picking through a batch and we go from 19% complete to 22.573729384674829273747% complete.

  • @momocchi
    link
    English
    81 year ago

    19.0000000000f/19.0000000000f

    • @affiliate
      link
      9
      edit-2
      1 year ago

      the pedantic answer is that, from a rigorous perspective, 99.9999999999999% isn’t the same as 100% because the decimals don’t repeat forever. but a more practical answer would be that they are the same number. because of how computers (usually) round numbers, the stuff showing up after the 8th decimal place is (usually) junk that can be ignored.

      an interesting example of this idea in practice has to do with the irrational number π, which nasa only approximates to 15 decimal places because that’s more than enough for most of the calculations they do (the linked page gives a better and more detailed explanation).

  • Flying Squid
    link
    71 year ago

    Computer: “You still suck, puny fleshbag.”