ChatGPT Isn’t as Good at Coding as We Thought::undefined

  • @abhibeckert
    link
    English
    6
    edit-2
    10 months ago

    So this isn’t a real example, it’s just something I slapped together now as an example of how I generally use ChatGPT 4.0. In a more realistic scenario, I’d be asking more detailed questions, pasting in my existing code, and asking the AI to write smaller sections (maybe 10 lines of code at a time).

    A single chat might run for five or six hours, continuously discussing an individual task I’m working on, and I won’t just be asking it to write a bunch code. For example in this case I might spend 10 minutes hashing out the exact contents of the ‘Activity’ record (or, maybe I already did that a month ago, and I might just paste in an example). I’d also be testing the code as I go and writing some of it myself, occasionally asking questions like ‘how do I format a date as 4 Feb 2016’ in JavaScript?’ or ‘this line of code fails with X. Why?’

    In those five or six hours, I estimate I’m able to get two or three days of work done (at my level of productivity before I started paying $20/month for ChatGPT+).

    I’m also only pasting the code - ChatGPT also explains the code it outputs, and often those explanations are more useful than the actual code.

    In JavaScript write a “reserveSeat(‘A42’, ‘Adult’)” function that sends an ActivityStreams 2.0 activity to (this.apiUrl)/reserve and inform the user if it failed to be reserved

    class SeatReservationService {
      constructor(apiUrl) {
        this.apiUrl = apiUrl;
      }
    
      async reserveSeat(seatNumber, seatType) {
        // Construct an ActivityStreams 2.0 compliant JSON object
        const activity = {
          '@context': 'https://www.w3.org/ns/activitystreams',
          'type': 'Reserve',
          'actor': {
            'type': 'Person',
            'name': 'User'
          },
          'object': {
            'type': 'Seat',
            'id': seatNumber,
            'category': seatType
          }
        };
    
        try {
          // Send the activity object to the specified API endpoint using an HTTP POST request
          const response = await fetch(`${this.apiUrl}/reserve`, {
            method: 'POST',
            headers: {
              'Content-Type': 'application/json'
            },
            body: JSON.stringify(activity)
          });
    
          // Handle the server’s response
          if (response.ok) {
            // If the request was successful, log a success message
            console.log('Seat reserved successfully.');
          } else {
            // If the server responds with a status code other than 2xx, throw an error
            throw new Error('Failed to reserve seat. Server responded with ' + response.status);
          }
        } catch (error) {
          // Inform the user if the reservation failed
          console.error('Failed to reserve seat:', error);
          alert('Failed to reserve the seat. Please try again.');
        }
      }
    }
    
    // Usage
    const service = new SeatReservationService('https://example.com/api');
    service.reserveSeat('A42', 'Adult');
    

    Write a PHP request handler that does $this->reserveSeat(‘A42’), and checks if it failed with a SeatNotAvailable exception

    class SeatNotAvailable extends Exception {
        public function __construct($message = "Seat not available", $code = 0, Exception $previous = null) {
            parent::__construct($message, $code, $previous);
        }
    }
    
    class SeatReservationService {
    
        public function reserveSeat($seatNumber) {
            // Mockup condition that the seat 'A42' is not available
            if ($seatNumber === 'A42') {
                throw new SeatNotAvailable("Seat $seatNumber is not available.");
            }
    
            // Normally here you'd interact with a database or another kind of storage
            // to actually reserve the seat.
            // ... (reserve seat logic) ...
            
            return true;
        }
    }
    
    class SeatReservationController {
    
        private $service;
    
        public function __construct() {
            $this->service = new SeatReservationService();
        }
    
        public function handleRequest() {
            try {
                $this->service->reserveSeat('A42');
                echo "Seat reserved successfully.";
            } catch (SeatNotAvailable $e) {
                echo $e->getMessage();
            }
        }
    }
    

    Write a PHPUnit test for SeatReservationController, and a Jest one for SeatReservationService

    createMock(SeatReservationService::class);
            $serviceMock->method('reserveSeat')
                        ->willReturn(true);
    
            $controller = new SeatReservationController($serviceMock);
            $this->expectOutputString("Seat reserved successfully.");
            $controller->handleRequest();
        }
    
        public function testSeatNotAvailableException() {
            $serviceMock = $this->createMock(SeatReservationService::class);
            $serviceMock->method('reserveSeat')
                        ->will($this->throwException(new SeatNotAvailable()));
    
            $controller = new SeatReservationController($serviceMock);
            $this->expectOutputString("Seat not available");
            $controller->handleRequest();
        }
    }
    
    const SeatReservationService = require('./SeatReservationService'); // Adjust the path as necessary
    
    test('reserves a seat successfully', async () => {
        const service = new SeatReservationService('https://example.com/api');
        global.fetch = jest.fn(() => 
            Promise.resolve({
                ok: true
            })
        );
    
        await service.reserveSeat('A41', 'Adult');
        expect(fetch).toHaveBeenCalledWith(
            'https://example.com/api/reserve', 
            expect.objectContaining({
                method: 'POST',
                headers: {
                    'Content-Type': 'application/json'
                },
            })
        );
    });
    
    test('fails to reserve a seat', async () => {
        const service = new SeatReservationService('https://example.com/api');
        global.fetch = jest.fn(() => 
            Promise.resolve({
                ok: false,
                status: 400
            })
        );
    
        global.console = { error: jest.fn(), log: jest.fn() };
    
        await service.reserveSeat('A42', 'Adult');
        expect(console.error).toHaveBeenCalledWith(
            'Failed to reserve seat:', 
            expect.any(Error)
        );
    });
    
    • @TropicalDingdong
      link
      English
      610 months ago

      In those five or six hours, I estimate I’m able to get two or three days of work done (at my level of productivity before I started paying $20/month for ChatGPT+).

      This is very comparable to the ROI I would say that I’ve been seeing for my programming work. I feel like a super hero, or a 22 year old on adderall. I know everything I need to do for any project, but between client meetings, executive meetings, business development, meetings with the product team, mentoring, the actual amount of focused time I get is so little. I can offload a huge amount of the “I know how to do this and I’ll know if you do it wrong, but please do this for me” to the machine. This past May I took on a task that would have taken a comparable person, probably 6 months, and I knocked it out in 2.5 weeks. If you already know what you are doing, ChatGPT is steroids.