I use selenium in python with chromedriver to scrape my bank account balances everyday so I don’t have to log into multiple websites. I don’t think Chase likes this and they actively try to stop it with their use of shadow roots, etc., but I’ve still managed to scrape Chase up till now. As of a couple weeks ago, I’m getting a really strange behavior, where if I run the python script there is an element not found error message for an element that clearly is there. Weirdly, if I put a breakpoint in the code and then simply resume execution once it hits the breakpoint then the element is found (by ID) and my script runs, but otherwise it doesn’t. Timers don’t help. I’ve tried sleeping for up to 20 seconds and the element is still not found, but something about hitting a breakpoint in the python script allows the element to be found. Anyone else run into this?
The line is:
element = driver.find_element("id", "requestAccounts")
It sounds like there’s some js running after the page is fully loaded, but you haven’t interacted with the page yet. If there’s something inserting a DOM element in that gray area, then the element wouldn’t be matched correctly. Perhaps try scrolling the page if possible. I haven’t used selenium in a while so I’m not sure how to solve the exact problem you’re seeing.
Oh, wow, that worked. Thanks!