Most oil is consumed in the transportation sector. So going to EVs and lowering the number of cars is enough to make a huge dent in refined products. Refineries can only lower production to something like 80% of full capacity. Obviously an option is to export the gasoline and diesel, but that increases transport costs. So it will always have a problem in competing with local refineries. Hence they are going to go bankrupt.

As for the workers most of the refineries are in greater LA or the Bay Area. The problem will be the Bakersfield refineries.