Maybe, but why not do one hobby for work then have other hobbies that you do for fun? I would much rather grow to kind of hate pottery at my pottery business and still play an instrument and bake pastries in my free time than despair in an IT job and bake pastries and do pottery in my free time and figure that instrument dream will just never happen.