Economics is dictated by three aspects. Land, labour and capital. The first two are self explainable but the last is more nebulous. Capital is usually related to production, something that can generate value/wealth/goods. An example is machinery in a factory, tractors for farmers and even money itself as it can be invested in stocks. Land also has the potential to be capital, oil, fertile soil and natural resources.

Back then most economic output depended on land, labour and natural capital, all of which can be attained via imperialism. However in the modern era countries are much less dependent on such metrics. Technology has filled up labour shortages, education increases output of individuals and are much less land intensive. Not to mention today a country can import all of its domestic needs.

This is why Japan, south Korea and Taiwan are NOT dirt poor. They have little land, labour or natural capital, but invested in lots of capital, producing more wealth and eventually becoming developed economies.

TLDR. No, you do not need imperialism to grow a capitalist economy