You can eject the app to exit CRA's umbrella. All of your configuration and code still works, you just also gain access to the project config and Webpack config that CRA formally dealt with for you.

A lot of the time, simply using react-app-rewired is sufficient in place of ejecting, especially if the only thing you need are some Webpack plugins.