Can I use CRA to make a library in React?

If I have a project using CRA can I somehow reconfigure webpack so it doesn't need an index.html and just export the components to be published to NPM? I couldn't find this anywhere, I just want to know if it's possible and if so how to do it