Unfortunately, babelrc.config.js makes things very difficult for Parcel's cache. Since it is a JS file rather than static JSON, it can have dependencies, side effects, read files from the filesystem, return different value every time, etc. This basically makes applications that use it completely non-cacheable. In my opinion, allowing JS in configs was a major design mistake.
I am aware that babel has an API to control cacheability of configs, but it isn't really designed for use in a cache that is serialized to disk. It is designed to avoid re-calling the config function, not to avoid re-compiling files. In addition, it is not enabled by default. The API relies on callback functions to determine if the cache should be invalidated, but that only works when the callback can be stored in memory - we cannot serialize it to disk. So when you restart Parcel, we still need to invalidate the cache for every file and recompute the babelrc.config.js. This will make rebuilds much much slower than you are used to with Parcel.
If we support this, we will need to add custom logic to Parcel to disable caching if babelrc.config.js is seen, and probably warn users about why their builds get much slower. I would recommend just using a .babelrc in the root of your monorepo instead of babelrc.config.js. If you can avoid writing JS configs, do.