I agree with you. As long as browsers are not implementing the minimum font-size via some property (applied in the UA stylesheet), then there shouldn't be any technical reason why doing what you suggest should be unfeasible.

Minimum font-size does not affect em/rem lengths in computed values on elements. Blink aligned with Firefox for M79 fixing 308862. The reason for fixing it was that a minimum font-size would break a lot of real-world layouts. We discovered that when there was a bug which accidentally wrote a minimum font-size of 6px from the settings UI and we got many reports about it.

The typical failing sites would set the font-size on the html element to a size lower than the minimum font-size expecting rem units for other elements to not be affected by the minimum font-size.

So, I don't think the statement about relative units in the first comment is accurate. At least not for Chrome or Firefox. The width and height of this box is 100px regardless of minimum font-size:

<style>
  div { font-size: 1px; width: 100em; height; 100em; background: green }
</style>
<div></div>

With typical browser defaults, this box would be 100px wide.
If I change my default font size to 24px, the box will be 150px wide.
If I leave my default font size at 16px but change my minimum font size to 24px, the box will remain 100px wide. Thus, text content inside the box will be enlarged but the box will remain the same size. Media queries cannot be used to detect/correct this scenario since they will still be relative to the 16px default font size as well.

<style>
  div { width: 6.25rem; height: 6.25rem; background: green }
</style>
<div>This text should fit comfortably inside</div>

The CSS Working Group just discussed [mediaqueries] Account for minimum font size in relative length media queries, and agreed to the following:

• RESOLVED: Have the MQ resolve their em against the actual default size which includes minimum.

I like that wording! Thanks all!