Further troubleshooting I discovered the low voltage output (~3V via USB out) is caused by the battery being nearly empty or physically removed, even when 5V USB in is provided.  This surprised me a little bit, but is workable.  I started wondering if perhaps the resets are caused by insufficient current from the battery and the power module not pulling from the USB input (or solar in a real world deployment), but unfortunately I'm stuck as I'm not sure how I can even test that without buying a bunch of batteries and testing them all.

I should also clarify that I did install the cooling fin.  The unit does certainly generate heat though it never is too hot to touch/handle and best I can tell the resets are not due to overheating, especially because the RPi always automatically reboots.