I would personally go for 5 series, but that's because I like the size and practicality. As much as I'd love to own M3, it's totally not practical for my daily use. Same as you, I'm not tracking my car, so even my 535 doesn't get used anywhere to it's full potential, I would imagine it'd be the same or worse with M3.
You on the other hand already have large vehicle (truck) and already know that you love M3. I think chances are high you'd be missing it too much.
Impossible to predict, just guessing here.
