That's mainly a thing of avoiding foreign object damage on the engines.
Since the reverse thrust airflow cycle with the "brutally" separated airflow of the bypass stream behaves differently to the normal thrust one, this is a thing to be taken into account when running those large engines above unpaved surfaces.

I think a video of a landing on a wet runway pretty much shows the way the airflow tends to blow things in front of the engine where it partially gets sucked in again.
So while you can keep a runway surface rather clean, the unpaved strips are a more unsafe factor.

As said, it's not much of a problem for the plane to use those two engines only or even go spoiler + brakes only.
With the spoiler's increased angle the weight on the wheels increases and therefore you can increase the break pressure too. I wouldn't call breaking this thing an easy job, but if you have plenty of big wheels and plenty of weight forcing them on the ground, the only limits you may run into are temperature based ones, since the break apparatus itself can take the load and also provide enough deceleration then.
The brake to vacate feature then takes care that you are using the runway up to the predicted taxiway for leaving it, so it adjusts the brake pressure to match that point.

The other avoidance from the Airbus side was a cost and weight factor I think.
The first years of promoting the big wale even had concepts where no reversers at all were fitted.

The cost factor was one for the airlines too and is always connected to the expected higher break wear of course, so one could see the '2 out of 4' reverser outcome as a tradeoff solution.