This reverts commit b0606d90f0.
This seems to prevent libegl-mesa0 from being installed (which for some
reason isn't failing the Azure jobs - the failure seen later is that
ccache is not installed).
This is to allow using more recent C++20 features in upcoming commits.
Version 3.1.6 is what is installed on Ubuntu 22.10 and works with the
C++20 features we want.
They currently reside under Build/<arch>, meaning that they would be
redownloaded for each architecture/toolchain build combo. Move them to a
location that can be re-used for all builds.