[Dev] Let's talk about Infrastructure and Hosting

bill-auger bill-auger at peers.community
Sun Nov 17 02:45:37 GMT 2019

the bandwidth requirements are surely the most difficult to put
a figure on; though we do have some degree of flexibility in
that area, at least for the near future - bandwidth is also the
most costly expense; so it does require the most judicious
attention - with no changes to the current mirrors and
stratification, i would guesstimate that actual increase in
bandwidth demand, above what we are using now, will probably be
roughly 2-3 TiB's per month - thats based on something lukeshu
told me about two years ago - my memory is a bit fuzzy on the
exact numbers; but i do know that we had fewer mirrors at that
time - we should probably contact 1984, to see if we can get a
better estimate of this year's usage

there are a few ways that we can partition bandwidth usage, if
the constraints become too tight, though none are without caveats

if all of the web services were moved to a new server and
winston were reserved exclusively for in-bound publishing and
hosting of the up-to-date package repos as the only tier0, that
would lower the bandwidth requirements for the new server
greatly, at least in the short-term - a very modest bandwidth
server could handle the much lighter web and email traffic, plus
the presumably low demand for sources and repos of
recently-outdated packages for the rare emergency downgrade
situations - if all web services were on one server though, that
would be the one with the large memory requirement

that arrangement would also free a lot of disk space on winston -
probably just enough to fit five complete arches of up-to-date
packages; with winston remaining at its current limits of both
bandwidth and disk space

any way we dice it though, the addition of two new ports will
nearly double parabola's total bandwidth requirement, probably to
beyond the generosity of any single host sponsor; so we may need
to re-organize mirrors into tiers anyways, or to split the tier0
repos across multiple servers, if the new server affords
bandwidth on par with what winston is handling now - though the
latter would complicate libretools and/or the rsync mirroring
protocol - on the other hand, the former would increase the load
on whichever mirrors that would agree to become tier1 - i have no
prediction at the moment regarding which, if any can handle the
extra load

More information about the Dev mailing list