[Dev] Let's talk about Infrastructure and Hosting

[Andreas Grapentin]

Hi,

As you know, a good chunk of the infrastructure that makes parabola is
currently hosted on a virtual machine called winston, which is
generously provided by the hosting provider 1984. We used to have proton
as well, but when that died, the things that it hosted were migrated
over to winston.

It's come to our attention that this machine is currently struggling to
cope with the load it has to bear, as is evident by the fact that it is
as of this writing at ~70% swap utilization, and requires more and more
frequent reboots just to keep our basic infrastructure operational. As
you can probably imagine, this is not a healthy mode of operation for a
distribution, and is something that is eating a bunch of hacker time to
keep on top of.

These developments have prompted freemor, bill-auger and me to keep an
eye open for other hosting options that would maybe give our services
and bit more breathing room, and maybe some space to grow and expand.

bill-auger had floated the idea to get in touch with the people over at
vikings (vikings.net) -- in CC -- who offer (among other things) hosting
services on exclusively libre-friendly server hardware, which would be
perfect for parabola. We did already have a brief chat with their team
to figure out what their available hosting options are, and now we need
to discuss amongs ourselves what sort of infrastructure we want going
forwards.

I think it would make sense to start by listing all the services that
the new infrastructure should host, and estimating from there what sort
of physical or virtual machine or machines we need, and how we
distribute the services.

Off the top of my head, these are the services that I know we run:

  - webserver for parabola.nu, including
     - wiki
     - main website
     - git repositories
     - issue tracker (redmine?)
     - forum
  - Tier 0 package repositories
     - repo importer
  - mailman

But I'm probably missing a bunch of things. This is where I'd like
someone more experienced with our infrastructure to step in and explain
what we have and need. I would suggest we start maintaining this
information in the wiki as well, assuming no such article exists
already. This might come in handy down the line.

My hunch as a Software Engineer and Architect would be that it could
make sense to partition these services in two, or maybe three VMs, just
to have a more fine-grained control over fault tolerance and recovery in
case something does go belly-up, and to be able to separate the critical
from the non-critical infrastructure. But this is something we need to
discuss down the line, if we have a clear idea about our service
landscape.

While we do this, I would also suggest we take another look at which
part of our infrastructure could and / or should be converted to
packages, instead of unaccounted for files. We could conceivably add a
separate undocumented repo for server-only packages. And while we're at
that, I would also like to take a gander at automatic provisioners, such
as ansible, that would allow us to describe our server roles and then
automatically deploy them as needed, to have a self-documenting
deployment of our services.

But first, let's complete the list of services that parabola needs to
function, and figure out how we want to partition these and what sort of
resources they need. Everything beyond that is still far away. :)


I have to say; I am personally very excited to see these changes unfold.
I believe that parabola is currently at a place where we need to
stabilize our infrastructure and establish ourselves as a serious
distribution. And a collaboration with vikings could be a great chance
to take a leap toward that direction.

However, I also appreciate that parabola is an adhocracy, and that each
of you has their own unique views on these issues and options. If you
have any concerns about these developments, please bring them to our
attention as well.

I'm looking forwards to everyones input.

Best,
Andreas

oaken-source



-- 

------------------------------------------------------------------------------
my GPG Public Key:                 https://files.grapentin.org/.gpg/public.key
------------------------------------------------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <https://lists.parabola.nu/pipermail/dev/attachments/20191105/08167f7d/attachment.sig>