[Dev] Let's talk about Infrastructure and Hosting

[bill-auger]

On Tue, 5 Nov 2019 23:28:36 +0100 Andreas wrote:
> I would suggest we start maintaining
> this information in the wiki as well

there are dedicated wiki pages for the servers, with each server
having its own sub-article - they have not been updated though,
since everything from proton was moved to winston

https://wiki.parabola.nu/Hacking:Servers

i have added a revised list of services to the wiki for the
purpose of this discussion, so that it can be further revised 

https://wiki.parabola.nu/Hacking:Servers/In-Progress-2019-11

on that list, i sorted the services primarily as
essential/non-essential to suggest that all essential services
remain on winston, with backup instances on another box ready
for emergency use - all those labeled as "essential" have been
on on winston all along, while the web and non-essentials were
split between proton and winston - the web and non-essential
services are the ones we should consider partitioning across
other boxes - the web services especially are the ones most
likely to go crazy with resource usage, and the most likely to be
hassled by bots

lukeshu explained to me that the separation between proton and
winston was such that all clients of mysql were on one box and
all clients of pg were on the other - when we migrated
everything to winston, lukeshu was leery that mixing these could
cause performance problems

i do hope that lukeshu chimes in on this discussion - more than
anyone, he has been the main architect of the parabola
infrastructure, and is still the one who is most familiar with
it's subtleties and quirks


On Tue, 5 Nov 2019 23:28:36 +0100 Andreas wrote:
> which part of our infrastructure could and / or should be
> converted to packages, instead of unaccounted for files.

AFAIK there is very little running on winston that is not
published publicly already - most of the web services are not
packages because they were forked from arch or the upstream as
git repos; and our patches are in that form - i think mailman
and cgit are the only web services that are managed by pacman -
most internal services are pacman packages, and most of the
custom tools are packaged from code in the git repos


On Tue, 5 Nov 2019 23:28:36 +0100 Andreas wrote:
> like to take a gander at automatic provisioners, such as
> ansible,

a while back lukeshu put some things under the holo
configuration management system - that probably the part of
the system that i know the least about; but it presumably
does a similar job as ansible - also, etckeeper is managing
arbitrary configuration files; and the git repo manages it's own
metadata via git hooks in a similar way as etckeeper does