Skip to content

Onboarding

Onboarding new Tails sysadmins

This document describes the process to include a new person in the Tails sysadmin team.

⚠ This process should become obsolete at some point during the Tails/Tor merge process.

Documentation

Our documentation is stored in this wiki. See our role description as it gives insight on the way we currently organize. Check the pages linked from there for info about services and some important pages in GitLab which we need to keep an eye on.

Security policy

Ensure the new sysadmin complies with our team's security policy (Level B):

Also, see the integration of Tails and TPA security policies in:

Accesses

Once we have the necessary information, there are some steps to do to get the new sysadmin in the team.

OpenPGP

  • Have the new sysadmin to generate an authentication-capable subkey for their OpenPGP key.
  • Have the new sysadmin upload their OpenPGP key, including the authentication subkey, to hkps://hkps.pool.sks-keyservers.net and hkps://keys.openpgp.org; the latter requires an email-based confirmation.

Git repositories

We have a meta-repository that documents all important repositories. During the onboarding process, you should receive a signed copy of the known_hosts file in that repository to bootstrap trust on those SSH servers.

Onboarding steps:

  • Add the new sysadmin to .sysadmins in gitlab-config.git.
  • Add the new sysadmin's SSH public key in the keys directory in gitolite@git.tails.net:gitolite-admin, commit and push.
  • Add the new sysadmin to the @sysadmins variable in conf/gitolite.conf in gitolite@git.tails.net:gitolite-admin, commit and push.
  • Add her OpenPGP key to the list of git-remote-gcrypt recipients for sysadmin.git and update README accordingly.
  • Password store: credentials are stored in TPA's password-store, see onboarding new staff.
  • Send the new sysadmin a signed copy of the known_hosts file that contains the hashes for the SSHd host key for git.tails.net and also share the onboarding info with them.

GitLab

Sysadmin issues are tracked in Torproject's Gitlab

Onboarding steps:

Mailing lists

We currently use the following mailing lists:

  • sysadmins at tails.net, a Schleuder list used for:
    • accounts in external services
    • communication with upstream providers
    • general requests (eg. GitLab accounts, occasional bug reports)
    • cron reports which eventually need acting upon
  • tails-notifications at lists.puscii.nl, used for Icinga2 notifications

Onboarding steps:

Monitoring

We use Icinga2 with the Icingaweb2 web interface. The shared passphrase can be found in the Password Store (see the Git repositories section).

pass tor/services/icingaweb2.tails.boum.org/icingaadmin

Misc

  • Send an email on assembly@tails.net to announce this new sysadmin if not already advertised.
  • Point the new sysadmin to the admin account in pass for https://icingaweb2.tails.net/icingaweb2/
  • Give access to the tails-sysadmins@puscii.nl calendar that we use for self-organizing and publishing info like who's on shift, meetings, sprint dates, etc.

  • Monitoring can be configured in Android using the aNag app with the following configuration:

    • Check "Allow insecure certificate", because the cert doesn't include the onion address. (This can be further improved in the future)
    • Check "Enabled".

SSH and sudo

See [[doc/ssh-jump-host]].

An example SSH config file can be seen here.

All public systems are reachable via the tails.net namespace and, once inside, all private VMs are accessible via their hostnames and FQDNs. TCP forwarding works so you can use any public system as a jumphost.

Physical servers and VMs hosted by third-parties have OOB access, and such instructions can be found in sysadmin-private.git:systems/.

Onboarding steps:

  • Send the new sysadmin the SSH connection information (onion service, port, SSHd host key hashes) for all our systems.
  • For hieradata/common.eyaml and hieradata/node/stone.tails.net.eyaml:
    • Add the user name to the sysadmins entry of rbac::roles.
    • Add the user data to the rbac::users hash, including the new sysadmin's SSH and OpenPGP public keys.
  • Commit these changes to our Puppet manifests repository and push.
  • Check that the new sysadmin can SSH from the mtl-libvirt-01.tails.net virtualization host to VMs hosted there and elsewhere, e.g. misc.mtl-libvirt-01 and isoworker17.mtl-libvirt-02.
  • Ensure the new sysadmin uses a UTF-8 locale when logged into our systems. Otherwise, some Puppet facts (e.g. jenkins_plugins) will return different values, and Puppet will do weird things.
  • Ask tachanka-collective@lists.tachanka.org to add the new sysadmin's OpenPGP key to the access list for ecours' OOB interface.
  • Ask noc@lists.paulla.asso.fr to add the new sysadmin to their OOB interface.
  • Login to service.coloclue.net and add the new sysadmin's SSH key to the OOB interface.