I think you’re describing protocols rather than algorithms For a chat program you can look at xmpp, or matrix, both are open protocols for chat apps. You can find info about how they communicate in RFCs or Google “xmpp architecture”

https://en.m.wikipedia.org/wiki/XMPP links to the RFCs for that.

Turnwise and widdershins. I read it in a book once.

Self hosting what?

In general

1. Install docker
2. Copy and paste docker run commands from the getting started guide
3. Learn from mistakes