Skip to content

Instantly share code, notes, and snippets.

@attacus
Last active March 13, 2024 00:16
Show Gist options
  • Save attacus/cb5c8a53380ca755b10a5b37a636a0b9 to your computer and use it in GitHub Desktop.
Save attacus/cb5c8a53380ca755b10a5b37a636a0b9 to your computer and use it in GitHub Desktop.
Create your own encrypted chat server with Riot and Matrix

This guide is unmaintained and was created for a specific workshop in 2017. It remains as a legacy reference. Use at your own risk.

Running your own encrypted chat service with Matrix and Riot

Workshop Instructor:

This workshop is distributed under a CC BY-SA 4.0 license.

What are we doing here?

The goal of this workshop is to teach you how to configure and run your own Matrix/Riot service. By the end of the workshop, you should be able to log into secure chat rooms and invite others to the same server.

Don't panic if you don't finish all of these steps today. You can take these instructions with you and continue/start again at home any time you like.

If you do finish everything within the time we have, try out some of our bonus challenges at the end of the workshop.

What are Matrix and Riot?

Disclaimer: the instructors are not getting paid by anyone or receiving any incentives to say any of this. We're telling you about it because we like it, and because we think you might find it useful.

Matrix and Riot work together to provide a chat service which behaves in a similar way to popular services like Slack. Unlike Slack, however, this tech stack is free and open source. This means you, your team, or your company can easily host your own Matrix servers so that all information that passes through there can remain within the control of your organisation instead of a third party. If you are frequently sharing sensitive things like passwords, internal URLs, or business information as part of your work - this workshop is for you!

With Matrix and Riot, you can run chatbots, integrate Giphy, create new channels for different topics, or start one-to-one conversations. You can also configure audio and video calling. The stack features Slack and IRC integration for those who don’t want to move (there’s always at least one: https://xkcd.com/1782/).

Matrix also provides native support for encrypted chat rooms. If you set it up, chat channels on your server can be end-to-end encrypted. Anyone on your server can verify their fingerprints with each other out-of-channel to make sure that the people in the room are the ones you want there.

Please remember that, as with any encrypted messaging service, the room is only as secure as the people in it. Encryption isn't magic. Don't write anything you wouldn't want to see your name against publicly.

Pre-requisites:

  • A computer
  • Internet connection (BYO or the venue wifi. If you're using the venue wifi, please be considerate.)
  • Basic knowledge of the Linux terminal, SSH, and command line text editors
  • Riot client installed on your laptop or phone (get it at https://riot.im/)

Optional pre-requisites

If you want to take your work home with you, you'll also need to supply:

  • Your own domain name
  • VM running Debian 8 on a cloud service

Installation Guide

DNS settings

Note: if you just want to learn how Matrix and Riot work today and don't want to mess around with DNS config just yet, ask one of the instructors for temporary domain details, and skip straight to "Installing the Matrix server".

Please remember that all temporary workshop domains and VMs will not live long after the workshop; to run your own service in the real world, you'll need to set up your own infrastructure.

Register a domain

If you don't already have a domain you want to use for this, you can buy one through any domain registrar. If you didn’t set one up before the workshop, it’s going to take too much setup time right now to work with it in this session, so go ahead and ask for one of our temporary hostnames and come back to this later.

(If you don’t have a registrar of choice, we recommend serversaurus.com.au - they're awesome local folk who care about privacy. Again, we are not being paid to say that, they’re just nice people.)

Add DNS records to your DNS panel

e.g.:

server01.securechatworkshop.com. | 300    IN    | A    | 1.2.3.4
_matrix._tcp.server01.securechatworkshop.com. |    300 IN | SRV | 10 0 443 server01.securechatworkshop.com.

Installing the Matrix server

The following guide will set up Synapse, which is Matrix's homeserver implementation.

Disclaimer

If you're using one of our VMs, please be considerate. We're hosting you so you can learn, and so we don't have to spend the first hour of this workshop copying a VM onto everyone's laptop with a USB stick. Please do not use our VMs for anything other than completion of the workshop.

Preparing the machine

First: launch your Debian 8 VM, and SSH in. If you didn't bring your own, you can use one of our temp VMs. Ask instructors for details.

You'll need to be a privileged user to run most of these commands. You can do this by running sudo -i.

We would also recommend typing in your commands directly instead of copying and pasting from here, to reduce the risk of encoding errors (and because pasting in commands from a website on the Internet is generally not good practice!).

The Matrix/Synapse package lives in a non-standard repository. We're going to add this repository to our machine's list of sources:

echo 'deb http://ftp.debian.org/debian jessie-backports main' >> /etc/apt/sources.list

And then we're going to make sure the machine knows that the repo is there:

apt-get update && apt-get dist-upgrade -y

Next, we need to install a few packages that will help us later. Our VMs are very barebones by default. Run the following:

apt-get install -y apt-transport-https lsof curl python python-pip
apt-get install -y certbot -t jessie-backports

At this point we need to add a configuration file to ensure that Matrix knows where to read from. The file needs to be called /etc/apt/sources.list.d/matrix.list

Open this up in your favourite text editor. We are an equal opportunity workshop and take no sides in the text editor wars. If you don't have an editor of choice and want some guidance, ask one of the instructors (or the person next to you) for a recommendation.

Inside /etc/apt/sources.list.d/matrix.list, add the following two lines:

deb https://matrix.org/packages/debian/ jessie main
deb-src https://matrix.org/packages/debian/ jessie main

Installing Matrix

With that out of the way, it's time to actually install Matrix. Run the following:

curl https://matrix.org/packages/debian/repo-key.asc | apt-key add -
apt-get update
apt-get install matrix-synapse -y

Warning:

You might get a package error at this point which will cause the matrix-synapse install to fail. If this is the case, we'll install Aptitude (an alternative Debian package manager) to help us resolve the conflict semi-painlessly.

Run apt-get install aptitude.

Install the python-cffi package with Aptitude: aptitude install python-cffi

When Aptitude offers to resolve the conflict, accept the solution which says:

Install the following packages:
1)  python-cffi-backend [1.9.1-2~bpo8+1 (jessie-backports, now) -> 1.4.2-2~bp

This will be the second or third option Aptitude gives you. Choose "no" until you see this solution offered, and continue with this installation of python-cffi.

Once that's done, let's try the Matrix install again:

apt-get install matrix-synapse

If it's still not working, we suggest a sacrifice to the apt-get gods, or asking an instructor or the person next to you for help.

Configuring the Matrix server

The installation process requires some basic config.

You will be asked to provide a hostname for your server. If you didn't configure your own hostname earlier, use the hostname from the handouts. (e.g. server01.securechatworkshop.com)

If asked for reporting anonymous stats, choose ‘no’. Nobody wants that.

Then, start your server:

systemctl start matrix-synapse

Adding encryption support

It's time to encrypt this sucker.

Use certbot to generate a Let's Encrypt certificate. Read more about it here if you're curious: https://certbot.eff.org/

Run: certbot certonly (add the --register-unsafely-without-email flag if you want. Don't use this flag in prod - but for the purposes of this workshop, this is only a temporary server, and you probably don't want the spam).

Choose the "spin up a temporary webserver" option.

Configuring nginx

To make this thing truly HTTPS-ready, we need to configure a reverse proxy. We'll use nginx for this, so install it:

apt-get install nginx -y

Then add the following configuration to /etc/nginx/conf.d/matrix.conf:

server {
    listen 443 ssl;
    server_name matrix.securechatworkshop.com;

    ssl_certificate     /etc/letsencrypt/live/matrix.securechatworkshop.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/matrix.securechatworkshop.com/privkey.pem;
    ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers         HIGH:!aNULL:!MD5;

    location /_matrix {
        proxy_pass http://localhost:8008;
        proxy_set_header X-Forwarded-For $remote_addr;
    }
}

Make sure you replace matrix.securechatworkshop.com with the relevant server name.

Once that's saved, restart nginx by running: systemctl restart nginx

Fine-tuning Synapse

Add a shared secret to the config file at /etc/matrix-synapse/homeserver.yaml:

registration_shared_secret: <add random characters here, whatever you want your secret to be>

In that same config file, we also need to comment out the following:

#trusted_third_party_id_servers:
#    - matrix.org
#    - vector.im

Synapse caches conversation information in RAM where possible, and will use as much as you give it. For small implementations, (>50 users), you probably only need about 512MB of RAM. You can configure this by adding an environment variable to the following file: /etc/default/matrix-synapse:

SYNAPSE_CACHE_FACTOR 0.02

And to make sure it all takes, restart the service:

systemctl restart matrix-synapse

Register the first Matrix user

One of the things you probably want out of this chat server is to, y'know, chat with people. To do that, we need some user accounts, starting with your own.

Create a new user by running the following, and answering the prompts:

register_new_matrix_user -c /etc/matrix-synapse/homeserver.yaml https://localhost

- New user localpart [root]: {add your name/handle here}
- Password:
- Confirm password:
- Make admin [no]: yes
- Sending registration request…
- Success.

Optional: to save having to register new users via CLI on your server every time, you can enable GUI user registration through the Riot client by editing /etc/matrix-synapse/homeserver.yaml and changing the following setting:

enable_registration: true

Otherwise, to register additional users, run register_new_matrix_user -c /etc/matrix-synapse/homeserver.yaml https://localhost again to manually configure more accounts.

Don't make them all admins!

Time to Riot

Riot is the fancypants front-end client for the server we just set up.

If you don't have it already, you can download the app for your OS of choice at https://riot.im/

One you have it, run it.

Riot may try to auto-connect you to their default servers. If this happens, log out. We want the Riot login screen for the next part.

Let's connect Riot to the server we just configured. Add your hostname (either your BYO hostname, or the here's-what-we-prepared-earlier hostname on your handout):

Home server URL: (e.g. server01.securechatworkshop.com)

Identity server URL: (e.g. server01.securechatworkshop.com)

Now log in with the user you configured in your server in the previous section of this doc.

Create a new secure room

  • Click on the gear icon and turn on end-to-end encryption by ticking ‘Enable encryption’
  • Click ‘Save’

Security checkup

  • Who can access this room? -> Only people who have been invited (default)
  • Who can read history? -> Members only (since they joined)
  • URL previews -> Disable URL previews
  • To invite users in the room -> Moderator

More config

There's a bunch of stuff you can do with the Riot client. Explore the interface.

Here are some things to try:

  • Invite your friends to the room you just created
    • compare key fingerprints before chatting in encrypted rooms
  • Have someone else create a room (or prevent someone else from creating a room)
  • Integrate Giphy for maximum lulz
  • Add a GitHub bot
  • Enable voice and video calling
  • Try the mobile client

Additional things to do after the workshop

There's server config we skipped over because this is a pretty short workshop.

We'd recommend going back to your server and doing some of these things later if you actually want to use Matrix/Riot properly.

  • Deploy a firewall with iptables
  • Configure auto-update (unattended-upgrades)
  • Protect your Debian server with two-factor authentication
  • Replace sqlite database with psql (if you expect lots of users): https://github.com/matrix-org/synapse/blob/master/docs/postgres.rst
  • Configure email notifications (enable_notifs) - beware, may leak sensitive data!
  • Add room integrations
    • GitHub bot
    • RSS bot
    • Giphy
  • Add TURN support for audio/video calls (this is also not 100% secure, use with caution)
  • Set up certificate auto-renewal for Let's Encrypt by running crontab -e and inserting the following line: @daily certbot renew --quiet --post-hook "systemctl reload nginx"

You can find all of the docs you could ever need (and the Matrix community itself) right here: https://matrix.org/docs/guides/faq.html#servers

Post-event infrastructure

To set up a Matrix server on your own infrastructure, you will need to provide your own domain name and your own server. We used AWS EC2 t2.micro instances as the servers for our workshop today - you may need something bigger if you have a large organisation to host.

After you buy a domain, configure your DNS settings as outlined in the "Add DNS records" section above, and then add these into your server's config in place of our temporary ones in the guide. Don't forget to generate another SSL cert through Let's Encrypt for your new hostname.

Make sure to restart Matrix after you're done making config changes, or they won't work properly.

Happy chatting!

Credits

This workshop was originally developed for the B-Sides Canberra 2017 conference by Lilly Ryan and Gabor Szathmari.

@viraj2252
Copy link

Very usefull

Copy link

ghost commented Sep 29, 2017

These are fantastic instructions, I was amazed at how quickly everyone got through the exercise!
I've been playing around with a server a little since AppSecDay and found the following handy for the super paranoid:
With the certbot command, I used:
certbot certonly --hsts
This allows me to force HSTS which I think is good.
The other thing I changed was in the homeserver.yaml file I changed the bind address to be '127.0.0.1' so that I didn't leave unnecessary ports exposed. This was for both the encrypted and unencrypted ports.
Not big changes at all, but rather very small changes.

@atb00ker
Copy link

atb00ker commented Jan 14, 2018

I made an ansible script to preform these actions: https://galaxy.ansible.com/atb00ker/matrix-synapse/
(There are some changes, please read before executing)

@ncamacho97
Copy link

why is it so hard lol not working

@Len-PGH
Copy link

Len-PGH commented Mar 16, 2018

apt-get -t jessie-backports install matrix-synapse -y
This will get past the error and not have to use aptitude

@Subair-tc
Copy link

Any JavaScript SDK for implementing the Riot inside my website ?

@krombel
Copy link

krombel commented Apr 6, 2018

Just to note something:
This configuration will not work for federation.
You mapped federation (via SRV) to port 443 and used there a letsencrypt certificate.
To let it work at least for a time synapse has to be instructed to use the same certificate for federation. If synapse points to the wrong one it tells the other servers a wrong fingerprint which lets them abort further federation.

As this "What certificate do you have?"-Check is cached every time you update the certificate federation will be blocked again until the cache times out.

Another problem: You propose server01.securechatworkshop.com as identity server here although it does not seem so that there is an identity server listening. Although this does not cause to crash it leads to the situation that identity lookups will not be able to be performed.

Just as a reference I would like to link to an issue which mentions some points for a recommended setup:
matrix-org/synapse#2438

@RameshThota87
Copy link

RameshThota87 commented Jun 20, 2018

Hi,
We have Homeserver running on port number 8448(https) and 8008(http)
And we installed Matrix-Appservice-Slack with the port number 8181

We tried to execute the Slack bridge setup with the following command,
Start the actual application service. You can use forever
node app.js -c config.yaml -p $MATRIX_PORT

we did not provide any user credentials. But we are facing the following issue.

Loading config file /home/user/matrix-appservice-slack/config/config.yaml
Matrix-side listening on port 8182
Slack-side listening on port 8181 over http

Accepting pending invites
[-] GET https://matrix.XXXXXX.com/_matrix/client/r0/sync (AS) Body:
[-] GET https://matrix.XXXXXX.com/_matrix/client/r0/sync (AS) HTTP 401 Error: {"errcode":"M_UNKNOWN_TOKEN","error":"Unrecognised access token."}
Unhandled rejection M_UNKNOWN_TOKEN: Unrecognised access token.

I have few queries:

  • Is Home server(HS) and Application server (AS) are in same server or on different?
  • Are both ports required to be accessible in public network?

Please help.

Thanks in advance.

@warp1337
Copy link

Great workshop

@aaronraimist
Copy link

aaronraimist commented Oct 21, 2018

The advice in here on SYNAPSE_CACHE_FACTOR is pretty bad. You probably want to actually increase the cache factor from the default.
matrix-org/synapse#3939
matrix-org/synapse#3876

You also almost certainly want to use Postgres even if you have only one person on the home server. matrix-org/synapse#2917

You probably don't want to mess with registration_shared_secret matrix-org/synapse#2885

I highly recommend using something like https://github.com/spantaleev/matrix-docker-ansible-deploy instead of this guide.
(or wait for matrix-org/matrix.org#206 which will hopefully be written very soon)

@axelsimon
Copy link

I suggest updating this guide to install the Python 3 version of Synapse, matrix-synapse-py3, rather than the less efficient, Python 2-based matrix-synapse.

Also, I think it's recommend to use apt now than apt-get. This has the benefit of being a slightly simpler command name to remember and is visually a bit nicer, both of which are helpful to newcomers.

@stevenuba
Copy link

Hello, I am new on debian, I am having issues by adding the repo echo 'deb http://ftp.debian.org/debian jessie-backports main' >> /etc/apt/sources.list

Console says permission denied. I found on a forum that I can edit the sources.list by using a text editor so I did that but when I run apt-get update && apt-get dist-upgrade -y it says:

W: GPG error: https://matrix.org jessie InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY AD0592FE47F0DF61
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?

So finally I cant install apt-transport-https lsof curl python python-pip
and certbot -t jessie-backports

I am running a debian 8 on aws lightsail

@salteggs
Copy link

salteggs commented Jul 8, 2019

Jessie backports is now giving a 404 Not Found. I did some research, and found that Debian Jessie is no longer supporting backports, see discussion in the maillist here:

[email protected]>  With jessie entering LTS we are following
up with shutting down
> jessie-backports (and jessie-backports-sloppy).  We deprecated LTS
> support for backports already with the release of stretch, and a thread
> that we started on debian-backports about it didn't raise any new ideas
> how to make this working, so if you are using backports on jessie we
> suggest you to take this as a strong recommendation to finally do the
> upgrade to stretch.

Any chance on updating the documentation for a newer release (or CentOS) ?

@HackerTutor
Copy link

@salteggs I am also recieving this error. I think this guide is outdated now.

@attacus
Copy link
Author

attacus commented Jul 19, 2019

I’ve updated the gist to note that this guide is not maintained.

@BullShark
Copy link

No instructions for CentOS?

@Guo2006
Copy link

Guo2006 commented Feb 3, 2020

No instructions for CentOS?

I think you should better use Ubuntu 18. I had a bad experience with centos and synapse. Think we should choose operating system according to our tasks.

@imaistecnologia
Copy link

I did some tests installing the matrix on ubuntu! But I can only enable in app mode! In the browser, you get a static matrix page! Is it possible to enable client mode in the browser, with the server itself?

@ravindrsharma99
Copy link

Can someone tell me where i can find server-side code for SSO login?

@laurinivaldi
Copy link

Can someone tell me where i can find server-side code for SSO login?

That'd be wonderful :-)

@BullShark
Copy link

No instructions for CentOS?

I think you should better use Ubuntu 18. I had a bad experience with centos and synapse. Think we should choose operating system according to our tasks.

That's not the case for any other server running in CentOS. I really don't care for Ubuntu. If there's a problem with it running under CentOS, that's a bug that should be dealt with.

Copy link

ghost commented Feb 11, 2021

Some users are really waiting for anonymous nameservers(Tor/I2P)
github.com/matrix-org/synapse/issues/7088

@aitizaz-ul-haq
Copy link

i dont know any thing about adding dns information how to i do that i have a domain
but dont know any thing about setting up the dns configuratoin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment