ejabberd: Enable XEPs

According to https://github.com/iNPUTmice/ComplianceTester, the following XEPs are not enabled in our ejabberd configuration, but should be:

• XEP-0237: Roster Versioning (mod_roster)
• XEP-0313: Message Archive Management (mod_mam)
• XEP-0363: HTTP File Upload (mod_http_upload)
  • Add help text to explain implications of enabling.
  • Add a checkbox to enable/disable.
  • When enabled, configure mod_http_upload with default settings.
  • Configure mod_http_upload_quota with max_days=365, access_hard_quota=1.1GiB, access_soft_quota=1.0GiB.
  • Implement monitoring of storage used.
• XEP-0357: Push Notifications (mod_push)
• XEP-0065: SOCKS5 Bytestreams (mod_proxy65)

Ejabberd bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=851212

mod_roster can be enabled by default in the Debian package. For the other two, we will need to understand the requirements and security/privacy implications.

Created by: mray

Apart from finding sensible defaults: wouldn't it be worth exposing enabed XEPs to Plinth?

Created by: JoKeyser

I agree with both of you; I think we should find sensible, cautious defaults, but ultimately leave the choice to the user, along with a warning about the implications and point to resources if they need more info.

So to get ready for implementation, I propose

1. Expose checkboxes to enable/disable XEPs on ejabberd configuration page, along with explanations.
2. Potentially monitor the size of the mnesia database used by mod_mam.
3. Potentially monitor the used storage for mod_http_upload.

There is a nice mapping of ejabberd configs to XEPs at https://gitlab.com/hanno/ejabberd-config.

Created by: JoKeyser

@jvalleroy, did you manage to test vagrant box's ejabberd with https://github.com/iNPUTmice/ComplianceTester? If so, how? I've somewhat blindly tried by first apt-get install default-jdk and then tried to run it within the vagrant using localhost... but it seems to get stuck, without a clear error message (at least for me):

root@freedombox:/vagrant # java -jar ComplianceTester-0.1.jar testuser@localhost testpassword
Use compliance suite 'Advanced Server Core Compliance Suite' to test localhost

Timeout while waiting on advertised authentication mechanisms.

Use compliance suite 'Advanced Server IM Compliance Suite' to test localhost

Use compliance suite 'Advanced Server Mobile Compliance Suite' to test localhost

Use compliance suite 'Conversations Compliance Suite' to test localhost

Created by: JoKeyser

I've got the ComplianceTester working now for the vagrant image, making further development easier. It's currently a bit annoying since you need to create a new certificate for ejabberd, because java seems to insist that the hostname is in the cert's Common Name, which currently isn't the case in the vagrant image. The steps below assume you're in the Plinth root folder (/vagrant in the virtual image), and run the ComplianceTester from the host machine (another way may be to run it from within vagrant, but then you need to first install java).

First, you need to forward ejabberd's c2s port to the host machine, by adding this line into Vagrantfile, make sure to restart vagrant afterwards:
config.vm.network "forwarded_port", guest: 5222, host: 5222

You'll want the vagrant image also reachable by its default hostname, so add this line to /etc/hosts/:
127.0.0.1 freedombox.localdomain freedombox

Within vagrant, create a new certificate for ejabberd:

root@freedombox:/vagrant# openssl req -x509 -subj '/CN=freedombox' -nodes -newkey rsa:1024 -keyout key.pem -out cert.pem
root@freedombox:/vagrant# cp key.pem /etc/ejabberd/ejabberd.pem 
root@freedombox:/vagrant# cat cert.pem >> /etc/ejabberd/ejabberd.pem
root@freedombox:/vagrant# service ejabberd restart

On the host, you have to add the new, self-signed certificate to a trusted database file, e.g. a local one, secure.ts, to not get confused:

user@host:~/Plinth# keytool -importcert -keystore secure.ts -storepass changeit -file cert.pem

Finally, you can run the ComplianceTester and get some useful output:

user@host:~/Plinth# java -Djavax.net.ssl.trustStorePassword=changeit -Djavax.net.ssl.trustStore=secure.ts -jar ComplianceTester-0.1.jar testuser@freedombox testpassword

Use compliance suite 'Advanced Server Core Compliance Suite' to test freedombox

running XEP-0115: Entity Capabilities…          PASSED
running XEP-0163: Personal Eventing Protocol…           PASSED
passed 2/2

Advanced Server Core Compliance Suite: PASSED


Use compliance suite 'Advanced Server IM Compliance Suite' to test freedombox

running XEP-0115: Entity Capabilities…          PASSED
running XEP-0163: Personal Eventing Protocol…           PASSED
running Roster Versioning…              PASSED
running XEP-0280: Message Carbons…              PASSED
running XEP-0191: Blocking Command…             PASSED
running XEP-0045: Multi-User Chat…              PASSED
running XEP-0198: Stream Management…            PASSED
running XEP-0313: Message Archive Management…           FAILED
passed 7/8

Advanced Server IM Compliance Suite: FAILED


Use compliance suite 'Advanced Server Mobile Compliance Suite' to test freedombox

running XEP-0115: Entity Capabilities…          PASSED
running XEP-0163: Personal Eventing Protocol…           PASSED
running XEP-0198: Stream Management…            PASSED
running XEP-0352: Client State Indication…              PASSED
running XEP-0357: Push Notifications…           FAILED
passed 4/5

Advanced Server Mobile Compliance Suite: FAILED


Use compliance suite 'Conversations Compliance Suite' to test freedombox

Server is ejabberd 16.09
running XEP-0115: Entity Capabilities…          PASSED
running XEP-0163: Personal Eventing Protocol…           PASSED
running Roster Versioning…              PASSED
running XEP-0280: Message Carbons…              PASSED
running XEP-0191: Blocking Command…             PASSED
running XEP-0045: Multi-User Chat…              PASSED
running XEP-0198: Stream Management…            PASSED
running XEP-0313: Message Archive Management…           FAILED
running XEP-0352: Client State Indication…              PASSED
running XEP-0363: HTTP File Upload…             FAILED
running XEP-0065: SOCKS5 Bytestreams (Proxy)…           PASSED
running XEP-0357: Push Notifications…           FAILED
running XEP-0368: SRV records for XMPP over TLS…                FAILED
passed 9/13

Conversations Compliance Suite: FAILED

Created by: JoKeyser

On to the third, XEP-0363: HTTP File Upload. This is the most complicated one; it needs more configuration, including the firewall. That means more defaults - let's discuss them here and document the decision process. I think we also need mod_http_upload_quota, to regulate how much disk space a user can use. I'm afraid the alternative is that it keeps filling until the disk is full.

The core assumption behind my suggestions below is that ejabberd serves only a few users, maybe 10-15 (not hundreds). I think these few users should not be hampered by too little upload space (>1 GiB, if possible). On the other hand, we're targeting small machines, and ejabberd should not unduly interfere with other services (<50% of all space). I'm assuming here at least 32 GiB in the box, so a hard limit of 1.1 GiB per user would result in just above 50% of that machine if all is used.

Maybe some settings need to be exposed in Plinth? Maybe reporting them to the admin is suffcient? Maybe some defaults should be adaptive to the actual machine that is being configured?

Below are the interesting settings I think we should discuss the defaults of. For those I don't mention, I think ejabberd's defaults clearly make sense. See https://docs.ejabberd.im/admin/configuration/#mod-http-upload for the details. Those of mod_http_upload_quota are just below.

Drafted defaults for mod_http_upload:

• max_size: Maximal size of a single file. Ejabberd's default is 100 MiB. Seems fine?
• docroot: Where to place the files. Ejabberd's default is on root partition. Seems fine, but we could give the admin a choice, in case they connect another disk?

Drafted defaults for mod_http_upload_quota:

• max_days: Number of days until files are deleted. Ejabberd's default is infinity. I'd rather like something finite, like 365?
• access_hard_quota: Hard limit per user. If exceeded, oldest files get erased. Ejabberd has no default. I suggest 1.1 GiB.
• access_soft_quota: Soft limit per user. Unclear to me what happens if exceeded. Ejabberd has no default. I suggest 1.0 GiB.

Created by: JoKeyser

A rather random item for context: The maximal upload size of a single file in Matrix Synapse is also 100MiB per default. So really seems fine.

Hi, Newer ejabberd version in Buster 18.12.1-2 uses port 5443 for Bosh,ws,api and Http upload using single ejabberd_http module.

 - port: 5443
  ip: "::"
  module: ejabberd_http
  request_handlers:
    "/api": mod_http_api
    "/bosh": mod_bosh
    ## "/upload": mod_http_upload
    "/ws": ejabberd_http_ws
    ## captcha: true

Newer installations do not need port 5280 open in the firewall. Older installations which upgrade to the newer version will need it though.

Also, XEP-0357: Push Notifications are now supported in ejabberd.

changed the description

changed the description

added Help Wanted To Do labels and removed Question label

marked the checklist item XEP-0357: Push Notifications (mod_push) as completed

changed the description

Isn't XEP-0357 Push Notifications (mod_push) not actually the support for the centralized push services? Sounds like it to me.

XEP-0060: Publish-Subscribe for decentralized operation? https://xmpp.org/extensions/xep-0060.html

changed the description

I listed some tasks for enabling mod_http_upload.

added Doing label and removed To Do label

marked the checklist item XEP-0363: HTTP File Upload (mod_http_upload) as completed

removed time estimate

added To Do label and removed Doing label