Threads Email Ingestion – a primer

This document explains a little about the message ingestion mechanisms of Threads – or, more simply, how Threads gets hold of messages.

In Threads, we term any sort of digital interaction between two or more people as a message. Here we discuss just one particular type of message, the email. The company or organisation that uses Threads we call the subscriber. Users are those people authorised to use Threads.

There are essentially 3 ways Threads can ingest email, Pulling, Pushing and by Transparent Proxy.

1 Pull Ingestion
Threads logs into the subscriber’s email server as though it was the user email client, and reads or pulls down messages. Threads can ingest from any arbitrary number of email accounts so, for a small number of users, it would be practical to ingest from each of them in turn.

Threads_pull.jpg
Advantages:

Easiest to set up
Requires no involvement of IT department
Useful for ingesting from ad-hoc accounts set up for specific projects
Disadvantages:

Requires disclosing account authentication details
May get interrupted due to automatic password changes
Inefficient because Threads must scan all message to avoid re-ingesting messages

2 Push Ingestion
Threads logs into an intermediate email account – either an email account provided by Threads, or an email account nominated by the subscriber – to collect messages copied/uploaded or pushed by the user/subscriber. Once read from the intermediate account, Threads deletes the email. The same intermediate account can be used to aggregate emails pushed from many different sources.

Threads_push.jpg
It should be realised that copying email into an intermediate account is not the same as forwarding it. When a mail is copied from one account to another, all the header information remains unaltered, whereas this is not the case with a forwarded email. It is essential that the email header information stays intact, otherwise Threads cannot file it correctly.

The method of pushing emails can vary according to the user and subscriber’s circumstances.

2.1 PUSHING FROM A USER CLIENT
To push mail from a user’s email client, it is necessary to add the intermediate account details to the email client software (eg to Microsoft Outlook or MacOS Mail). Once this is done, emails can be copied manually (eg option drag) or by setting up a rule to do so automatically.

Advantages:

Requires no involvement of IT department
Can use rules to control or filter emails pushed
Disadvantages:

More involved to set up
Only operates with the user is logged into email account
Manual moving of emails is subject to human error
2.2 PUSHING FROM THE SUBSCRIBER’S SERVER
To push mail from a user’s email server requires cooperation from the subscriber’s IT manager or a user with administrator privileges. It involves setting up a server-based rule to copy messages to the Threads intermediate account. As with client-based rules, these can be quite sophisticated and filter the emails that are pushed to Threads.

Advantages:

Automatically applies to all registered users
Rules control/filter emails pushed
Highly efficient
Disadvantages:

More involved to set up
Requires administrator privileges on email server
3 Transparent Proxy
This method is similar to that used by most spam filtering services. In spam filters the address of the subscriber’s domain for handling email is substituted by the domain of the service provider so that all incoming email goes to the service provider first. The spam is extracted and the remainder, passed on to the subscriber.

This must be done with the full cooperation of the subscriber and but would require legal authorisation.

Unless they are technically aware, most company directors do not realise that use spam filtering involves all their company email being intercepted by a third party.

Advantages:

Automatically applies to all registered users
Most efficient way to ingest emais
Disadvantages:

All email is unconditionally sent to Threads
Requires legal authority of the subscriber
If the service fails, then email is cut off