Threads Feature Log

2016.1 Network Call Ingestion/Threads Telephony Appliance

Threads originally ingested phone calls by means of a process we called PBX Call Ingestion that involved custom software specific to the subscriber’s telephone system (PBX). The disadvantage of PBX Call Ingestion was that different software was needed for every different phone system – of which there are many.

With Network Call Ingestion, phone call traffic is intercepted directly from the subscriber’s network and so is independent of the phone system. It is necessary to have a small low-cost computer (PC) connected to the subscriber’s network to collect the calls – we call this the Threads Telephony Appliance – and is available through a Threads channel partner.

2016.2 Automated Account Setup and monitoring

A new web application called the Threads Manager allows new subscribers to create a Threads account, specify payment details and later view storage usage. Previously, this had to be done by JPY.

2016.3 Free Email Ingestion Account

In conjunction with the automated account setup, the Threads Manager creates an email account from which Threads can ingest messages. This obviates the need for the subscriber to create an email account or to provide authentication details of existing accounts.

2016.4 Merging of companies

This allows users to combine multiple company records into a single record and hence simplify company management. In Threads, a company may have multiple physical addresses and it is much easier to have one company record for each legal entity. This is particularly useful when importing company records from legacy CRM systems.

2016.5 Merging of contacts

This allow users to combine multiple contact records into a single record and hence simplify contact management. In Threads, a contact may have multiple contact channel addresses and it is simpler to have one contact record for each real person. This is particularly useful when importing contact records from legacy CRM systems.

2016.6 Threads Enron Database Update (TED)

TED is a read-only version of Threads. Updates to Threads are not implicitly incorporated into TED. This has been updated to reflect the last 2016 version of Threads.

2017.1 File/attachment handling

File attachments are viewable in their own grid accessed from a button in the message view. The grid renders each file as a thumbnail which may be magnified or downloaded. The attachment grid also provides its own search and sort tools. When attachments are ingested, document text is extracted and, in the case of graphic documents, Optical Character Recognition (see 2017.2) is automatically performed to extract text where available. This text is a searchable resource.

2017.2 Optical Character Recognitions – Searching scanned documents

When a scan of a printed document is attached to an email, what is sent is not a compute readable text file like from Word say, but a picture comprising millions of pixels. As such, searching the millions of digital pixels, is of no help in finding a word. What is necessary is to use machine intelligence to work out the shapes of the characters and the words they make up. We call this process Optical Character Recognition or OCR. Threads can now automatically perform OCR once the message and attachment have been ingested.Once this happens, the scanned documents can be searched just like regular email body.

The following is an example of a relatively low quality scan sent as an attachment to an email…

And here is the resulting OCR text, which can be searched. Without OCR, if a user searched for the phrase “Door & Window Shop” for example, this message would not be found.

OCR involves a significant amount of computation which may take some minutes to complete. However, since a user typically does not search for a message for days or weeks after it has been ingested, this is not generally an issue.

2017.3 Automatic Speech Recognition – Searching and transcribing phone calls

Previously, Threads’ users could search, sort and classify phone call from associated metadata such as participants, telephone numbers, time and duration of calls. This was extremely useful in finding calls and seeing them in the context of other messages but it did not permit users to access the content of the call, except by listening to it. Automatic Speech Recognition (ASR) analyses the acoustic waveform and endeavours to transcribe it into text, just as a human would. By so doing, the call can be searched for keywords.

Because Threads ingests calls directly from the network (see 2016.1) rather than from the phone system, it is generally able to pre-process the conversation and hence provide much better accuracy than might be achieved otherwise.

We provide two viewing options for call transcriptions. The Dialogue View shows each caller speech in a speech bubble. This makes it very easy to follow the conversation, and to scrub through a long conversation, isolating the section of interest.

The Block View segments each speaker’s text into one continuous block per speaker. Although this is difficult to follow, it is very helpful if the user wishes to cut and paste sections of a call transcription.

Just like OCR, ASR involves a significant amount of computation which may take some minutes to complete. However again like OCR, since a user typically does not search for a (phone call) message for days or weeks after it has been ingested, this is not generally an issue.

2017.4 Automatic company creation

Prior to this feature, when messages were exchanged between companies not already in the Threads database, Threads created contacts without a parent company – we call these orphan contacts. However, it is often useful to thread together several messages between the same company, even if the company has not yet been identified by a user or imported from an existing address book. With automatic company creation, any message ingested involving an email channel with a domain that is not known to Threads and not generic (i.e. not gmail.com, yahoo.com, etc), will cause the automatic creation of a new company, using the domain name as the company name. For example and email from say, “fred@widgets.com” will result in the creation of a company called “Widgets” related via the contact channel value “fred@widgets.com” to a contact called “Fred”. So called orphan contacts, are now only be used for contact channels with generic domains.

2017.5 Private/shared contact channels

Previously, messages (emails and phone calls) from unknown contacts were visible only to the addressed user. This prevented private messages from being shared. However, once the contact is linked to a known company, then their messages may become shared (depending on admin preferences). This meant that users were discouraged from entering the company details of non-work-related contacts. Private/shared contact channels allow users to selectively decide which contacts are personal, so that even if the company details are entered, specific contacts can still be identified as private.

This feature has involved a fundamental redesign of architecture for filtering messages. The previous “traffic light” filters have changed thus:

“Contacts I know”                 is now                 “Known shared Contacts”

“Contacts I might know”         is now                “Tentative [shared/private] Contacts”

“Contacts I don’t know”        is now                “Known private Contacts”

If a message is exchanged using a previously unknown (to Threads) contact channel, then Threads will create a new contact channel and automatically decide whether the contact channel should be shared or private, dependent upon some rules and admin preferences. However, because this has not involved a human decision, then the channel is marked as “tentative”, so users can immediately see whether a message involves new contact channels that need to be confirmed. Once the user has manually confirmed or changed the status of channel, then it is is no longer tentative.

2017.6 Google speech

It has always been a design goal to offer Threads’s subscribers to the best message processing services available. Threads subscribers can choose based on performance and price. With the introduction of automatic speech recognition (ASR) to Threads we offered the Speechmatics engine. Since then, Google has introduced its speech recognition API and we have now completed the integration into Threads. Although most subscribers will choose just one speech recognition engine, we have added the option to switch between engines to compare transcriptions (see 2017.7)

2017.7 Printable transcriptions

Threads uniquely displays ASR transcriptions in an html rendition of a typical SMS display format, highlighting spoken words and allowing users to scrub through transcriptions. This user interface greatly enhances the user experience and makes it very easy to locate and isolate sections of a call. We have added the ability to print transcriptions. In the screenshots below, we show the Google Speech and Speechmatics transcriptions for the same call.  The print icon appears next to the “Show Speechmatics version”…

2018.1 Contextual changes to channel privacy settings

It is often necessary to change the default status of a contact channel from “private” to “shared” and vice versa (see 2017.5). Previously, this necessitated drilling into the contact record and selecting the appropriate privacy. This feature allows the user to change the privacy of a channel directly from the message list. To use this feature, first expand the with the chevron button and then hover the mouse pointer over any contact showing a tentative channel (amber blob). This will render a pop-up menu from which private or shared may be selected.

 

2018.2 Dynamic deployment of speech recognition services

A design goal of Threads is to support as many automatic speech recognition systems as are commercially available. We have begun to roll this out with services from Speechmatics and Google Speech (2017.5).

Although most subscribers will wish to subscribe to only one service, many will wish to experiment with various services to decides which performs most cost-effectively.

To assist this, we allow users to  subscribe to several services and choose among transcriptions available. Subscribers can disable services selectively from the Administrator Preferences…