Mail gateway (Mail Processing)

Mail gateway is the system component responsible for receiving, sending, and processing emails related to tickets and process execution.

Mail gateway functions

• Managing the inbound and outbound mail queue (SenderMessage)
• Sending notifications about events
• Sending outbound mail from the message queue
• Loading inbound mail into the message queue
• Storing inbound message processing rules (MailRule)
• Processing inbound messages according to configured rules (MailProcessing)

Mail channels

Mail channels are configured by the system administrator in the Administration (Admin) section.

The channel direction is determined by the specified protocol:

• SMTP — outbound message channel;
• IMAP — inbound message channel.

Attributes

• Name (Name) - String - the channel name displayed in the system. Required attribute.
• Protocol (Proto) - ChannelProto - mail protocol. Possible values:
  • SMTP - outbound message channel;
  • IMAP - inbound message channel.
• Encryption (Encryption) - ChannelEncryption - encryption. Possible values:
  • NONE;
  • SSL;
  • TLS.
• Authentication type (Auth type) — ChannelAuthType — authentication type. Possible values:
  • NONE — no authentication;
  • LOGIN — authentication with username and password;
  • XOAUTH2 — authentication using an access token from the specified OAuth2 configuration.
• OAuth2 configuration (OAuth entry name) — String — OAuth2 configuration name. Required for the XOAUTH2 authentication type.
• Host (Host) — String — mail server hostname or IP address.
• Port (Port) — Integer — server port. Filled in automatically or can be specified manually. Automatic values based on the Protocol and Encryption fields:
  • SMTP, NONE = 25;
  • SMTP, SSL = 465;
  • SMTP, TLS = 587;
  • IMAP, NONE = 143;
  • IMAP, SSL = 993;
  • IMAP, TLS = 143.
• Sender address (Address) — String — the email address substituted into the From: header. Specified only for the SMTP protocol (outbound message channel).
• Username (Username) — String — username. Required for LOGIN and XOAUTH2.
• Password (Password) — String — password. Specified for the LOGIN authentication type.
• Space (Workflow) — String — space (Workflow) identifier. Selected from the list of spaces. Required for the IMAP protocol.
• IMAP folder (IMAP Folder) — String — IMAP protocol only. IMAP folder name. Required attribute. Default value: INBOX.
• Main (Main) — Boolean — SMTP protocol only. A successfully registered main channel is used to send notifications and messages where the channel is not specified.
• Delete received (Delete received) — Boolean — IMAP protocol only. Indicates that after a message is received it must be deleted from the server.
• Debug (Debug) — Boolean — enables debug output for the channel in the system logs.
• Active (Active) — Boolean — allows the channel to be registered in the application memory.
• Registered (Registered) — Boolean — a server-computed flag indicating successful channel registration in the current runtime state. The field is read-only, is not accepted from API clients, and is not stored in the database.

Key principles

Channel registration (Channel Service)

• On startup and on every update, ChannelService loads channel settings but registers only active channels (active = true) that pass validation successfully.
• For LOGIN, a username and password are required. For XOAUTH2, a username and an existing OAuth2 configuration (oauthEntryName) are required.
• OAuth2 configurations are loaded only if at least one active XOAUTH2 channel exists. An OAuth2 load error does not prevent registration of channels with LOGIN and NONE authentication types; XOAUTH2 channels without an available configuration are not registered.
• A new set of senders, receivers, channels, and OAuth2 configurations is first assembled separately and then atomically replaces the current runtime state. Therefore services never see a partially updated set of channels.
• If the initial runtime state cannot be assembled at startup, initialization fails. If a subsequent background refresh fails, the previous working set continues to be used.
• Setting changes are applied to the runtime state on the next background refresh at the interval specified by sender.channel-init-delay-in-millis (by default — every minute), or after an application restart.
• In the channel API list, the registered field shows the actual result of the current registration. active = true allows a registration attempt but does not guarantee registered = true.

Creating an outbound message (Sender Message Factory)

• System components and external services create messages for sending in the message queue (SenderMessage) (attribute direction = OUTBOX).
• If the message specifies channelName, the corresponding registered channel is used. If channelName is not specified, the registered channel with main = true is selected. If there is no registered main channel, creating or sending the message fails.
• The Sender (From) field is filled using the template: senderMessage.sender <channel@address>, where:
  • senderMessage.sender - the sender name specified in the message;
  • channel@address - the email specified in the selected channel settings.
• If senderMessage.sender is empty, the Sender field is set to the value from channel@address.
• The number of send attempts (retries attribute) is set from the configuration parameter sender.retries (default — 3).

Sending an outbound message (Sender Service)

• The message sending service (SenderService) checks the message queue (SenderMessage) at the interval specified by the configuration parameter sender.send-delay-in-millis (by default — every minute).
• Messages with the following attribute values are selected: retries > 0 and completed = false.
• If sending succeeds, the system sets the attribute values: retries = 0, completed = true.
• If sending fails, the system decreases the retries attribute by 1 and stores the error description in the exception field.

Receiving inbound messages (Receiver Service)

• The message receiving service (ReceiverService) obtains the list of registered inbound message channels at the interval specified by the configuration parameter sender.receive-delay-in-millis (by default — every minute).
• For each channel, the service connects to the IMAP server and retrieves messages.
• Retrieved messages are stored in the message queue (SenderMessage) with the following attributes:
  • direction = INBOX;
  • completed = false;
  • raw = the received message in raw format.
• If the channel has the attribute deleteReceived = true, the message is deleted. Otherwise the message is marked as read.

Processing inbound messages (Mail Processing)

After an inbound message is received and placed in the queue (SenderMessage), the system processes it using the following algorithm:

1. Selecting messages for processing

• The inbound mail processing service (MailProcessingService) periodically (the interval is set by sender.receive-delay-in-millis, default — 1 minute) selects all messages from the queue:
  • with direction INBOX;
  • with retries > 0;
  • with completed = false.

2. Processing a message

   For each selected message, the processMessage() method is executed:

• Checking for a ticket number in the email subject

  If the email subject contains a ticket number (in the format [Ticket#XXXX]):

  • the ticket is looked up by number;
  • if the ticket is found — a comment is created with the email content and attachments;
  • processing completes.

• Checking a link to an existing comment

  If the email headers contain In-Reply-To or References that point to a comment SenderMessageId:

  • the system finds the comment and the related ticket;
  • a comment is created with the email content and attachments;
  • processing completes.

• Creating a new ticket

  If neither a ticket by number nor a link to a comment is found:

  • mail processing rules (MailRule) are selected;

  • active rules are sorted by descending priority (priority);

  • for each category (OutputType):

    • WORKFLOW — the space (Workflow) is determined;
    • TICKET_TYPE — the ticket type;
    • SERVICE_ENTITY — the service;
    • the values are assigned to the new ticket.

  • a new ticket is created:

    • title — email subject (subject);
    • content — email body (HTML or plain text);
    • attachments are added;
    • the sender is determined — if it matches a person or organization, the corresponding fields are filled in.

  • a comment of type INBOUND is created with the email content and attachments.

3. Processing result

• On successful processing:

  • the message is marked: completed = true, retries = 0.

• On a processing error:

  • the retries value is decreased by 1;
  • the error description is stored in the exception field;
  • the message remains completed = false.

4. Retries

• Reprocessing is performed on subsequent task runs;
• the message continues to be processed while retries > 0;
• the number of attempts is set by the sender.retries parameter (default = 3).

Rule matching (Mail Rule)

The rule catalog (Mail Rule) is used for automatic processing of inbound messages. Each rule defines how to process an email depending on its fields (addresses or subject).

Rule attributes:

• Rule type (RuleType) — the email field the rule is applied to:

  • TO — the To field;
  • FROM — the From field;
  • DELIVERED-TO — the Delivered-To field;
  • CC — the Cc field;
  • SUBJECT — the email subject.

• Rule value (RuleValue) — a regular expression (regexp) applied to the value of the selected field.

• Output type (OutputType) — the type of object the processing result refers to:

  • WORKFLOW — space (Workflow);
  • SERVICE_ENTITY — service;
  • TICKET_TYPE — ticket type.

• Output value (OutputValue) — the object identifier (UUID) that the new ticket will be bound to.

• Priority (Priority) — rule priority (the higher the value, the earlier it is applied).

• Active (Active) — rule activity flag.

Rule application algorithm:

1. All active rules are selected from the catalog;
2. For each rule, matchesRule() is executed:

• the regular expression is applied to the selected email field;
• if a match is found — the rule is considered triggered.

3. If several rules are triggered:

• For each OutputType (WORKFLOW, TICKET_TYPE, SERVICE_ENTITY), the rule with the highest priority (priority) is selected.

4. The applied rules control the fields of the ticket being created.

Rule usage examples:

Priority:

• The higher the priority, the earlier the rule is applied for that OutputType;
• If several rules of the same type are triggered — the rule with the highest priority is selected.

Message queue (SenderMessage)

The message queue is an internal mechanism for storing and processing inbound and outbound emails in the system.
All email messages (inbound and outbound) are written to the message queue (SenderMessage table).

The queue is used for:

• Asynchronous sending of outbound emails;
• Receiving and storing inbound emails;
• Resending on errors;
• Storing message processing history;
• Linking messages to comments, tickets, and events.

Attributes

Message lifecycle

Notes

• All inbound emails are stored in raw form (full MIME) so they can be reprocessed.
• The attachments attribute contains references to files (Attachment) uploaded with the email or attached to the outbound message.
• The link to comments is made by the messageId ↔ senderMessageId field in the comment.