Aura file manager processes flowchart

Initialization sequence

sequenceDiagram
    participant A as Orchestrator
    participant B as HttpMonkeyPatcher
    participant C as AuraChannelsConfiguration
    participant D as AuraMongoDb.Connector
    participant E as FileQueueDao
    participant F as FileService
    participant G as FileManagerServer
    participant H as ClientOAuthToken4P
    participant I as StorageFileManager
    participant J as 4P Whatsapp API
    participant K as Aura Global Azure storage
    participant L as QueueManager
    participant M as QueueAgent
    A ->>+ B: init
    B -->>- A: instance
    A ->>+ C: init
    C ->>+ K: loadChannelsConfiguration
    K -->>- C: Channels configuration
    C -->>- A: instance
    A ->>+ D: init
    D -->>- A: instance
    A ->>+ E: init
    E -->>- A: instance
    A ->>+ F: init
    F ->>+ H: init
    loop 4P oauth tokens
        H ->>+ J: grantClient
        J -->>- H: TokenSet
    end
    H -->>- F :initialized
    F ->>+ I: new
    I -->>- F: instance
    F -->>- A: instance
    A ->>+ G: init
    G -->>- A: instance
    A ->>+ L: init
    L -->>- A: instance
    A ->>+ M: init
    M -->>- A: instance

Sequence explanation

1. HttpMonkeyPatcher utility is loaded, mostly to monitor HTTP traffic through Prometheus.
2. AuraChannelsConfiguration is loaded, which retrieves channels configuration from Aura Global Azure storage. Aura file manager needs it in order to retrieve validation configuration for the selected channel.
3. AuraMongoDb.Connector and FileQueueDao are loaded to access MongoDB.
4. FileService initializes other different services.

• ClientOauthToken4P: It connects to Kernel WhatsApp API and retrieves tokens for different channels (WhatsApp).
• StorageFileManager: It initializes and configures the connection against Aura Global Azure storage.

3. Queue components are started: QueueManager and QueueAgent, in charge of message management.
4. FileManagerServer is started, which exposes aura-file-manager API.

API functionalities

In this section, aura-file-manager functionalities are documented using sequence diagrams.

Detailed information about these endpoints can be found in the aura-file-manager API definition.

Upload and validate files

• GET /aura-services/v1/files

Upload and validate operation. It allows to upload and validate a file based on previously defined validations.

sequenceDiagram
    actor User as Channel user
    participant K as Groot
    participant A as Bot (FileManagerRecognizer)
    participant B as Bridge (AsyncCallback plugin)
    participant C as FileManagerServer
    participant D as FileController
    participant E as FileService
    participant I as FileQueueDao
    participant J as QueueManager
    participant F as ClientOauthToken4P
    participant G as Aura Global Azure storage
    participant H as FileValidator

    User->>+K: Unprocessed attachment
    K->>-User: 204
    K->>+A: Unprocessed attachment
    A-->>-K: 204
    A->>+C: POST /files
    C-->>-A: 204
    C->>+D: uploadAndValidateFiles
    D->>+E: storeFiles
    E->>+I: insertIncomingFileRequest
    I-->>-E: Ok
    E->>+J: addTask
    J-->>-E: Ok
    D-->>-C: 204    
        alt Have queue space?
        J->>+E: processFile
        loop multiple attachments
            E->>+F: getTokenByClientId
            F-->>-E: Token
            E->>+G: uploadFile
            G-->>-E: Upload result
            E->>+G: getFileContents
            G-->>-E: File contents
            E->>+H: validateFile
            H-->>-E: Validation
            E->>+G: getSasUrl
            G-->>-E: Url
            E-->>-E: Attachment data
        end
        D-->>C: Attachments data
        C-->>B: Attachments data
        B-->>K: Attachments data
        K-->>A: Attachments data
        A-->>K: Attachments data
        K-->>User: Attachments data
    end

Actors

• Channel user: User that is interacting with the dialog.
• Groot: aura-groot main bot over distributed architecture .
• Bot: aura-bot responsible of handling the attachments processing. It is done through the aura-file-manager recognizer.
• Bridge: aura-bridge responsible of handling attachments once processed and sending data back to aura-bot.
• FileManager microservice: Microservice responsible of processing attachments.
  • FileManagerServer: Server responsible of handling request to the microservice.
  • File: Controller that handles requests of attachments processing.
  • FileService: Service where the attachments files are validated and uploaded to Aura Global Azure storage.
  • ClientOauthToken4P: Small utility to retrieve and maintain authorization Kernel tokens.
  • FileQueueDao: Decentralized autonomous organization (DAO) to manage BD operations.
  • QueueManager: Component in charge of managing queue messages.
• Aura Global Azure storage: Repository where the files are going to be stored temporarily. By default, they are accessible for one hour.
• aura-file-validator utility: Utility responsible for validating files content.

Sequence explanation

• A channel user has uploaded a file/s as requested by a dialog that supports attachments.
• When the attachments are detected by aura-bot (through the aura-file-manager recognizer), the attachments are sent to aura-file-manager microservice to be processed.
• Once the attachments are received by Aura file manager, basic checks are done synchronously (all parameters needed are present and have correct format), the request added in queue is stored and a ACK is sent to the aura-file-manager recognizer.
• After sending the ACK, the real attachments processing starts asynchronously.
• If queue is not full, the attachments begin to process.
• The direct link is sent directly to Aura Global Azure storage, delegating on it the responsibility of its uploading.
• If the repository uploads the file correctly, then aura-file-manager downloads it and, through the use of the aura-file-validator utility, validates its content.
• Once the file is validated, a direct download link is retrieved from the repository, and the data is sent to aura_bridge asyncCallback endpoint.
• aura-bridge translates the message and sends it to aura-groot.
• aura-groot translates the message and sends it to aura-bot, where aura-file-manager recognizer will forward it to the originating dialog.
• If queue is full, when one aura-file-manager pod has space enough, the attachments will be processed by QueueAgent.