Ingestion Process Automation

Flow diagram

Flow of calls made to launch the generate-db process.

@startuml
title Ingestion Process Automation Flow

' Define participants with themed colors and clear names
actor User

participant "Azure Blob Storage" as AzureStorage #A2C4E0
participant "Gateway API" as GatewayAPI #bfb1f2
participant "Config Watcher" as ConfigWatcher #f296ee
participant "Deployment API" as DeploymentAPI #f77cbc
participant "Generate DB Process" as GenerateDBProcess #D9EAD3


' === Upload Files Stage ===
User -> AzureStorage : Upload training files
AzureStorage --> User : Response 200 OK

' === Launch generate-db ===
User -> GatewayAPI : Request to /aura-services/v2/operations/data to launch ingestion process
GatewayAPI -> ConfigWatcher : Request to Config Watcher
ConfigWatcher -> GenerateDBProcess : Start generate-db process
GenerateDBProcess --> ConfigWatcher : Response 200 OK
ConfigWatcher --> GatewayAPI : Response 200 OK
GatewayAPI --> User : Response 200 OK


' === Processing Stage ===
GenerateDBProcess -> AzureStorage : Read training files
AzureStorage --> GenerateDBProcess : Response 200 OK
GenerateDBProcess -> GenerateDBProcess : Processing training files

' === Logs Querying ===
... Logging queries can occur anytime ...


User -> GatewayAPI : Request to /aura-services/v2/operations/data/{presetId}/logs
GatewayAPI -> ConfigWatcher : Request to get logs
ConfigWatcher -> DeploymentAPI : Response 200 OK
DeploymentAPI --> ConfigWatcher : Response 200 OK
ConfigWatcher --> GatewayAPI : Response 200 OK
GatewayAPI --> User : Response 200 OK

' === Status Query ===
... Status queries can occur anytime ...

' === Status process ===
User -> GatewayAPI : Request to /aura-services/v2/operations/data/{presetId}/status to get status
GatewayAPI -> ConfigWatcher : Request to get status
ConfigWatcher -> DeploymentAPI : Response 200 OK
DeploymentAPI --> ConfigWatcher : Response 200 OK
ConfigWatcher --> GatewayAPI : Response 200 OK
GatewayAPI --> User : Response 200 OK


@enduml