Skip to content

Workflows Overview

This document explores the concept of Workflows within Raikoo. Workflows provide a robust mechanism for orchestrating complex AI tasks, breaking them down into sequential or parallel steps (operations) and managing their interaction with a shared workspace.

Workflows: The Blueprint for Automated AI Tasks

A Workflow represents a plan for a complex AI task. It consists of a structured sequence of operations, each designed to perform a specific action within the task. Workflows can be thought of as the "blueprint" or "recipe" for executing a multi-step AI process.

Workflow Structure:

  • Name: A human-readable label for the workflow, aiding in identification and organization.
  • Description: A detailed description of the workflow's purpose, explaining the overall goal of the task.
  • Persona Name: The name of the AI persona responsible for executing the workflow.
  • Parameters: Key-value pairs that provide input settings and configurations for the workflow. These values often reference data within the workspace.
  • Workspace: A shared space for storing data files and other resources used by the workflow.
  • Operations: A list of individual operations that make up the workflow's execution plan.

Workflow Operations: Building Blocks of a Workflow

Workflow operations represent the individual steps within a workflow. Each operation performs a specific action, such as:

  • Data retrieval: Loading data from the workspace or external sources.
  • Text generation: Using a language model to create text based on provided inputs.
  • Data processing: Performing specific transformations on data in the workspace.
  • File manipulation: Creating, modifying, or deleting files within the workspace.

Operation Structure:

  • ID: A unique identifier for the operation, allowing for tracking and management within the workflow.
  • Operation ID/Name: A reference to the specific operation type, linking it to a corresponding operation manager responsible for its execution.
  • Model Selector: An optional specification of the language model to be used for the operation, overriding the default model selection for the workflow.
  • Persona Name: An optional assignment of a specific AI persona for this operation, overriding the default persona for the workflow.
  • Dependencies: A list of other operations that must be completed before this operation can be executed.
  • Request: A reference to data or a file within the workspace that serves as input for the operation.
  • Primer: An optional reference to data within the workspace that provides a leading response to the AI assistant; useful to coerce a response that follows a certain format.
  • Parameters: Key-value pairs specifying additional settings or configurations for the operation.
  • Context: A list of references to files or data within the workspace that provide additional context or information for the operation.
  • Output: A reference to a file or data within the workspace where the operation's result will be stored.

Workflow Managers: Orchestrating the Workflow's Execution

A WorkflowManager is responsible for overseeing the execution of a complete workflow. It manages the individual operations within the workflow, tracks their progress, and communicates updates and results.

Key Responsibilities of a WorkflowManager:

  • Initialize workflow status: Sets the initial state of the workflow, based on the workflow plan and any existing progress.
  • Manage workspace: Handles updates, refreshes, deletions, moves, and copies of data and files within the workspace.
  • Run operations: Initiates and manages the execution of individual operations within the workflow.
  • Track operation progress: Monitors the status of each operation, determining which operations are ready to run and handling any dependencies.
  • Communicate status updates: Provides updates on the workflow's overall progress and the status of individual operations.
  • Handle errors: Catches and manages errors that occur during workflow execution.

Workflow Operation Managers: Executing Individual Operations

Each workflow operation is handled by a specific WorkflowOperationManager, responsible for executing the operation and reporting back its results.

Key Responsibilities of a WorkflowOperationManager:

  • Prepare the operation: Loads required data from the workspace, sets up the operation context, and prepares the input prompt for the operation.
  • Execute the operation: Calls the appropriate operation processor (e.g., language model, data processing engine) to perform the requested action.
  • Process results: Receives and interprets the results of the operation.
  • Update workspace: Stores the operation's results in the designated location within the workspace.
  • Communicate status updates: Provides updates on the operation's progress and any potential issues encountered.

The Workspace: A Centralized Data Repository

The workspace serves as a central repository for data, files, and other resources used by the workflow and its operations. Workflow managers and operation managers interact with the workspace to access and manipulate data, ensuring that all operations within the workflow have access to the latest information.

Key Features of the Workspace:

  • Data storage: Stores files, data, and other assets related to the workflow.
  • File manipulation: Enables operations to create, modify, or delete files within the workspace.
  • Data retrieval: Allows operations to retrieve data or files from the workspace.
  • Context management: Maintains the operational context by storing information about the workspace's state and the results of previously executed operations.

Execution Modes: Parallel vs Serial

Workflows can be executed in two primary modes: serial and parallel.

Serial Execution:

  • Operations are executed one after the other, in the order defined in the workflow plan based on their dependencies.
  • An operation cannot begin execution until the previous operation is complete.
  • Suitable for tasks that require sequential processing, where the output of one operation depends directly on the output of the preceding operation.

Parallel Execution:

  • Operations can be executed concurrently in batches, running independently of each other, if their dependencies have been met.
  • Ideal for tasks where operations do not depend on each other's results and can be performed concurrently to speed up the overall execution time.

Conclusion: Workflows for Efficient AI Task Automation

Workflows provide a powerful and flexible framework for managing complex AI tasks. By breaking down tasks into modular operations, managing dependencies, and efficiently interacting with the workspace, Workflows enable:

  • Automation: Automating complex workflows without requiring manual intervention.
  • Modularization: Designing and reusing individual operations across different workflows.
  • Scalability: Handling large and complex tasks by executing operations in parallel.
  • Efficiency: Optimizing the execution of tasks by managing dependencies and resources.

As AI applications continue to evolve and become more complex, the use of Workflows becomes increasingly essential. By providing a structured and efficient method for orchestrating AI processes, Workflows empower developers to build robust and scalable AI solutions.