How can we help you?

View Categories

Task Creation and Management

1 min read

FlashLearn tasks serve as structured, JSON-based API call definitions that encapsulate individual data processing requests. Each task is constructed from a JSON object, representing a discrete unit of work that the LLM agent will process. This design makes it easy to create, manage, and persist tasks for asynchronous or distributed processing.

Input Format #

  • List of Dictionaries:
    Tasks are created from inputs formatted as a list of dictionaries. Each dictionary represents a unique data record or API call. For example:
  [
    {"comment_text": "I love this product, it's everything I wanted!"},
    {"comment_text": "Not impressed... wouldn't consider buying this."}
  ]
  • JSON Definition for API Calls:
    Each task is essentially a JSON-based definition describing the API call’s configuration. This includes the request’s parameters and directives for processing. FlashLearn translates these definitions into API calls to the underlying LLM.

Creating and Using Tasks #

  • Task Construction:
    When you call the method to create tasks (e.g., using skill.create_tasks(inputs)), FlashLearn transforms your list of dictionaries into a series of JSON tasks. These tasks are structured with details such as task ID, input parameters, and any associated metadata.

  • Parallel Processing:
    Once tasks are created, they can be executed concurrently. Tasks are dispatched to the LLM client in parallel, significantly reducing processing time for large datasets.

Storing Tasks as JSONL #

  • JSON Lines (JSONL) Format:
    Tasks can be stored in JSONL files, where each line in the file is a valid JSON object. This format is particularly useful for:

    • Logging: Keeping a log of tasks for auditing or debugging purposes.
    • Persistence: Saving tasks for deferred processing, batch job execution, or future replay.

  • Example of Saving Tasks as JSONL:

  import json

  # Assume 'tasks' is a list of task objects created by FlashLearn
  with open('tasks.jsonl', 'w') as jsonl_file:
      for task in tasks:
          jsonl_file.write(json.dumps(task) + '\n')

Reading and Re-Loading Tasks #

  • Loading Tasks from JSONL:
    You can read the stored JSONL file back into your application. This enables you to re-run tasks or perform further processing.
  tasks = []
  with open('tasks.jsonl', 'r') as jsonl_file:
      for line in jsonl_file:
          tasks.append(json.loads(line))
  • Seamless Integration:
    Once loaded, these tasks can be passed back to FlashLearn’s processing engine (e.g., using skill.run_tasks_in_parallel(tasks)), ensuring that the API call definitions remain consistent across different executions.

Summary #

Task creation in FlashLearn is designed to be flexible and robust:

  • You start with simple, structured inputs (lists of dictionaries).
  • The library converts these into standardized JSON definitions representing API calls.
  • These definitions can be easily stored and read using the JSON Lines format, allowing for efficient persistence and batch processing.

By managing tasks as JSON, FlashLearn maintains a clear, consistent structure that simplifies debugging, auditing, and scaling your LLM-driven workflows.