Korben

Hackable personal assistant

Plugin Architecture: Making Korben Truly Extensible

The Extensibility Problem

When we first built Korben, adding new functionality required editing multiple files:

  1. Create your task implementation
  2. Import it in registry.py
  3. Add it to the TASKS dictionary
  4. Remember to do this for flows too
  5. Update documentation

Every. Single. Time.

This worked fine for the core team, but it created friction for:

  • Contributors - Making a simple addition required understanding the registry system
  • Personal extensions - Your custom tasks got mixed with core code
  • Maintenance - Import statements and dictionaries grew unwieldy
  • Discoverability - No way to see what’s available without reading registry.py

We needed something better. Something that let you just write code and go.

Enter: Auto-Discovery Plugins

The new plugin architecture is dead simple:

  1. Create a folder in src/plugins/
  2. Write a tasks.py or flows.py file
  3. Done. Your functions are automatically discovered and available.

No imports. No registry edits. No configuration files. It just works.

Example: Adding a Weather Plugin

Let’s say you want to add weather forecasts to Korben. Here’s the entire process:

# Create plugin directory
mkdir -p src/plugins/weather

# Create tasks.py
cat > src/plugins/weather/tasks.py << 'EOF'
"""Weather plugin tasks."""
import requests

def get_forecast(**kwargs):
    """Get weather forecast for a location."""
    location = kwargs.get('location', 'San Francisco')
    api_key = os.environ.get('WEATHER_API_KEY')
    
    response = requests.get(
        f"https://api.weather.com/forecast/{location}",
        headers={'Authorization': f'Bearer {api_key}'}
    )
    
    data = response.json()
    return f"Weather for {location}: {data['temp']}°F, {data['conditions']}"

def _api_helper(endpoint):
    """Private helper - not registered (starts with _)."""
    pass
EOF

That’s it! Now run:

python korben.py --list
# Shows: get_forecast (automatically discovered!)

python korben.py --task get_forecast --location "Paris"
# Weather for Paris: 18°C, Partly Cloudy

No registry edits. No imports. Just code.

How It Works: Convention Over Configuration

The auto-discovery system uses conventions instead of configuration:

File Structure Convention

src/plugins/
└── weather/
    ├── __init__.py          # Plugin marker (can be empty)
    ├── tasks.py             # ✅ All public functions → tasks
    ├── flows.py             # ✅ All public functions → flows  
    ├── lib.py               # Helper code (not registered)
    ├── config.yml.example   # Configuration template
    └── README.md            # Documentation

Naming Conventions

  • ✅ Public functions (no _ prefix) → Registered automatically
  • ✅ Flow names ending in _workflow → Suffix removed (e.g., daily_weather_workflow → daily_weather)
  • ❌ Private functions (starting with _) → Not registered
  • ❌ Files other than tasks.py and flows.py → Not scanned

Discovery Process

At startup, Korben automatically:

  1. Scans src/plugins/ for directories with __init__.py
  2. Imports tasks.py and flows.py from each plugin
  3. Introspects to find all public functions
  4. Registers them in global TASKS and FLOWS dictionaries
  5. Validates plugin dependencies (more on this below)

All of this happens transparently - you just write code.

Building Flows with Plugins

Let’s extend our weather plugin with a flow:

# src/plugins/weather/flows.py
"""Weather plugin flows."""
import controlflow as cf
from src.plugins.weather import tasks as weather_tasks
from src.plugins.utilities import tasks as utility_tasks

@cf.flow
def daily_weather_workflow(**kwargs):
    """Daily weather report flow.
    
    Auto-registered as 'daily_weather' (removes _workflow suffix).
    """
    # Get forecast
    forecast = weather_tasks.get_forecast(**kwargs)
    
    # Format as HTML
    html = utility_tasks.markdown_to_html(text=f"# Daily Weather\n\n{forecast}")
    
    # Email it
    utility_tasks.send_email(
        subject="Daily Weather Report",
        content=html,
        **kwargs
    )
    
    return forecast

Now you can run:

python korben.py --flow daily_weather
# Automatically emails you the weather forecast!

The flow composes tasks from multiple plugins (weather + utilities), demonstrating the power of the composable architecture.

Plugin Dependencies

What if your plugin depends on another plugin? Declare it:

# src/plugins/weather/tasks.py
"""Weather plugin tasks."""

# Declare dependencies
__dependencies__ = ['utilities', 'email']

def get_forecast(**kwargs):
    # Implementation...
    pass

Korben validates dependencies at startup:

  • ✅ Dependencies exist → Plugin loads normally
  • ❌ Dependencies missing → Plugin disabled with clear warning
  • 🔒 Prevents runtime errors → No “module not found” surprises

This ensures your plugin fails fast with helpful error messages, not cryptic runtime failures.

Real-World Example: The Movies Plugin

Let’s look at how the built-in movies plugin is structured:

src/plugins/movies/
├── __init__.py
├── tasks.py              # discover_movies, get_movie_details
├── flows.py              # trending_movies_workflow, popular_movies_workflow
├── lib.py                # TMDB API client
├── config.yml.example    # Configuration template
└── README.md             # Documentation

tasks.py - Core functionality:

"""Movie discovery tasks."""
from src.plugins.movies.lib import tmdb_client

def discover_movies(**kwargs):
    """Query TMDB for movies matching criteria."""
    genres = kwargs.get('genres', [])
    min_rating = kwargs.get('min_rating', 7.0)
    return tmdb_client.discover(genres=genres, min_rating=min_rating)

flows.py - Workflow orchestration:

"""Movie workflow orchestrations."""
import controlflow as cf
from src.plugins.movies import tasks as movie_tasks
from src.plugins.utilities import tasks as utility_tasks

@cf.flow
def trending_movies_workflow(**kwargs):
    """Find trending movies and email recommendations."""
    movies = movie_tasks.discover_movies(**kwargs)
    html = format_movies_html(movies)
    utility_tasks.send_email(subject="Trending Movies", content=html)
    return movies

Everything the plugin needs is self-contained. If you want to remove movies functionality, just delete the movies/ folder. If you want to share it, zip up the folder and send it. Clean separation.

Benefits of the Plugin Architecture

1. Zero Friction for Contributors

Before:

# registry.py
from src.core.tasks import weather_tasks
...
TASKS = {
    ...
+   "get_forecast": weather_tasks.get_forecast,
}

After:

# Just create src/plugins/weather/tasks.py
# Done!

2. True Pluggability

Plugins are completely self-contained:

  • ✅ Add plugin → works immediately
  • ✅ Remove plugin → no broken imports
  • ✅ Share plugin → just share the folder
  • ✅ Version plugin → standard Python packaging

3. Discoverability

python korben.py --list

Shows everything available across all plugins. No need to read code or documentation to discover features.

4. Separation of Concerns

plugins/
├── movies/        # TMDB integration
├── books/         # ISBNdb integration  
├── podcasts/      # Audio processing
├── mallory/       # Security news
├── utilities/     # Generic tools
└── your_plugin/   # Your custom stuff

Each plugin has a clear scope. No cross-contamination of concerns.

5. Easier Testing

Each plugin can be tested in isolation:

# tests/plugins/test_weather.py
from src.plugins.weather import tasks

def test_get_forecast():
    result = tasks.get_forecast(location="Paris")
    assert "Paris" in result

No need to mock the entire registry or worry about other plugins.

Migration: Before and After

Before: Manual Registry

# registry.py - manually maintained
from src.core.tasks import podcast_tasks
from src.core.tasks import email_tasks
from src.core.tasks import utility_tasks
from src.core.flows import podcast_flows
# ... 20+ import lines

TASKS = {
    "download_podcasts": podcast_tasks.download_podcasts,
    "transcribe_podcasts": podcast_tasks.transcribe_podcasts,
    "send_email": email_tasks.send_email,
    # ... 30+ manual entries
}

FLOWS = {
    "podcasts": podcast_flows.podcast_workflow,
    # ... 10+ manual entries
}

After: Auto-Discovery

# registry.py - auto-populated
from src.lib.core_utils import discover_plugins

# That's it! Everything else is automatic
TASKS, FLOWS = discover_plugins('src/plugins/')

The registry shrunk from 100+ lines to 3 lines. Everything else happens automatically.

Best Practices for Plugin Development

1. Self-Contained Design

Everything the plugin needs should live in its directory:

my_plugin/
├── tasks.py       # Task implementations
├── flows.py       # Workflows
├── lib.py         # Plugin-specific helpers
└── README.md      # Documentation

2. Clear Naming

Function names become CLI commands:

def get_forecast(**kwargs):      # korben.py --task get_forecast
def daily_weather_workflow():    # korben.py --flow daily_weather

Choose names that are clear and action-oriented.

3. Use Private Helpers

Keep the CLI clean by prefixing internal functions:

def get_forecast(**kwargs):      # ✅ Public - registered
    return _api_call('forecast')

def _api_call(endpoint):         # ❌ Private - not registered
    return requests.get(endpoint)

4. Document Your Plugin

Include a README.md explaining:

  • What the plugin does
  • Required environment variables
  • Configuration options
  • Usage examples

5. Declare Dependencies

Be explicit about what your plugin needs:

__dependencies__ = ['utilities', 'email']

This prevents confusing runtime errors and makes requirements clear.

Building Your First Plugin

Ready to extend Korben? Here’s a template:

# 1. Create plugin directory
mkdir -p src/plugins/my_plugin

# 2. Create __init__.py (can be empty)
touch src/plugins/my_plugin/__init__.py

# 3. Create tasks.py
cat > src/plugins/my_plugin/tasks.py << 'EOF'
"""My plugin tasks."""

def my_task(**kwargs):
    """Task description here."""
    # Your implementation
    return "Task completed!"

def another_task(**kwargs):
    """Another task."""
    # More implementation
    return "Done!"

# Private helper - not registered
def _helper_function():
    pass
EOF

# 4. Create README.md
cat > src/plugins/my_plugin/README.md << 'EOF'
# My Plugin

Description of what this plugin does.

## Usage
```bash
python korben.py --task my_task

EOF

5. Test it!

python korben.py –list # Should show my_task python korben.py –task my_task


That's all it takes. You just extended Korben!

## What's Next?

The plugin architecture opens up exciting possibilities:

- **Plugin marketplace** - Share plugins with the community
- **Remote plugins** - Load plugins from git URLs
- **Plugin configuration UI** - Visual plugin management
- **Hot reloading** - Update plugins without restarting
- **Plugin versioning** - Dependency resolution for plugin versions

## Try It Yourself

```bash
# Clone Korben
git clone https://github.com/korbenai/korben
cd korben/local/korben

# List all available plugins and tasks
python korben.py --list

# Create your own plugin
mkdir -p src/plugins/demo
echo 'def hello(**kwargs): return "Hello from my plugin!"' > src/plugins/demo/tasks.py
echo '' > src/plugins/demo/__init__.py

# Run it!
python korben.py --task hello

Conclusion

The plugin architecture transformed Korben from a framework with extension capabilities to a true platform for personal automation.

By embracing conventions over configuration and automatic discovery, we removed the friction that made extension painful. Now, adding functionality is as simple as creating a folder and writing Python code.

Want to add weather forecasts? Create a plugin. Want to integrate with Notion? Create a plugin. Want to automate your smart home? Create a plugin.

Just create a folder with tasks.py and go.

That’s the power of auto-discovery: no config files, no headaches, no tears—just pure plugin joy. Big bada boom, multipass approved!

Check out the GitHub repo to explore the plugin architecture and build your own plugins!