Workshop 1: Getting Started with Oshun AI Integrations

Duration: 2 hours Level: Beginner Prerequisites: Basic TypeScript knowledge

Workshop Overview#

In this workshop, you'll learn how to set up and use the Oshun AI provider integrations. By the end, you'll be able to synthesize speech, generate images, and discover models using our unified provider system.

Learning Objectives#

By the end of this workshop, you will be able to:

Set up and configure AI providers
Synthesize speech using ElevenLabs
Generate images using ComfyUI/RunComfy
Discover and search models on Civitai
Handle errors gracefully
Monitor provider health

Agenda#

Time	Topic
0:00 - 0:15	Introduction and Setup
0:15 - 0:35	Part 1: Provider Configuration
0:35 - 0:55	Part 2: Text-to-Speech
0:55 - 1:15	Part 3: Image Generation
1:15 - 1:35	Part 4: Model Discovery
1:35 - 1:50	Part 5: Error Handling
1:50 - 2:00	Q&A and Wrap-up

Setup#

Environment Setup#

Clone the repository (if not already done):

bash

git clone git@github.com:GreyChimp/oshun.git
cd oshun

Install dependencies:
bash
```
pnpm install
```

Set up environment variables:

bash

export ELEVENLABS_API_KEY=your_elevenlabs_key
export RUNCOMFY_API_KEY=your_runcomfy_key
export CIVITAI_API_KEY=your_civitai_key

Verify setup:

bash

echo $ELEVENLABS_API_KEY | cut -c1-8
# Should show first 8 characters of your key

Part 1: Provider Configuration (20 minutes)#

Concept: Provider Pattern#

All Oshun AI integrations follow the same provider pattern:

text

┌─────────────────────────────────────────┐
│           Your Application              │
├─────────────────────────────────────────┤
│         Provider Interface              │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│  │ElevenLabs│ │ RunComfy │ │ Civitai  │ │
│  │ Provider │ │ Provider │ │ Provider │ │
│  └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│       │            │            │       │
├───────┼────────────┼────────────┼───────┤
│       ▼            ▼            ▼       │
│   ElevenLabs    RunComfy    Civitai     │
│      API          API         API       │
└─────────────────────────────────────────┘

Exercise 1.1: Basic Initialization#

typescript

import { ElevenLabsProvider } from '@oshun/ai-providers';

// Create a basic provider
const provider = new ElevenLabsProvider({
  apiKey: process.env.ELEVENLABS_API_KEY!,
});

console.log('Provider created successfully!');

Try it: Run this code and verify it doesn't throw errors.

Exercise 1.2: Configuration Options#

typescript

import { ElevenLabsProvider, ElevenLabsConfig } from '@oshun/ai-providers';

// Full configuration example
const config: ElevenLabsConfig = {
  apiKey: process.env.ELEVENLABS_API_KEY!,
  timeout: 30000,        // 30 second timeout
  maxRetries: 3,         // Retry up to 3 times
  logRequests: true,     // Log requests for debugging
};

const provider = new ElevenLabsProvider(config);

// Health check
const isHealthy = await provider.healthCheck();
console.log(`Provider healthy: ${isHealthy}`);

Discussion Questions#

Why do we use environment variables for API keys?
What happens if the health check fails?
When would you increase the timeout?

Part 2: Text-to-Speech (20 minutes)#

Concept: Speech Synthesis#

ElevenLabs converts text to natural-sounding speech. Key concepts:

Voice ID: Unique identifier for a voice
Model ID: The AI model to use (affects quality/speed)
Voice Settings: Fine-tune the output (stability, similarity)

Exercise 2.1: List Available Voices#

typescript

const voices = await provider.listVoices();

console.log(`Found ${voices.length} voices:\n`);

for (const voice of voices.slice(0, 5)) {
  console.log(`- ${voice.name} (${voice.voice_id})`);
  console.log(`  Category: ${voice.category}`);
  console.log(`  Labels: ${JSON.stringify(voice.labels)}`);
}

Your task: Find a voice that matches:

Female gender
American accent
Conversational use case

Exercise 2.2: Basic Synthesis#

typescript

const text = 'Hello! Welcome to the Oshun AI workshop.';
const voiceId = 'EXAVITQu4vr4xnSDxMaL'; // Sarah

const audioBuffer = await provider.synthesize({
  text,
  voiceId,
});

console.log(`Generated ${audioBuffer.length} bytes of audio`);

// Save to file
import { writeFileSync } from 'fs';
writeFileSync('output.mp3', audioBuffer);
console.log('Saved to output.mp3');

Your task: Change the voice and text to create your own audio file.

Exercise 2.3: Voice Settings#

typescript

const audioBuffer = await provider.synthesize({
  text: 'This is a test with custom voice settings.',
  voiceId: 'EXAVITQu4vr4xnSDxMaL',
  voiceSettings: {
    stability: 0.3,        // More expressive
    similarity_boost: 0.9, // Close to original voice
    use_speaker_boost: true,
  },
  modelId: 'eleven_v3',   // Latest model
});

Your task: Experiment with different stability values (0.0 to 1.0) and observe the difference in output.

Checkpoint#

At this point, you should be able to:

List voices and find one matching criteria
Generate speech from text
Customize voice settings

Part 3: Image Generation (20 minutes)#

Concept: Workflow Execution#

ComfyUI uses node-based workflows for image generation. RunComfy executes these workflows in the cloud.

text

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Submit    │───▶│   Execute   │───▶│   Return    │
│  Workflow   │    │  on Cloud   │    │   Results   │
└─────────────┘    └─────────────┘    └─────────────┘

Exercise 3.1: Provider Setup#

typescript

import { RunComfyProvider } from '@oshun/ai-providers';

const comfyProvider = new RunComfyProvider({
  apiKey: process.env.RUNCOMFY_API_KEY!,
  maxConcurrentJobs: 5,
  defaultTimeoutMs: 300000, // 5 minutes
});

Exercise 3.2: Submit a Job#

typescript

const job = await comfyProvider.executeWorkflow({
  workflowId: 'txt2img-sdxl-v1',
  inputs: {
    prompt: 'A majestic dragon flying over mountains at sunset',
    negative_prompt: 'blurry, low quality, watermark',
    width: 1024,
    height: 1024,
    steps: 25,
  },
});

console.log(`Job submitted: ${job.id}`);
console.log(`Status: ${job.status}`);

Exercise 3.3: Wait for Completion#

typescript

const completedJob = await comfyProvider.waitForCompletion(job.id, {
  timeoutMs: 300000,
  onStatusChange: (status) => {
    console.log(`Status changed: ${status}`);
  },
});

if (completedJob.status === 'completed') {
  console.log('Generation complete!');
  for (const output of completedJob.outputs || []) {
    console.log(`Output: ${output.url}`);
  }
} else {
  console.log(`Job ended with status: ${completedJob.status}`);
}

Your task: Generate an image with your own prompt. Try different subjects and styles.

Checkpoint#

At this point, you should be able to:

Submit image generation jobs
Monitor job status
Retrieve generated images

Part 4: Model Discovery (20 minutes)#

Concept: Model Registry#

Civitai hosts thousands of AI models. You can search, filter, and discover models for different use cases.

Exercise 4.1: Provider Setup#

typescript

import { CivitaiProvider } from '@oshun/ai-providers';

const civitaiProvider = new CivitaiProvider({
  apiKey: process.env.CIVITAI_API_KEY, // Optional for browsing
  enableCache: true,
  cacheTtlSeconds: 300,
});

Exercise 4.2: Search Models#

typescript

const results = await civitaiProvider.searchModels({
  query: 'anime',
  types: ['LORA'],
  baseModels: ['SDXL 1.0'],
  sort: 'Highest Rated',
  limit: 10,
});

console.log(`Found ${results.items.length} models:\n`);

for (const model of results.items) {
  console.log(`${model.name}`);
  console.log(`  Type: ${model.type}`);
  console.log(`  Downloads: ${model.stats.downloadCount.toLocaleString()}`);
  console.log(`  Rating: ${model.stats.rating.toFixed(2)}`);
  console.log();
}

Exercise 4.3: Get Model Details#

typescript

const modelId = 12345; // Replace with a model ID from search

const model = await civitaiProvider.getModel(modelId);

console.log(`Model: ${model.name}`);
console.log(`Description: ${model.description?.slice(0, 200)}...`);
console.log(`Creator: ${model.creator.username}`);

console.log('\nVersions:');
for (const version of model.modelVersions.slice(0, 3)) {
  console.log(`  - ${version.name}`);
  console.log(`    Base Model: ${version.baseModel}`);
  console.log(`    Trigger Words: ${version.trainedWords?.join(', ') || 'None'}`);
}

Your task: Search for models in a category you're interested in. Find a model that could work for a project idea.

Checkpoint#

At this point, you should be able to:

Search models with filters
Get detailed model information
Understand model types and versions

Part 5: Error Handling (15 minutes)#

Concept: Graceful Degradation#

AI APIs can fail. Good error handling ensures your application degrades gracefully.

Exercise 5.1: Try-Catch Pattern#

typescript

try {
  const audio = await provider.synthesize({
    text: 'Test',
    voiceId: 'invalid-voice-id',
  });
} catch (error) {
  if (error instanceof Error) {
    console.error(`Error: ${error.message}`);

    // Check for specific error codes
    const apiError = error as Error & { code?: string; retryable?: boolean };

    if (apiError.code === 'VOICE_NOT_FOUND') {
      console.log('The specified voice does not exist.');
    } else if (apiError.retryable) {
      console.log('This error is temporary. Try again later.');
    }
  }
}

Exercise 5.2: Health Monitoring#

typescript

async function checkAllProviders() {
  const providers = [
    { name: 'ElevenLabs', check: () => provider.healthCheck() },
    { name: 'RunComfy', check: () => comfyProvider.healthCheck() },
    { name: 'Civitai', check: () => civitaiProvider.healthCheck() },
  ];

  console.log('Health Check Results:');

  for (const { name, check } of providers) {
    try {
      const healthy = await check();
      console.log(`  ${name}: ${healthy ? 'OK' : 'UNHEALTHY'}`);
    } catch (error) {
      console.log(`  ${name}: ERROR - ${error}`);
    }
  }
}

await checkAllProviders();

Exercise 5.3: Statistics#

typescript

const stats = provider.getStats();

console.log('Provider Statistics:');
console.log(`  Total Requests: ${stats.totalRequests}`);
console.log(`  Successful: ${stats.successfulRequests}`);
console.log(`  Failed: ${stats.failedRequests}`);
console.log(`  Avg Latency: ${stats.averageLatencyMs.toFixed(2)}ms`);

Q&A and Wrap-up (10 minutes)#

Key Takeaways#

Unified Provider Pattern: All integrations follow the same pattern
Configuration Matters: Set appropriate timeouts and retries
Error Handling: Always wrap API calls in try-catch
Monitoring: Use health checks and statistics

Next Steps#

Complete the beginner exercises
Explore the code examples
Attend Workshop 2: Advanced Patterns

Resources#

Homework#

Create a script that:
- Lists 5 voices from ElevenLabs
- Generates speech with each voice
- Saves the audio files
Create a script that:
- Searches for LoRA models on Civitai
- Gets details for the top 3 results
- Prints a summary report
Add error handling and statistics to your scripts