Week 2: Multimodal Generation

Topics

• How noise schedules and denoising loops work
• Latent diffusion versus pixel diffusion
• Sampling strategies and tradeoffs

Prompt Design

• Subject, style, camera, lighting, and quality tokens
• Negative prompts for cleaner outputs
• Prompt libraries and reusable style profiles

portrait of a startup founder, cinematic light, 85mm lens,
soft shadows, neutral background, photorealistic, ultra detailed
negative: blurry, watermark, extra fingers, low quality

Adapter Techniques

• Pose, depth, edge, and segmentation controls
• LoRA for style transfer with low compute cost
• Reusable checkpoints and versioned experiment tracking

Practical Patterns

• Image captioning and OCR + summarization loops
• Visual Q&A over product screenshots and documents
• Safety checks for generated visuals

Hands-On Labs

• Lab 6: Generate campaign visuals for three brand styles
• Lab 7: Build a prompt-to-poster mini app
• Lab 8: Add pose control with ControlNet
• Lab 9: Build a multimodal image critique assistant

Simple explanation of this week

This week teaches you how to generate and control images using text instructions. Instead of just typing random prompts, you will learn a repeatable method: write a clear scene prompt, add quality/style details, include negative prompts, then tune settings until outputs are usable.

Daily workflow (2 to 3 hours/day)

1. Day 1: Generate 20 images from the same prompt using different steps and samplers. Record differences.
2. Day 2: Build a reusable prompt template with sections: subject, setting, style, camera, quality, negative.
3. Day 3: Apply one control method (pose, edge, or depth) and compare controlled versus uncontrolled output.
4. Day 4: Build a basic vision-language flow: image input, caption extraction, then rewrite caption into marketing copy.
5. Day 5: Combine everything into one mini studio where user enters brand style and gets 3 campaign images.

Common mistakes and fixes

• Mistake: Vague prompts. Fix: Use concrete nouns, style, lens, and lighting details.
• Mistake: Distorted hands/faces. Fix: Add negative prompts and increase guidance strength carefully.
• Mistake: Inconsistent brand style. Fix: Save and reuse template blocks for all outputs.
• Mistake: Overprocessing. Fix: Keep one change per experiment so results are traceable.

Assignment to complete Week 2

Create a Brand Asset Generator that takes three inputs: brand personality, product type, and campaign tone. Output at least 6 final images in two style groups. Include your prompt templates and explain how you improved output quality.

Submission checklist

• Prompt template file with reusable blocks
• Before/after examples showing quality improvements
• At least one controlled generation method (ControlNet or equivalent)
• Short notes on ethical concerns and content safety filters used
• Demo output set grouped by style

Pass rubric

• Control: You can intentionally change composition/style on demand
• Consistency: At least 80% outputs match requested brand style
• Process: You document experiments and decisions clearly
• Usability: Final assets are good enough for a sample campaign