AI Best Practices for Creative Writing / Role-Playing (RP)

This guide provides recommendations for getting the best experience out of AI models during creative writing or role-playing sessions.

1. Choosing Your Model

  • Try a few models: Experiment to find what you prefer.
  • Q6 vs Q8: Generally, Q8 and Q6 are very similar. Stick with Q6 if you encounter issues.
  • Q4_K_M: A good balance exists between maintaining quality (Q) and keeping speed manageable.
  • Context Size: Most models are trained for 8k context. Using a maximum context size of 16k is often beneficial for RP, as it usually provides enough capacity without significant downsides.

2. Context Size Requirements (16k)

Running a 16k context size requires sufficient RAM/VRAM. Here are estimates for common models:

  • -- 100B: ~10GB
  • -- 70B: ~7GB
  • -- 24B: ~5GB
  • -- 12B: ~4.5GB
  • -- 7B: ~3GB

Estimating Total VRAM/RAM

To estimate the total VRAM/RAM needed for a specific GGUF file with 16k context:

  1. Find the file size of the GGUF model.
  2. Add the estimated RAM requirement for the 16k context (from above).
  3. This sum gives you a rough idea of the necessary VRAM/RAM.

3. Finding the Right B+Q Combination

To balance performance and quality:

  1. Start High: Choose the highest B (parameter count) with the highest Q (quantization level) you can run comfortably.
  2. Adjust if Too Slow:
    • If the model is too slow, reduce the Q level (e.g., from Q6 to Q4 or Q3). Avoid going below Q3 if possible, as lower Q levels significantly increase potential errors.
    • Once you find a suitable Q for the high B, consider lowering the B by one step and repeat the process starting with Q6.
  3. Iterate: Continue adjusting B and Q levels until you find a combination that meets your speed requirements without unacceptable loss of quality.
  4. Evaluate Models: Within the range you find acceptable, try different models and choose the one you prefer based on style, personality, or specific capabilities. Remember, there's no single "best" model; our goal is tailored RP experiences, not standardized benchmarks.

4. General Rules for Interaction

Elephant in the Room

  • Avoid Negative Commands: Don't explicitly forbid the AI from mentioning certain topics (e.g., "Samatha shouldn't mention..."). The AI might not correctly interpret the negation.
  • Use Positive Guidance: Instead, instruct the AI on desired behavior or formulate requests positively (e.g., "Guide Samatha to focus on...").

Don't Speak or Act for the Player

  • Problem: If you don't want the AI to impersonate your character or speak for you, simply stating "Don't speak for the player" might not be sufficient.
  • Solution:
    • Positive Guidance: Clearly tell the AI what you want it to do instead, or provide guidance on how to interact (e.g., "Avoid telling User or other characters what to do or think, but provide opportunities for them to role-play their character's response to the developing narrative.").
    • Character Definition: Ensure your character definition clearly states the desired interaction style (active vs. passive, etc.).

5. Character Card Recommendations

Where to Find Inspiration

  • Discord Communities: Look for well-crafted character cards in the Discord channels of API providers, GUI providers, and LLM groups that fine-tune models. These communities often share high-quality examples.
  • Avoid Generic Sources: Be cautious with reference cards found on general card platforms or websites, especially those focused on world types (like Isekai). While they can be fun starting points, they may not translate well to RP without specific character rules.

How to Use Found Cards Wisely

  • Review Critically: Always review any downloaded or referenced card.
  • Check for Issues: Look for spelling mistakes and instances of the "Elephant in the room" (commands telling the AI not to do certain things).
  • Enhance Details: Rewrite the character description with more detail, ensuring it includes a personality trait.
  • Add Interaction Rules: Include specific instructions for how the AI should interact with you (e.g., "The AI should actively engage the user within its defined role." or "The AI should primarily respond to user prompts.").
  • Define Goals: Clarify the goal for the user within the role-play (e.g., "The user is to explore the consequences of their actions," or "The user is to guide the character through a specific quest").
  • Context is Key: Depending on the model, even with good rules, interactions can sometimes become inappropriate. Setting clear interaction guidelines upfront helps maintain a safer and more controlled RP environment.

By following these practices, you can tailor the AI experience to better suit your creative writing and role-playing needs. Remember that experimentation and adaptation are key! Good luck.