To: perchance dev / ai-text-plugin maintainer Re: Repetition degeneration (token loops) — request repetition-penalty option

I’m hitting repetition degeneration in my generator (FurAI / ai-furry-generator). During streaming the model in Simplified/Traditional Chinese both VERY frequently falls into a tight token loop and emits runs like:

他地大掌猛"地地地地" 死死"地地地" 按在… 地地地地地地地地地地地地地地地地地地地地地地地地地地地地地地地地地地地地

— a single CJK character repeated 4+ times, sometimes continuing for the rest of the generation. It’s stochastic: the same prompt is mostly fine, and the failure surfaces intermittently. Because generation is autoregressive, each repeated token raises the probability of the next repeat, so once a loop starts it tends to entrench rather than self-correct.

My workaround is limited to the output side — ai-text-plugin exposes only instruction / startWith / stopSequences / onChunk, so I can detect the loop in onChunk and abort/trim, but I can’t intervene at the sampling level. That catches symptoms, not the cause.

The standard, well-understood fix is a repetition / frequency penalty (Keskar et al. 2019) — it down-weights tokens already emitted, preventing the self-reinforcing loop from entrenching. It’s a default knob in llama.cpp, vLLM, and OpenAI’s API (frequency_penalty, presence_penalty).

Two possible asks, either helps:

Apply a modest repetition penalty server-side in the text service if one isn’t already enabled; or Expose it as an ai-text-plugin option (e.g. repetitionPenalty / frequencyPenalty) so authors can tune per-call. Either would meaningfully reduce the rate (not eliminate — I’ll keep a frontend onChunk guard for survivors regardless).

Thanks