The Signal

Everything that matters in AI, with our take.

Updated through the day. Every headline links straight to the source. The two lines underneath are ours.

agents151 research125 infra70 claude63 capital52 labs51 safety42 culture37 security35 anthropic31 regulation30 openai24 evaluation18 enterprise13 reasoning12 robotics12

arXiv cs.LGPaper4d ago

How I Aligned Your Model: Scaling Up Multilingual Safety Training with Templates

The core finding is uncomfortable: models that appear safe in English fail in translated or culturally adapted attacks. This is a real gap in safety evaluation. The template augmentation pipeline they built is clever and the MSafeTemp corpus is a useful artifact. If you're shipping a multilingual model: you need to measure safety in each language independently, not assume English safety generalizes. For safety researchers: this is a pattern to watch for in other modalities.

safety multilingual alignment research

arXiv cs.CLPaperClaude Watch12d ago

CulturalRiddles: A Multicultural Benchmark of Riddles

This exposes a real limitation: even Claude Opus struggles with cultural reasoning that requires local context and compositional thinking. The benchmark itself is solid community-built work. If you're building multilingual or culturally-aware agents, test on CulturalRiddles before shipping. If you're an LLM provider, this is a signal that cultural reasoning is a capability gap worth closing.

research benchmarks multilingual evaluation

arXiv cs.CLPaper18d ago

Language Is a Security Surface for Agent Guards

This is the first empirical proof that agent guard evaluations are language-specific in ways that matter. If your deployment touches non-English users, your guard's verdicts are not portable. Run the same audit on your guard stack now, especially if you inherit a guard from another vendor or a benchmark. The implication for vendors: safety evaluation needs to be multilingual from day one or your guard is selling false confidence.

agents safety evaluation multilingual

arXiv cs.CLPaper19d ago

Lost in Translation: Why Lower Bias Scores Don't Mean Fairer Models

This punctures a common misreading of multilingual model evaluations: non-English scores are systematically lower due to translation and StereoSet items, not fairer reasoning. If you're building multilingual systems and comparing fairness metrics across languages, you're probably misinterpreting lower scores as improvement. Fix your benchmarking assumptions before shipping claims about multilingual fairness. This is important for accuracy in public statements about your models.

research fairness multilingual bias-audits