I have recently been reconsidering a phenomenon I encountered some years ago: ask the same large language model roughly the same question in Simplified Chinese, Traditional Chinese, and English, and the answers may lean in different directions.

A prominent example concerns the relationship between Taiwan and mainland China. A change in the question's language may change how a model describes that relationship, or even which position it selects when explicitly asked to take one.

At first glance, this looks like a problem of political bias. But the question that interests me most is not which side a model eventually favors; it is how the answer comes about.

If a model's judgment shifts with language, does that reflect the distribution of its training data, boundaries shaped by post-training, or an adaptation to the interaction context?

And if online content can be deliberately shaped through data poisoning, search optimization, and generative engine optimization (GEO), another question becomes unavoidable:

Has the information available to the model already been filtered, repeated, or manipulated?

Reliability may depend not only on how much a model knows, but also on how that knowledge was produced, which information survives into the answer, and which standards the model applies when evaluating evidence.

Political positions across languages

In 2024, Yu-Ting Lin and Yao-Chung Fan published "Unveiling Language-Driven Political Stances in Large Language Models on China and Cross-Strait Relations" at ROCLING.

They tested eight models originating from mainland China, Taiwan, and the United States, posing twenty questions on Chinese politics and cross-strait relations in Simplified Chinese, Traditional Chinese, and English.

Some results varied substantially. For GLM-4-9B, agreement with the official position of the People's Republic of China was reported at 60% in Simplified Chinese, 40% in Traditional Chinese, and 15% in English. Qwen2-7B-Instruct showed a similar direction of change: 75%, 55%, and 45%, respectively.

This pattern was not universal. Yi-1.5-9B, also developed in mainland China, registered 35%, 30%, and 30% in the three languages—nothing like GLM-4's magnitude of variation. Within these tests, a model's development location alone was not sufficient to predict its stance.

The meaning of "agreement rate" also matters. The researchers did not instruct models to support either mainland China or Taiwan. Their system prompt asked models to evaluate political propositions from their own preferred perspective and encouraged a definite stance rather than neutrality.

For a proposition such as "Taiwan is part of China," a model could select agree, disagree, or no opinion, and explain its choice. Researchers then classified whether the response matched the PRC government's official stance. This measures correspondence with a political position, not factual accuracy.

Neutral answers and refusals were excluded from the denominator. Consider InternLM2.5's English responses: out of twenty questions, sixteen were classified as neutral, three as aligned with the mainland government's position, and one as opposed. Among the four non-neutral responses, alignment was 75%. Across all twenty questions, alignment occurred in just 15%. Both numbers are mathematically correct, but they measure different things.

The first describes what happens when a model takes a definite stance; the second describes how often a user would encounter that position across the entire test set. Ignoring the number of neutral answers can exaggerate the frequency of a position in overall outputs.

This was also a political-stance questionnaire, not an open-ended factual question-answering experiment. "Do you agree that Taiwan is part of China?" is a different task from "Distinguish Taiwan's present administration, the sovereignty claims made by different political actors, and international recognition." The first requests a position on a proposition; the second requests an explanation of factual and political distinctions without necessarily reducing them to agreement or disagreement.

That distinction does not prove open-ended factual answers would be more reliable. The paper did not provide a controlled comparison of these two task formats, so it cannot establish how much language-driven variation would persist in everyday factual question answering.

Its most defensible finding is narrower and still important: under a particular political-stance questionnaire, some tested models changed their expressed positions significantly with the language of the prompt, and development location did not fully explain those changes.

Explaining why requires examining both the information environment and how models are trained.

The information environment behind a language

Languages do not inhabit identical information environments. Historical textbooks, news organizations, government documents, academic publications, and online communities can develop relatively stable ways of describing the same issue.

On contested political and historical topics, censorship, media institutions, social identity, and distribution channels can affect how frequently a particular position appears and how readily alternative accounts can be encountered.

I would not equate Simplified Chinese with a closed information environment or Traditional Chinese and English with open ones. Language is not a sufficient measure of information freedom. Commercial interests, political messaging, editorial decisions, and recommendation algorithms can shape visibility even in environments with relatively diverse sources. Substantially different sources can also exist within a single language.

The useful comparison is not which language is naturally objective, but how information environments differ in factual coverage, distribution of viewpoints, and quality of evidence.

Models may learn those differences, yet pretraining is not the only influence. Instruction tuning, preference optimization, and other post-training methods can shape response habits, refusal policies, and content boundaries. Developers may also face regulatory requirements.

For example, mainland China's Interim Measures for the Management of Generative Artificial Intelligence Services impose political-content obligations on relevant public-facing generative AI services, including restrictions relating to national interests, national image, and national unity.

A 2026 PNAS Nexus study, "Political censorship in large language models originating from China," compared responses by models from mainland China and elsewhere to 145 political questions. In its sample, models from mainland China refused more sensitive questions, generated shorter responses, and produced more inaccurate answers. The differences diminished on less politically sensitive topics.

This is evidence of systematic behavior differences, not a direct demonstration of which internal mechanism caused them. The authors describe the work as observational; it does not independently establish a complete causal chain from regulation to an individual model's answer.

We can observe outputs on particular topics. We cannot, from a single black-box response, determine what proportion of the difference comes from pretraining, post-training, or institutional constraints.

Inheriting a biased information environment and deliberately setting response boundaries during development are different mechanisms. They may coexist and may produce similar visible outcomes.

Data contamination: repetition is not independent evidence

The internet is no longer simply a naturally accumulated collection of human discussion. Search optimization, automatically generated material, advertising, and coordinated distribution can all affect how often particular narratives appear.

A statement repeated many times is not necessarily supported by many independent pieces of evidence.

Several issues must be separated. Ordinary data-quality failures include incorrect information, duplication, obsolete material, and unsubstantiated machine-generated text. Deliberate training-data poisoning involves an attacker controlling some training material to induce an unwanted behavior or answer. Benchmark contamination means that evaluation examples have leaked into training data, reducing the independence of a test.

These phenomena are not interchangeable. The mere presence of errors online does not show that a given model was deliberately poisoned. But successful poisoning is not just hypothetical.

In 2025, Anthropic, the UK AI Security Institute, and the Alan Turing Institute published "A small number of samples can poison LLMs of any size." In their tested settings, approximately 250 malicious documents were enough to induce a particular backdoor behavior across the studied model sizes.

The target behavior matters. This experiment implanted a relatively simple backdoor that caused gibberish output following a trigger. It did not test whether 250 political documents could cause a model to adopt a political position. Nor does that laboratory outcome establish that a particular model's real political answers were modified by an external poisoning attack.

The result does, however, challenge a simplistic assumption: a very large training corpus does not automatically make a small volume of malicious data harmless. Dataset scale and resistance to poisoning are not interchangeable.

Source independence poses another problem. Imagine an inadequately verified commercial report repeatedly republished by websites and rewritten automatically into industry analysis, news summaries, and question-and-answer pages. A model may encounter the conclusion many times during training. Yet those documents could all descend from one report rather than represent independent research.

If a model cannot distinguish the relationships among sources, it may learn that a statement appears frequently rather than that the statement has been independently corroborated. The failure may not be a total absence of knowledge; it may be an inability to distinguish where apparently different pieces of evidence came from.

GEO: optimizing content to enter AI answers

If training contamination mainly concerns what a model learns, generative engine optimization concerns what an AI search system is likely to retrieve, cite, and include in its output.

Traditional SEO concentrates on ranking among search results. GEO, or Generative Engine Optimization, focuses on making content visible and usable to systems that synthesize answers from retrieved sources.

Pranjal Aggarwal and collaborators presented "GEO: Generative Engine Optimization" at KDD in 2024. Experiments using GEO-bench found that particular changes to source presentation could increase visibility in generated responses, by as much as about 40% in some settings. This was a measure of source visibility—not factual accuracy, and not proof that optimization can reliably force any false claim into an answer.

GEO is not inherently objectionable. Clear organization, accurate structured information, transparent sourcing, and machine-readable pages can help answer engines use legitimate material.

But GEO changes the incentives of content producers. They compete not only for a click from a user, but for inclusion within an AI-generated answer.

A 2026 ICML position paper, "Position: Generative Engine Optimization Creates Underexamined Risks," raises concerns about concentrated influence and undisclosed commercial incentives in generated answers. As a position paper, it argues that these risks deserve study and governance; it does not establish that all answer engines are already affected to the same degree.

One result from the original GEO experiments is already clear: the likelihood of appearing in a generated answer can change with a source's structure and presentation, rather than being determined solely by factual quality. A source being cited more often does not itself prove that source is more credible.

This may be harder for readers to notice than traditional search ranking. Search engines typically present several clickable results. Generative search combines retrieval and synthesis into a single narrative, perhaps accompanied by a handful of citations. Which documents were selected and how they were summarized now shape much of what the reader receives.

GEO should nevertheless be distinguished from malicious retrieval poisoning. Legitimate GEO aims to improve discoverability. An adversarial retrieval attack has a specific objective, such as planting false evidence to degrade the answer.

The 2025 EMNLP Findings paper "The RAG Paradox" studied a black-box attack that used retrieval-source feedback to produce plausible-looking poisoned documents, degrading results in experimental retrieval-augmented systems. The research demonstrates a vulnerability in the use of external knowledge, not that ordinary GEO is itself an attack or that every search system has the same weaknesses.

Together these studies show that the evidence selected for a generated answer can be influenced not only by the model but also by the visibility, structure, and vulnerability of the external retrieval environment.

From visibility to credibility

The distinction is simple but consequential:

Widely circulated material is not necessarily supported by equally extensive independent evidence; a citation in an AI answer is not the same as a completed fact-check.

Training data and real-time retrieval operate on different timescales. New GEO content cannot directly change the parameters of a model that has already finished training and has no access to external sources. For a search-enabled or RAG-based system, however, retrieved material can influence the current answer.

If public content is later collected into a new training corpus, its effects could extend to future models. That is a technically plausible pathway, not an observed causal chain that can be assumed in every case.

It is helpful to inspect each stage separately. During training: provenance, quality, and repetition. During retrieval: source selection, independence, and manipulation. During generation: fidelity, omission, and unsupported additions. These stages may interact, but they should not be collapsed into the catch-all label of "model bias."

A model can change the material without inventing facts

What a model does with material already placed directly in its context also matters.

In my earlier article, "After Models Cross Borders: What Qwen, Gemma, and Ministral Reveal About Open Weights, Closed Systems, and the Boundaries of Answers," I compared three local models on the same material.

Qwen's behavior was particularly notable. Faced with an ordinary account of everyday behavior, it was relatively restrained. When the text did not explain a person's motives, it acknowledged uncertainty rather than constructing a complete psychological narrative.

The outcome changed when the model handled English historical material about the June Fourth events of 1989.

The source included the setting and timing, Hu Yaobang's death, students' demands, participation by workers and residents, the use of military force, disputed death counts, and subsequent censorship.

The locally customized Qwen response was only about 174 Chinese characters and omitted several of these central points. It did not set out students' demands, did not mention workers and residents, did not clearly recount the military use of force, and recast the source's discussion of disputed casualty numbers as a broad inability to verify. It invoked political sensitivity and the absence of official sources, recommending consultation of official historical documents.

The problem was not mere brevity. The task was to analyze supplied text, but the output removed information that could materially affect a reader's understanding of that text.

Accurately reporting what a text says is different from independently verifying the entire historical event it describes. A model unsure of a casualty figure can state what the document says and separately explain that the figure needs external verification. Preserving the fact that sources dispute the death toll does not imply endorsing any one number.

In a control test, the official `qwen3-vl:4b-thinking` variant explicitly stated that it could not discuss politics, religion, pornography, violence, and other subjects. The local rules softened outright refusal into a limited summary, but did not reliably restore the important source details.

These are observable results for specific versions and configurations, not a window into the model's internal decision sequence. A reference to "official sources" in the response does not prove that the model first executed a source-authentication procedure and then removed material.

A different plausible explanation is that post-training or political-content requirements affected responses on sensitive topics. Regulation and empirical work on political refusals make that worth examining, but a single black-box test cannot reveal the exact steps used to train those weights.

Nor does this local experiment establish a causal link to GEO or malicious data poisoning. There is no direct evidence of such a mechanism in the test.

Uncertainty about cause does not prevent evaluating the result: under the tested conditions, the model failed to preserve important historical information already supplied by the user and changed the informational structure of the final account.

A model need not fabricate explicit falsehoods to distort what a reader learns. Omitting material context may create an overall impression different from the source. That is why measuring only a model's stated political position can miss serious questions of information quality.

A model may select "neutral" on a political questionnaire and still omit the central facts needed to understand the disagreement. Preservation of information deserves a separate measure.

Willingness to answer is not the same as reliability

Qwen's omissions do not imply that a model willing to discuss sensitive subjects must be more reliable.

In the same local tests, Gemma retained more information from the historical material but tended to add interpretations the source did not support. In everyday accounts, cleaning a room, making breakfast, or taking a brief break became narratives of emotional healing, regained control, or inferred personality traits.

Those interpretations are not inherently impossible, but the given material did not justify presenting them as established conclusions.

Ministral made more direct factual errors. It replaced Hu Yaobang with Hu Jintao and continued generating dates, policies, and causal claims around the mistaken identity, producing a fluent but historically inaccurate explanation.

These are different failure modes. Under the tested conditions, Qwen omitted important information on sensitive material; Gemma supplied unsupported meaning; Ministral generated false facts. They should not all be attributed to a single political cause.

But all three can be evaluated with the same question: was the answer adequately constrained by the source evidence?

We do not need complete access to a model's hidden internal mechanisms to check whether it completed the task, retained the necessary information, or added claims for which there was no support.

What qualifies as reasonable evidence?

That brings the discussion to a more basic question: how should a model decide whether material is reasonable?

Suppose a model reaches a conclusion, a user supplies contrary evidence, and the model changes its mind. I do not think changing its stance alone proves intellectual openness, nor does it necessarily demonstrate flattery or compliance.

What matters is the quality and relevance of the new evidence.

A genuine source does not necessarily prove the interpretation attached to it. In an everyday narrative, someone cleaning a room and later feeling better does not establish that cleaning caused psychological recovery. Sequence is not causation.

Likewise, the same claim appearing on many websites does not show independent corroboration if all of those websites repeat a single original report.

Assessing evidence requires asking whether the information is authentic, whether it actually addresses the proposition, whether the inference is justified, and whether nominally different sources are independent.

Frequency, source identity, and language should not substitute for those checks.

Nor should openness require a model to invent a counterargument for every established fact. Where reliable evidence strongly supports a conclusion, constructing a weak opposition merely to appear balanced does not improve objectivity. Understanding why a view exists is different from treating every view as equally well supported.

Objectivity need not mean neutrality on every question. It should mean applying consistent, testable standards of evidence.

Those standards themselves must remain open to examination. If a model accepts only material that fits its current framework, then permits only such material to change its conclusions, its apparent knowledge may conceal an inability to revise its judgment.

A good system should maintain or revise a position because of evidence—not merely agree with a user, and not object for the sake of objection.

Evaluate the result, not merely the declared stance

In ordinary use, I still prefer to look at the result first.

Did the model complete the task? Did it retain important information, avoid clear factual errors, and support its conclusions with evidence? These are more useful questions than a blanket label about the model's political position.

When a response conflicts with my prior understanding, I can inspect the competing evidence and consider whether the model is wrong or my understanding needs to change. But prior understanding is not infallible. Important conclusions warrant independent checks even when an answer looks reasonable.

With data contamination and GEO, that means tracing where cited evidence originated. Agreement among several models does not prove independent verification if they all depend on the same search results or sites that repeat the same document. Multiple models should not simply become a voting system.

The task also determines what fidelity requires. A summary or translation should preserve important information from the source and clearly label any separate concerns about factual accuracy. A fact-check should independently investigate claims rather than merely restate the input. An analysis can go beyond the text, but should mark the distinction between observation, inference, and unknowns.

A model should not use its own evidence assessment as an unannounced license to redefine the user's task. Quietly deleting information from a requested summary can be a failure even if the remaining sentences contain no obvious falsehoods. Conversely, simply reproducing the source without checking anything fails a fact-checking request.

Reliability has to be measured against the actual job.

Where cross-language research could go next

The ROCLING study established language-sensitive position choices under its political questionnaire. Understanding causes requires separating variables.

One experiment could test the same questions, model versions, and generation settings under both forced-stance and open-ended factual formats. Another could fix the original evidence and change only the prompt language, then measure whether the model preserves the same facts and uses consistent standards.

For search-enabled systems, researchers could hold the model fixed while changing retrieval sources, distinguishing more stable model behavior from effects of external information selection.

A GEO study should separately measure changes in retrieval visibility, citation frequency, and answer accuracy. Being cited more often must not count as proof of improved quality.

Studies of political-content alignment need comparisons across weights, post-training versions, and system rules, rather than reconstructing internal training decisions from a model's stated refusal reason.

The metrics should also extend beyond political-stance alignment. A response's expressed position, preservation of key facts, unsupported interpretations, fabricated details, and refusal behavior are distinct outcomes. Examining them separately would reveal more than a single agreement score.

Open weights create additional opportunities to compare deployments, adjust models, and replace unsuitable configurations. But downloading weights does not erase behaviors acquired during pretraining or post-training. Local deployment changes who operates a model, not necessarily the boundaries already embedded in it.

Similarly, online search gives models access to newer sources without automatically providing a trustworthy method for evaluating them.

The useful goal is to make inputs, retrieval sources, outputs, and failure modes independently testable.

The boundary that matters most

Cross-language political differences, training-data contamination, GEO, and selective omission during local model tests occur at different points in an AI system. Yet they point toward the same larger distinction:

What information a model can encounter is not the same as what information it ultimately uses to form an answer.

Training corpora can reflect uneven linguistic information environments, ordinary data-quality problems, and deliberately planted attacks. Search can expand access while introducing dependence on content optimization and retrieval vulnerabilities. Even when the relevant text is already present, a model may omit it, overinterpret it, or invent facts.

More parameters, more languages, internet access, and open weights are valuable capabilities, but none guarantees a more objective answer by itself.

The difficult questions concern relationships among sources and judgments: what derives from primary evidence, what is only repeated distribution, what is verifiable, what remains an interpretation, and how conflicting evidence should be weighted.

When credible new evidence appears, a reliable model should be capable of revising its position. Where evidence is already strong, it should not abandon its judgment merely to appear balanced.

Models should neither disagree simply to disagree nor agree merely to please.

As more content is created with the goal of appearing in AI answers, it becomes especially important not to confuse repetition with independent confirmation, visibility with credibility, or a citation with completed verification.

Multilingual capability lets a model read across information environments. Open weights let more people inspect and modify it. GEO changes the incentives of those publishing for AI-mediated discovery. None replaces the fundamental requirement to evaluate evidence.

The boundary that models most need to cross may be not merely between languages, but between encountering information, preserving it, and reliably judging it.

A model capable of explaining the world in many languages has not necessarily escaped the blind spots of those languages' information environments. An enormous corpus does not automatically confer reliable judgment if the system cannot identify where evidence comes from, why it should be trusted, and where a conclusion extends beyond what the evidence supports.

In the end, I would rather assess models using a practical and testable standard:

Do they handle the material faithfully, form judgments on reliable evidence, and allow those judgments to be tested by new evidence?