You operate an AI.
But do you know how it really behaves?

An AI system is only as good as its actual behavior, whether in operation or during development. Tone, stance and statements can drift, with potentially liability-relevant consequences. Wesselmeier KI-Lösungen analyzes possible deviations in the language behavior of your model.

To the system behavior analysis Request a conversation
For existing AI systems Twelve test categories Reproducibly documented test protocol
Starting point

Large language models behave like employees with an invisible file.

Origin, training and calibration determine their behavior. Most operators see only the result, not the cause.

Three typical weaknesses show up in practice. One of them arises not from missing knowledge but from processing structure:

Example: Processing-induced misinformation
„Beide vermeiden echte Konfrontation. Beide verhindern echte Nähe."
English translation of the German model output: “Both avoid real confrontation. Both prevent real closeness.” Source: Claude Sonnet 4.6, on the disorganized and the anxious-preoccupied attachment style

The second half of the statement is false: an anxious-preoccupied attachment style does not prevent closeness. It actively seeks it. This faulty statement arose because no sequential verification step takes place. The parallel structure with a double “Both” is coherent and rhetorically strong. It raises the probability of the next token regardless of truth. The pattern pulls the content along, not the other way around.

This error type is not model-specific but architecture-related and occurs in all common LLMs.

The typical weaknesses:

01

Style inconsistency

The model answers semantically identical questions in measurably different styles, depending on wording, context length or preceding topics.

02

Sycophancy

Under user pressure, the model changes correct statements, not because new arguments exist but because disagreement was rated negatively in training.

→ Potentially liability-relevant for legal or medical information.
03

Register breaks & drift

In longer conversations the model leaves its defined role. Technical language and tone change unnoticed. This is measurable, but invisible without systematic testing.

System behavior analysis

Understanding language behavior, not just measuring it.

Structured analysis of language behavior for anyone who needs an independent assessment that a purely technical analysis cannot deliver.

Methodology & protocol

Structured test protocol with at least three runs per category. Attacker and judge models are separate from the target model, for clean, reproducible results.

Rating on a scale from 1 to 5, calibrated on the tested model. No overall score, only the complete category profile. Runs entirely on-premise.

Rating anchors are developed individually for each model: no generic grid, but calibrated to your specific use case.

The analysis describes system behavior at the time of testing.

  • Categories12 test categories
  • Minimum runs3 per category
  • RatingScale 1 to 5, category profile
  • PipelineAttacker → Target → Judge
  • InfrastructureOn-premise, no data leakage
  • Field of applicationRegulated professions & specialized models
Group A: Pressure-based

Sycophancy index

Stability of correct statements under user pressure.

Group A: Pressure-based

Boundary behavior

Handling of borderline or risky requests.

Group A: Pressure-based

Linguistic robustness

Behavior with faulty or informal input.

Group B: Consistency

Consistency across paraphrases

Equivalence of answers when a question is phrased differently.

Group B: Consistency

Cultural calibration

Fit for the DACH legal and communication context.

Group B: Consistency

Multilingual coherence

Stability of behavior when switching languages.

Group C: Longitudinal

Style consistency

Adherence to the defined style over long conversations.

Group C: Longitudinal

Persona drift

Stability of the defined role over time and topic changes.

Group D: Knowledge

Hallucination rate

Factual correctness on domain-specific questions.

Group D: Knowledge

Terminological precision

Correct and consistent use of technical terms.

Group D: Knowledge

Register fidelity

Uniformity of language register within a response.

Group D: Knowledge

Depth of explanation

Adapting answer depth to the recognizable level of knowledge.

System behavior adaptation

Behavior that must stay stable has to go into the weights.

A system prompt sits at the beginning of the context window. With every turn its attention weights decline, until it no longer competes effectively with the immediate conversation context. Prompt engineering does not solve this. At best it pushes the onset turn further back.

Fine-tuning writes behavior into the weights. Permanently.

Method A: RAG

Knowledge at runtime

The model retrieves knowledge. It knows it, as long as the document is available. Current, with source citation.

Method B: Fine-tuning

Behavior internalized

The model internalizes a way of working. It is it, independent of context and independent of conversation length. Technical language and tone stick.

Method C: Combination

Both together

Internalized behavior through fine-tuning, stable across every turn. Current knowledge through RAG, without a knowledge cutoff. The strengths of both approaches, structurally separated.

1

Requirements analysis

What should the model be able to do, and what explicitly not? Concrete behavior is defined before a single line of training data is created.

2

Data strategy & corpus selection

Which data fits the use case, and why? A reasoned expert recommendation instead of trial and error.

3

Synthetic data & annotation

Where no corpus exists, one is built. Expert annotation is done manually, not automated.

4

Implementation on your infrastructure

When building a model, it runs locally and in compliance with the GDPR. No black box, the model belongs to you. When analyzing existing systems, I work on the infrastructure already in place, whether local or cloud.

5

Evaluation & measurable proof

The rating grid shows, measurably and documented, whether the model does what it should and where it deviates.

More services

01
Model building

Specialized model from scratch

For anyone who does not yet run their own AI system.

  • Reasoned data strategy instead of trial and error
  • Fine-tuning, RAG or a combination
  • Measurable rating grid before and after adaptation
02
On-premise

Local installation

A compact server in your premises. The AI runs exclusively on your hardware, without a connection to external services, installed and ready for operation on compatible hardware.

  • Operation without cloud, without third parties, no data leakage
  • § 203 StGB (German professional secrecy law) & GDPR addressed structurally
  • Training for productive use
03
Behavior optimization

Targeted improvement after the analysis

Where the expert analysis identifies weaknesses, I develop targeted countermeasures, precisely matched to the results of the system behavior analysis.

  • Measures derived directly from the analysis results
  • Fine-tuning, prompt optimization or a combination
  • Follow-up analysis as measurable proof
Current project

Fallkonzeption

Fallkonzeption is not a system that decides, but distilled clinical literature that names patterns for you to weigh yourself.

Go to Fallkonzeption.de
Data security

Structurally secure, not just contractually.

The server is at your site. Sensitive data does not leave your premises. Professional secrecy under § 203 StGB and the requirements of the GDPR are thereby addressed structurally, not by a promise, but by the architecture.

For a legal assessment in an individual case, we recommend consulting your responsible chamber or a specialized lawyer.

  • No data leakage through local operation
  • HTTPS encryption
  • Two-factor authentication
  • Individual user management
  • Encrypted automatic backups
  • Remote maintenance only with your approval
Background

Dr. Hendrik Wesselmeier

Dr. Hendrik Wesselmeier

Neurolinguistics as an analytical tool for machine language behavior.

LLM errors have so far been classified at a technical-statistical level, not at the level of language. A neurolinguistic background makes it possible to name error types such as register breaks, presupposition errors or semantic drift, and to measure where others only see symptoms.

Wesselmeier KI-Lösungen is aimed at companies and regulated professions that want to use AI seriously, in a data-protection-compliant way and with traceable system behavior.

Contact

Get in touch.

info@wesselmeier.com