TSNBench：评测大模型在 TSN 领域能力的基准

Recent advances in large language models (LLMs) across different domains such as engineering (Jackson et al., 2025; Guo et al., 2025), medicine (Xie et al., 2025; Liu et al., 2023; Li et al., 2024), clinical practice (Kweon et al., 2024), computer networking (Sharma and Yegneswaran, 2023), telecommunications (Maatouk et al., 2026; Ferrag et al., 2026; Oluwaseyi et al., 2025; Gajjar et al., 2025), and automation (Shen et al., 2024) have shown groundbreaking performance in assisting engineers, practitioners, researchers (Huang et al., 2023; Sun et al., 2024), and doctors in solving real-world problems. System engineers are increasingly using LLMs to design and configure networks (Wang et al., 2024a), generate code, and analyze network logs. With this, they are entering new territory: safety-critical application domains such as autonomous vehicles, aerospace (Fiori et al., 2024; Sanchez-Garrido et al., 2021), defense (Elliott, 2023), and industrial communication (Zhang et al., 2024). In these contexts, the accuracy, reliability, and consistency of LLMs become far more than leaderboard metrics, as they become engineering requirements.

Time-Sensitive Networking (TSN) (31), standardized by the IEEE 802.1 Working Group (WG), is a layer-2 Ethernet technology that provides deterministic communication guarantees for safety-critical applications. TSN deployments typically separate traffic based on timing criticality. Safety-critical periodic communication with guaranteed latency and bounded jitter is categorized as time-triggered (TT) (Ademaj et al., 2019) traffic and is served using the IEEE 802.1Qbv timed-gate mechanism. TT transmissions are controlled by a Gate-Control List (GCL), computed offline using exact methods such as SMT-based synthesis (Craciunas et al., 2016) or heuristic approaches (Pop et al., 2016; Gavriluţ et al., 2018; Bujosa et al., 2022). In contrast, periodic or sporadic communication requiring bounded end-to-end latency but less stringent jitter control is classified as Audio Video Bridging (AVB) stream traffic (Böhm and Wermser, 2021; Bruckner et al., 2019). Consequently, Worst-Case Delay (WCD) estimation errors of tens or hundreds of microseconds are significant, as they can consume timing margins, violate deadlines, or lead to infeasible TSN configurations. In mission-critical deployments, such errors can have severe consequences. A misconfigured TSN network can cause, for example, a robotic arm to miss a critical assembly step, a brake system to fail on a highway, an aircraft control system to respond incorrectly, a defense mechanism to collapse, or a spacecraft to miss a vital signal. These failures may result from sub-millisecond timing violations caused by a single misconfiguration. These risks highlight the importance of accurate analysis and configuration in TSN systems, especially as LLMs are increasingly integrated into network management workflows. Therefore, their domain proficiency must be rigorously evaluated. However, to the best of our knowledge, no existing benchmark evaluates LLM proficiency in TSN.

To fill this gap, we introduce TSNBench, the first benchmark for evaluating LLM proficiency in TSN, comprising two complementary evaluation components. The first is a 939-question expert-validated multiple-choice question and answer (MCQA) dataset, generated from 83 peer-reviewed research papers using three LLMs from distinct model families and rigorously reviewed by five domain experts, each with over eight years of TSN research experience. The second is a set of open-ended questions requiring multi-step WCD computation for two widely deployed TSN mechanisms, namely Credit-Based Shaper (CBS) (32) and Cyclic Queuing and Forwarding (CQF) (33; J. Yan, W. Quan, X. Jiang, and Z. Sun (2020)), across varying network topologies and traffic flows, with ground truth computed using a verified Network Calculus (NC) solver (Zhao et al., 2018) for CBS and closed-form mathematical upper bounds for CQF (Wang et al., 2023). These open-ended WCD questions are intended as a closed-book stress test of standalone model capability, rather than as a deployment workflow for free-text LLM timing outputs. Detailed background on TSN, NC, CBS, and CQF is provided in Appendix 7, 8, 9, and 10, respectively.

While general-purpose benchmarks such as MMLU (Hendrycks et al., 2021) and MMLU-Pro (Wang et al., 2024b) evaluate broad subject knowledge spanning elementary mathematics, history, and law, they are fundamentally unsuited for safety-critical domain-specific evaluation. Answering a multiple-choice question about elementary school history is categorically different from answering TSN terminology questions and correctly computing a WCD under NC constraints for a given network topology. Without a benchmark that captures this distinction, there is no principled way to measure LLM progress in deterministic networking domains. TSNBench is designed precisely to expose this gap.

We evaluate 16 LLMs comprising open-source and closed-source models, as well as general-purpose and reasoning-specialized architectures. Our results reveal a striking dissociation, where models achieve 67 to 95% accuracy on MCQA yet fail substantially on open-ended WCD computation. The best-performing model, GPT-5, achieves a Mean Absolute Percentage Error (MAPE) of 36.2% on CBS, while most models exceed 80%. This is concerning in a domain where timing violations of tens of microseconds, even 1% of a 1000 μ \mu s deadline, may cause system failures.

Our key contributions are: 1. First expert-validated TSN benchmark: TSNBench evaluates LLM knowledge of TSN mechanisms through 939 expert-validated MCQs derived from peer-reviewed TSN literature. 2. Open-ended timing-analysis tasks: TSNBench includes open-ended WCD computation tasks for CBS and CQF with ground truth computed using a verified NC solver for CBS and closed-form mathematical bounds for CQF. 3. Evaluation across 16 LLMs: We evaluate both open-source and closed-source models, including general-purpose and reasoning-specialized models, and show that high MCQA accuracy does not reliably predict accurate WCD computation.

First expert-validated TSN benchmark: TSNBench evaluates LLM knowledge of TSN mechanisms through 939 expert-validated MCQs derived from peer-reviewed TSN literature.

Open-ended timing-analysis tasks: TSNBench includes open-ended WCD computation tasks for CBS and CQF with ground truth computed using a verified NC solver for CBS and closed-form mathematical bounds for CQF.

Evaluation across 16 LLMs: We evaluate both open-source and closed-source models, including general-purpose and reasoning-specialized models, and show that high MCQA accuracy does not reliably predict accurate WCD computation.

In summary, TSNBench provides the research community with the first rigorous evaluation resource for LLM proficiency in TSN, offering valuable insights to both the real-time networking community exploring LLM-assisted TSN management and the machine learning community seeking to understand the limits of LLMs in safety-critical, computationally demanding domains.

Benchmarking and datasets are essential for measuring LLM progress and identifying key gaps and limitations (Hendrycks et al., 2021; Wang et al., 2024b). General knowledge benchmarks such as MMLU (Hendrycks et al., 2021) and MMLU-Pro (Wang et al., 2024b) evaluate broad subject knowledge including elementary mathematics, history, computer science, and law, using multiple-choice questions. Domain-specific benchmarks have extended this paradigm to medicine (Xie et al., 2025; Liu et al., 2023; Li et al., 2024), clinical practice (Kweon et al., 2024), law (Guha et al., 2023), code generation (Hua et al., 2025; Huang et al., 2024), and scientific research (Sun et al., 2024). While these benchmarks have driven significant progress, they are not designed to evaluate safety-critical networking tasks. Most rely on multiple-choice evaluation, and none assess whether a model can perform the multi-step computational reasoning required in safety-critical networking domains. TSNBench addresses this gap by introducing MCQA and open-ended WCD computation questions with ground truth verified by state-of-the-art NC solvers, providing an evaluation of TSN that no existing general benchmark captures.

In the last few years, several benchmarks have evaluated LLM proficiency in networking and telecommunications domains. TeleQnA (Maatouk et al., 2026) presents an MCQ dataset for telecommunications, generated from research documents and 3GPP standards and validated by domain experts. 6G-Bench (Ferrag et al., 2026) presents an MCQ-based dataset for 6G networks containing 3,722 difficult questions validated through automated filtering and expert human review. Beyond question-answering benchmarks, NetConfEval (Wang et al., 2024a) evaluates LLMs on network configuration tasks and demonstrates that LLMs can simplify and automate complex network management tasks.

The application of LLMs to TSN management and orchestration is still at a very early stage, with only limited initial studies available. Windmann et al. (2025) explored the use of LLMs for configuring hybrid 5G/TSN networks by assisting users with manual configuration tasks and suggesting configurations in a 5G-TSN network. However, this work remains preliminary and does not provide experimental results. Overall, prior work does not provide a systematic benchmark or rigorous evaluation of LLM proficiency across TSN mechanisms, nor does it assess computational reasoning capabilities for WCD analysis. TSNBench fills this gap by providing the first structured benchmark covering both declarative TSN knowledge through MCQA and computational reasoning through open-ended WCD evaluation.

Unlike established domains such as medicine (Xie et al., 2025), 5G (Oluwaseyi et al., 2025; Maatouk et al., 2026), general human knowledge (Phan et al., 2026; Hendrycks et al., 2021; Wang et al., 2024b), coding (Hua et al., 2025; Huang et al., 2024), and law (Guha et al., 2023), no open-source TSN dataset exists for LLM evaluation (Zhang et al., 2024; Peng et al., 2023; Zanbouri et al., 2025; Adil et al., 2026). As highlighted in (Liu et al., 2023), the data source determines the reliability of a dataset, and generating a high-quality dataset is a crucial prerequisite for meaningful benchmarking. We describe the TSNBench construction pipeline below, with full details provided in Appendix 11.

Published research papers and standards are among the most reliable sources for building domain-specific datasets (Liu et al., 2023). Since TSN knowledge originates primarily from peer-reviewed research and IEEE 802.1 TSN standards, we curate a collection of open-access research documents as our source corpus. To avoid copyright issues and exclude papers with incorrect results or flawed methodologies, we include only published open-access papers. For papers not available in open-access form, we use arXiv versions that have been published or accepted, excluding unpublished preprints with unverified results. Where possible, we also collect author manuscript versions with proper attribution. To ensure quality, we prioritize highly cited papers from reputable venues while accounting for publication timeline, as recent papers naturally have fewer citations. In total, we collect 83 research papers covering a broad range of TSN mechanisms, including Time-Aware Shaper (TAS), CBS, CQF, NC-based schedulability analysis, performance evaluation, hardware experiments, combined shapers such as TAS+CBS (Zhao et al., 2022), and Multi-CQF (Alexandris et al., 2022). Detailed background on TSN, related work, and its mechanisms is given in Appendix 7.

TSN employs specialized vocabulary, similar to other communication domains (Andrews et al., 2014; Saad et al., 2020; Ma et al., 2019). A successful LLM that understands TSN should be able to reason correctly about TSN terminology. A model that cannot differentiate between TAS and CBS, or cannot correctly expand TSN-specific acronyms, cannot be considered proficient in TSN. To capture this dimension, we extract keywords and acronyms widely used in TSN literature and use them to guide MCQA generation. All terms are extracted from the 83 research documents using Claude Sonnet 4, as shown in Table 1, and stored in JSON format. Each document is preprocessed to remove non-relevant content, including author names, affiliations, figures, tables, URLs, and pseudocode. The model is instructed to extract only terms defined within the document, without relying on pretrained knowledge, and to provide each term’s acronym, full form, and one-to-two-sentence definition from the source. The extracted set is then reviewed by domain experts to resolve duplicates, retaining the longer definition in cases of conflict. Figure 1 illustrates this pipeline.

To optimize time and reduce manual effort, we use an LLM-based approach to generate MCQAs from research documents. The keyword file is provided alongside the research documents as additional input, serving as an independent source to complement research paper content during generation. We use three models from distinct families, namely Claude Sonnet 4, GPT-4o mini, and Llama 3.1 70B, as shown in Table 1. These models are deliberately selected to ensure diverse styles and reasoning capabilities, thereby reducing generative bias. The same system prompt is used for all models, and each research paper is assigned to exactly one model in a round-robin manner. Non-relevant sections, such as author information, affiliations, references, URLs, figures, tables, and pseudocode, are removed from each document before generation.

LLM-generated MCQAs cannot be used directly for benchmarking, as they may contain incorrectly formulated questions, incomplete options, or vague and incorrect answer choices. To address positional bias introduced by the generating model, answer options are shuffled randomly prior to human expert review, with the correct answer label updated to reflect the new ordering.

Given the safety-critical nature of TSN, rigorous human validation is essential. We engage five TSN domain experts: three senior professors with more than 15 years of research experience and two postdoctoral researchers with more than 8 years of expertise. Each question is independently evaluated with four outcomes: (i) accept - correct and clear; (ii) revise - requires modification for clarity or correctness; (iii) reject - the question is incorrect, misleading, or irrelevant; or (iv) doubtful - the expert is uncertain and passes it to remaining reviewers for consensus. Questions without consensus are discarded. Full review criteria are provided in Appendix 11.1 and Table 5. Table 2 summarizes the dataset statistics and Figure 2 illustrates the full pipeline.

While MCQA evaluates declarative TSN knowledge, open-ended questions assess whether LLMs can perform the multi-step mathematical reasoning required in real TSN deployment. We evaluate WCD computation, as WCD is a central key performance indicator (KPI) in TSN network design and directly determines whether a network meets its stringent timing requirements. We select two TSN mechanisms for this evaluation: CBS and CQF. CBS is widely deployed for audio-video traffic and requires NC-based analysis, making it mathematically demanding. CQF is a more recently standardized TSN mechanism whose WCD can be computed from a closed-form equation given routing and cycle duration (T T), providing a complementary evaluation that isolates formula application from NC complexity. Together, these two mechanisms span a meaningful range of WCD computation difficulty. Ground truth WCD values are computed using a verified state-of-the-art NC tool (Zhao et al., 2018) for CBS and closed-form mathematical upper bound for CQF. We release all ground truth WCD values alongside the questions to support future open-source community evaluations. Each open-ended question is formulated by domain experts, as shown in Figure 3, and comprises three components: network topology, flow information, and flow routing. In TSNBench, three topologies are used to cover a broad range of scenarios: (i) one-switch topology (Figure 15), (ii) medium-mesh topology (Figure 16), and (iii) ring topology, representing industrial networks (Figure 17). Each topology consists of end nodes and switches connected via Ethernet links, with unicast traffic flows transmitted from a sender to a single receiver. Flows consist of Ethernet frames whose maximum payload is bounded by the Maximum Transmission Unit (MTU). Further topology, flow, and routing details are provided in Appendix 11.4.

For both MCQA and open-ended evaluations, each prompt defines the model’s role as a TSN expert. For MCQA, we use zero-shot prompting with no in-context examples, representing a conservative approach that measures inherent TSN proficiency, ensuring that the output performance reflects the model’s domain knowledge rather than in-context pattern matching. For open-ended questions, we also use a zero-shot setting, providing no example WCD calculations or NC or CQF equations, ensuring the model independently recalls and applies the correct computational methodology. For both question types, the model is asked to provide a confidence score alongside its answer.

The open-ended prompt comprises three variable components: network topology, flow parameters, and pre-computed shortest path routes. The same prompt template is used across all 100 open-ended evaluation instances per mechanism, with only these three components varying. Fixed network constants are maintained throughout to ensure comparability across models and instances. A detailed discussion of the open-ended prompt design is provided in Appendix 11.3.

For the MCQA dataset, performance is measured as the percentage of questions answered correctly, reported as accuracy. For the open-ended questions, we evaluate the computational reasoning capability of each model by comparing its predicted WCD values against ground truth values. For CBS, ground truth WCD values are derived using NC-based Total Flow Analysis (TFA). Specifically, the worst-case delay upper bound D f h D_{f}^{h} for flow f ∈ ℱ M i h f\in\mathcal{F}_{M_{i}}^{h} at h h equals the worst-case delay upper bound D M i h D_{M_{i}}^{h} for all flows with the same priority M i M_{i} aggregating at h h, D f h = D M i h = h ​ D ​ e ​ v ​ (α M i h, β M i h) = sup t ≥ 0 { inf { τ ≥ 0 ∣ α M i h ​ (t) ≤ β M i h ​ (t + τ) } }, D_{f}^{h}\!=\!D_{M_{i}}^{h}\!=\!hDev(\alpha_{M_{i}}^{h},\beta_{M_{i}}^{h})=\!\sup_{t\geq 0}\left\{\inf\left\{\tau\!\geq\!0\mid\alpha^{h}_{M_{i}}\!(t)\leq\beta^{h}_{M_{i}}\!(t\!+\!\tau)\right\}\right\},\\ (1) where α M i h ​ (t) \alpha_{M_{i}}^{h}(t) represents the arrival curve of aggregate flows of priority M i M_{i} passing through h h, and β M i h ​ (t) \beta_{M_{i}}^{h}(t) represents the service curve for these corresponding flows. The end-to-end WCD for a flow is obtained by summing per-port delay bounds along its route. Full NC methodology and proofs are provided in Appendix 8.

For CQF, the worst-case end-to-end delay is given by the closed-form expression WCD = f i. ϕ + (SW num + 1) ⋅ T + ξ, \mathrm{WCD}=f_{i}.\phi+(\mathrm{SW_{num}}+1)\cdot\mathrm{T}+\xi, (2) where f i. ϕ f_{i}.\phi is the flow offset at the source node in μ \mu s, SW num \mathrm{SW_{num}} is the number of switches along the flow route, T \mathrm{T} is the cycle duration in μ \mu s, and ξ \xi denotes the network specific delays including processing delay, propagation delay, switching delay, and time synchronization error. The derivation and proof of this bound are provided in Appendix 10.

We evaluate 16 state-of-the-art LLMs spanning open-source and closed-source models across general-purpose and reasoning-specialized architectures. Table 6 in Appendix 12 provides the full list of models with their model IDs and organizations. All models are accessed via their respective official vendor APIs with no fine-tuning applied: GPT (OpenAI API), DeepSeek (DeepSeek API), Mistral (Mistral AI API), Claude (Anthropic API), Gemini (Google AI API), Grok (xAI API), and Llama and Qwen (Hugging Face inference router). All client-side operations, including prompt construction, API handling, response parsing, and metric computation, are performed on a standard workstation. To assess repeatability and stochasticity, each MCQA and open-ended question is evaluated three times under two temperature settings: deterministic (T = 0.0) and stochastic (T = 0.7). Since TSN is widely used in safety-critical domains, deterministic responses are essential, as non-determinism would undermine the reliability of LLM-based TSN reasoning. For models that do not expose a temperature parameter, evaluations use the vendor default configuration, as noted in Table 5. Full cost and latency details are provided in Appendix 12, Table 8.

Since the MCQs were generated using models from families included in the evaluation, as shown in Table 1, contamination is a potential concern. We therefore separate the evaluated models into generator families (Claude, GPT, Llama) and non-generator families (all remaining models) and compare their average MCQA accuracy. Generator-family models achieve an average accuracy of 88.8%, whereas non-generator-family models achieve 91.0%. The generator-family models do not perform better than the non-generator-family models, so we do not observe evidence of a systematic advantage. This analysis does not rule out all possible contamination pathways, but it addresses this specific concern. The open-ended timing tasks are less likely to be affected because their topology, flow, and routing inputs were constructed specifically for TSNBench.

Evaluation Metrics: Model performance on the MCQA dataset is measured using accuracy, defined as the percentage of correctly answered questions out of 939, averaged across three runs. We additionally report Expected Calibration Error (ECE) (Pavlovic, 2025) and Brier score (Hoessly, 2026) to evaluate the alignment between the model’s expressed confidence and its actual correctness. Calibration is particularly critical in safety-critical domains such as TSN, where high-confidence incorrect answers may lead to misleading configuration decisions, deadline violations, or network instability in industrial and automotive systems. We therefore also evaluate the Confidently Wrong (CW) rate to determine the fraction of incorrect answers where the model expresses high confidence (≥ \geq 0.8). All calibration metrics are computed on the full 939-MCQA dataset across three runs per model.

Results and Discussion: Table 5 reports accuracy, average (avg.) consistency, calibration, and average latency for all 16 models. The top performers are Claude Sonnet 4.5 (95.3%) and GPT-5 (95.0%), with Claude Sonnet 4.5 also achieving the lowest Brier score (0.0429), indicating strong accuracy and calibration. Llama 3.2 1B achieves the lowest accuracy (67.4%), consistent with its substantially smaller parameter count compared with the other models.

A notable finding emerges from the reasoning models. Despite their stronger general reasoning capabilities, o3, GPT-5, and DeepSeek-V3.2 (Thinking) do not outperform the best non-reasoning models on MCQA, all scoring below Claude Sonnet 4.5. This suggests that TSN MCQA performance is primarily driven by domain knowledge rather than general reasoning, and that reasoning-specialized architectures offer limited advantage on declarative knowledge retrieval tasks.

The calibration results reveal key differences across models. While most models are well-calibrated (ECE < 0.06), o3 has the highest ECE (0.1874) despite 94.7% accuracy, yet achieves the lowest CW rate (3.4%), rarely assigning high confidence to incorrect answers (refer to Figure 5). In contrast, many non-reasoning models have CW rates of 100%, assigning high confidence to incorrect answers. Mistral Medium 3.1 has the highest average confidence (0.9779) while maintaining 92.1% accuracy. All models have zero refusal rate, indicating that the MCQA dataset does not trigger response refusals.

Figure 6 presents the reliability plot for all 16 evaluated models on the MCQA dataset.

Each diagram shows the observed accuracy against the model’s expressed confidence, binned across the confidence range. A perfectly calibrated model would fall on the gray dashed diagonal line. This means the model’s confidence would perfectly align with its actual accuracy. The red shaded region indicates overconfidence, meaning the model’s confidence exceeds its actual accuracy. The green shaded region indicates underconfidence, meaning the model is more accurate than its expressed confidence suggests.

In safety-critical TSN deployments, overconfidence is significantly more dangerous than underconfidence. A model that is incorrect but expresses high confidence may mislead a network engineer with an erroneous WCD estimate or misconfigured scheduling parameters. By contrast, an underconfident model that expresses uncertainty on correct answers prompts additional verification.

The majority of the evaluated models sit in the high-confidence region (0.8 to 1.0) regardless of their actual accuracy. This indicates that the models tend to exhibit overconfidence.

Grok 4.1 Fast (NR), Mistral Medium 3.1, Mistral Large 3, and Ministral 3 8B achieve CW rates of 100%, meaning all incorrect answers fall in the high-confidence range. This represents the most critical calibration behavior for TSN deployment. GPT-4o, Gemini 2.5 Flash, Llama 3.2 1B, and Qwen3 8B similarly exhibit CW rates exceeding 95%. A notable exception is o3, which is the only model that falls predominantly in the green underconfident zone, with a CW rate of just 3.4%. Despite having the highest ECE (0.1874) among all evaluated models, o3 is the safest among the evaluated models from a calibration perspective, as it rarely expresses high confidence on incorrect MCQA answers. This highlights an important distinction between aggregate calibration metrics and safety-relevant calibration behavior. DeepSeek-V3.2 (NT) achieves the lowest ECE (0.0105), suggesting strong overall calibration, yet maintains a CW rate of 96.4%, demonstrating that a low ECE does not guarantee safe and realistic confidence behavior.

Evaluation Metrics: For the open-ended questions, we report two widely used metrics: Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE), computed per test case (TC). Each TC consists of n n flows, denoted f i f_{i} where i i = 1 ​ ⋯ ​ n 1\cdots n. For each flow f i f_{i}, y ^ T ​ C x, f i \hat{y}_{{TC}_{x},f_{i}} denotes the WCD predicted by the model for T ​ C x {TC}_{x} and y T ​ C x, f i y_{{TC}_{x},f_{i}} denotes the ground truth WCD of flow f i f_{i} for TC number x x, computed using a verified NC solver for CBS and using Eq. 22 for CQF. The MAE for each TC is defined as: MAE T ​ C x = 1 n ​ ∑ i = 1 n | y ^ T ​ C x, f i − y T ​ C x, f i |, \text{MAE}_{{TC}_{x}}=\frac{1}{n}\sum_{i=1}^{n}|\hat{y}_{{TC}_{x},f_{i}}-y_{{TC}_{x},f_{i}}|, (3) where x x denotes the TC index and x ∈ { 1, …, 100 } x\in\{1,\ldots,100\}. The MAPE for each TC is defined as: MAPE T ​ C x = 1 n ​ ∑ i = 1 n | y ^ T ​ C x, f i − y T ​ C x, f i | y T ​ C x, f i × 100 \text{MAPE}_{{TC}_{x}}=\frac{1}{n}\sum_{i=1}^{n}\frac{|\hat{y}_{{TC}_{x},f_{i}}-y_{{TC}_{x},f_{i}}|}{y_{{TC}_{x},f_{i}}}\times 100 (4) The overall MAE and MAPE for a model are obtained by averaging across all 100 TCs: MAE = 1 100 ​ ∑ x = 1 100 MAE T ​ C x, MAPE = 1 100 ​ ∑ x = 1 100 MAPE T ​ C x \text{MAE}=\frac{1}{100}\sum_{x=1}^{100}\text{MAE}_{{TC}_{x}},\qquad\text{MAPE}=\frac{1}{100}\sum_{x=1}^{100}\text{MAPE}_{{TC}_{x}} (5) We additionally report the median MAE across TCs as a robust measure against outlier TCs. Further example and details on the evaluation metrics are provided in Appendix 13 and Table 9.

Results and Discussion: Table 3 presents the WCD computation results for both CBS and CQF across all 100 TCs. The central finding is a striking dissociation between MCQA accuracy and computational reasoning performance. Models that achieve above 90% accuracy on MCQA still fail substantially on open-ended WCD computation, with the best-performing model, GPT-5, achieving a median MAE of 92.4 μ \mu s on CBS, which is concerning because industrial TSN traffic can have strict timing requirements (Ekrad et al., 2025). Detailed per-TC results are provided in Appendix 13.

For CBS, most models produce large errors, with many exceeding 200 μ \mu s MAE and 70% MAPE. Several models exhibit distinct failure modes. On CBS, Llama 3.2 1B responds to fewer than 50 evaluated TCs, returning all-zero WCD values for few TCs and partially incorrect values for some TCs, with incomplete flow coverage in all responses. Grok 4.1 Fast (Reasoning) returns truncated JSON, providing flow profile metadata but no WCD values, suggesting that the model hit an output length limit. DeepSeek-V3.2 (Thinking) returns empty responses for more than 70 TCs across both mechanisms. The NC-based computation required for CBS is mathematically demanding and complex, and the zero-shot setting reveals that most models cannot independently recall or correctly apply the full NC methodology. Among models that produce valid CBS responses, GPT-5 achieves the best performance (MAE 150.2 μ \mu s, MAPE 36.2%). Notably, OpenAI reasoning models and Grok 4.1 Fast perform better on CBS than non-reasoning models, with GPT-5 achieving substantially lower MAE than all non-reasoning models, suggesting that multi-step mathematical reasoning capability provides an advantage for NC-based WCD computation even when it does not improve MCQA accuracy.

For CQF, performance is more varied, with median MAE ranging from 1.2 μ \mu s (GPT-4o) to 1,046 μ \mu s (Ministral 3 8B), and MAPE ranging from 41.8% (Mistral Large 3) to 1705.5% (Ministral 3 8B). GPT-4o achieves the lowest median MAE on CQF (1.2 μ \mu s, MAPE 61.9%) despite failing completely on CBS, suggesting it can correctly apply the CQF closed-form equation. Mistral Large 3 achieves the lowest MAPE on CQF (41.8%), indicating the most accurate relative WCD estimation across all evaluated models. Llama 3.2 1B exhibits the most severe hallucination failure, fabricating up to 1,013 flows (flow 0–1012) instead of predicting WCD for the actual flows (fewer than 30 flows per TC), and returning WCD = 0 for all. Qwen3 8B fails to produce any response for either CBS or CQF due to repeated API timeouts. Ministral 3 8B, despite being a small model, produces valid responses for both CBS and CQF but with large errors (MAPE 25498.1% for CBS and 1705.5% for CQF), demonstrating that context handling is necessary but not sufficient for correct WCD computation.

Comparison across MCQA and open-ended questions: Figure 7 illustrates the performance differences between models across two evaluation types, MCQA and open-ended questions. The right-hand figure shows the MAE for a one-switch topology across different models, while the left-hand figure presents the MCQA accuracy. The MCQA accuracy remains high, above 80%, for all models except Llama 3.2 1B. However, the MAE is still significant for TSN flows with deadlines in the range of 1000 to 5000 μ \mu s. Figure 18 further presents the performance differences between models for MCQA and open-ended questions in a ring topology.

We present TSNBench, the first benchmark for evaluating LLM proficiency in Time-Sensitive Networking (TSN), comprising 939 expert-validated multiple-choice questions (MCQs) and 100 open-ended questions per mechanism for Credit-Based Shaper (CBS) and Cyclic Queuing and Forwarding (CQF). The ground truth WCD values are computed using a verified Network Calculus (NC) solver for CBS and closed-form mathematical upper bounds for CQF. We evaluate 16 LLMs and find that models achieve 67-95% MCQA accuracy yet fail substantially on open-ended WCD computation, with the best model (GPT-5) still achieving a Mean Absolute Percentage Error (MAPE) of 36.2% on CBS. Despite CBS being extensively researched and an older mechanism, models cannot correctly apply NC, whereas CQF, with its simpler closed-form equation, is handled more successfully, confirming that WCD computation performance is governed by mathematical complexity rather than mechanism maturity. TSNBench demonstrates that MCQ benchmarks substantially overestimate LLM capability in safety-critical domains.

Limitations and Future Directions: TSNBench has three primary limitations. First, the MCQA dataset is generated from open-access research papers, limiting coverage of certain mechanisms. Second, the open-ended evaluation covers only CBS and CQF. Extending to TAS is a natural next step, though its NP-hard gate control list (GCL) synthesis problem poses additional challenges beyond CBS and CQF. Third, the open-ended tasks evaluate standalone zero-shot model behavior under a closed-book prompt and should not be interpreted as a recommended deployment workflow for safety-critical TSN systems. Evaluating LLMs in settings where they produce checkable artifacts verified by deterministic analysis tools, and evaluating whether providing NC equations in the prompt improves WCD computation accuracy, are important directions for future versions of TSNBench.

While TSNBench fills a significant research gap and proposes a step forward towards evaluating TSN capabilities in LLMs, it has several limitations:

Dataset scope: TSNBench currently only covers CBS and CQF in open-ended questions. Evaluating other TSN mechanisms is necessary to fully cover the entire TSN mechanism.

Prompt Design: TSNBench does not provide any mathematical equation to the model as input for NC WCD calculation for CBS or the upper bound delay calculation of CQF.

MCQA Scope: MCQs are solely developed using published research papers and the IEEE standards are not used to generate the MCQs. Solving the license issue and utilizing standards to include MCQs using IEEE 802.1 standard will enhance the entire MCQA dataset.

Topology coverage: TSNBench open-ended question currently covers three different topologies: one-switch, medium-mesh, and ring topology. Covering diverse topologies and flow parameters will present a comprehensive evaluation.

To address the limitations of TSNBench, we propose the following additions and improvements in the future version of TSNBench. 1. Larger and more diverse dataset: Our current TSNBench dataset covers 100 TCs across three topology types. In future versions, we will include larger and more complex topologies with higher flow counts. As model performance improves, more complex open-ended evaluations should be integrated with complex topologies and combined TSN mechanisms. 2. Additional scheduling mechanisms: TSNBench currently evaluates CBS and CQF. Future versions should extend to TAS and ATS to cover a broader range of the TSN standard suite. 3. Updated MCQA: Our MCQA dataset was developed using open-source research documents. In future work, we will update the dataset with MCQAs formulated directly from TSN standards. 4. Fine-tuned and domain-adapted models. TSNBench currently evaluates general-purpose LLMs without any TSN-specific fine-tuning. Future versions should benchmark domain-adapted models trained on TSN standards and network calculus literature.

Larger and more diverse dataset: Our current TSNBench dataset covers 100 TCs across three topology types. In future versions, we will include larger and more complex topologies with higher flow counts. As model performance improves, more complex open-ended evaluations should be integrated with complex topologies and combined TSN mechanisms.

Additional scheduling mechanisms: TSNBench currently evaluates CBS and CQF. Future versions should extend to TAS and ATS to cover a broader range of the TSN standard suite.

Updated MCQA: Our MCQA dataset was developed using open-source research documents. In future work, we will update the dataset with MCQAs formulated directly from TSN standards.

Fine-tuned and domain-adapted models. TSNBench currently evaluates general-purpose LLMs without any TSN-specific fine-tuning. Future versions should benchmark domain-adapted models trained on TSN standards and network calculus literature.

TSNBench enables the real-time systems community and the machine learning community to objectively measure LLM performance and readiness for management and deployment assistance in safety-critical deterministic networks. By highlighting the critical aspects of TSN and the performance gap of the models between MCQA and computational reasoning, TSNBench alerts the incompetence of the models which may lead to misconfigurations and safety-critical issues. This benchmark provides a concrete direction to improve LLMs for deterministic networking. TSNBench further highlights the potential benefits of using LLMs thereby automating the management and deployment of TSN networks. Moreover, open-sourced ground truth WCD values computed by NC solvers provide a reliable resource for the entire community to further evaluate different benchmarking datasets.

While TSNBench is intended to advance research on LLM proficiency in TSN, we acknowledge the following potential negative impacts.

Overreliance on model outputs: Models trained on the open-access dataset provided by TSNBench may achieve high accuracy on WCD analysis tasks, which could lead practitioners to deploy such models directly in real-world deployments without independent verification. Any WCD values or network configuration decisions produced by an LLM should be verified using formally verified solvers and NC tools before real-world deployment.

False confidence from MCQA performance: Our results demonstrate that strong MCQA performance does not transfer to open-ended WCD estimation. A practitioner or system engineer who evaluates an LLM solely on MCQA benchmarks may incorrectly conclude that the model is suitable for TSN configuration tasks, leading to unsafe deployments in systems where timing guarantees are required.

Data contamination and benchmark overfitting: As TSNBench is released as an open-access dataset, future models may be trained directly on the benchmark questions, leading to inflated performance that does not reflect genuine TSN reasoning capability. We recommend that researchers introduce randomization in the test cases to prevent bias in results. Researchers should be cautious when interpreting results from models whose training data may overlap with the TSNBench dataset.

Misuse of the dataset: The dataset can be used to train models to configure TSN networks. Owing to the safety-critical nature of TSN applications, such models could potentially be exploited by attackers to manipulate network configurations, introduce timing violations, or deliberately cause deadline misses in industrial and automotive systems.

Time-Sensitive Networking (TSN) (Finn, 2018) is a set of amendments and additions to the IEEE 802.1 standards that, since its inception in 2012, has become one of the most relevant technologies for enabling deterministic and real-time communications over Ethernet networks. TSN extends standard Ethernet by introducing mechanisms for bounded latency, low jitter, and high reliability, making it suitable for applications such as industrial automation, automotive systems, and professional audio-video networks. Figure 8 showcases a simple TSN network with flows.

In TSN, communication between end-stations is based on the transmission of Ethernet frames across a network of interconnected Ethernet links and TSN switches. These switches, as well as the output ports of end-stations, implement a queuing architecture with up to eight First-In-First-Out (FIFO) queues, each associated with one of the eight traffic priorities defined in IEEE 802.1Q (31). TSN is not just limited to wired domain. The growing necessity of deterministic communication has extended to wireless domain gaining a significant interest in wireless-TSN networks. Although TSN is fundamentally an IEEE 802.1 bridged Ethernet technology, wireless and 5G-TSN (Debnath et al., 2023a) integration requires additional adaptation or translation functions, together with time-synchronization mechanisms that preserve deterministic latency guarantees across heterogeneous network segments. We showcase a 5G-TSN system in Figure 9, where TSN senders are sending mixed criticality traffic types to wireless receiver nodes over a TSN switch and 5G system in the network. Some of the most commonly used abbreviations in TSN are given in Table 4.

Frames are classified into traffic classes and assigned to egress queues based on their priority, with transmission selection typically governed by strict priority. Industrial TSN traffic is commonly categorized into traffic types such as isochronous traffic, cyclic-synchronous traffic, cyclic-asynchronous traffic, network-control traffic, alarms and events, configuration and diagnostics, and best-effort traffic (Ademaj et al., 2019). These traffic types require different timing guarantees: safety-critical isochronous traffic is typically mapped to time-triggered (TT) traffic, requiring guaranteed latency and bounded jitter, and is commonly handled by time-triggered mechanisms such as the Time-Aware Shaper (TAS) (Craciunas et al., 2016; Serna Oliver et al., 2018). In contrast, cyclic-synchronous or cyclic-asynchronous traffic that requires bounded end-to-end latency but less stringent jitter control is commonly mapped to AVB stream traffic and is often supported by the Credit-Based Shaper (CBS) (Zhao et al., 2018). TSN also defines mechanisms such as Asynchronous Traffic Shaping (ATS) (Specht and Samii, 2016; Debnath et al., 2023b; Nasrallah et al., 2019), Frame preemption (FP) (Debnath et al., 2024), and Cyclic Queuing and Forwarding (CQF) (Wang et al., 2023; Debnath et al., 2025a; Yan et al., 2020) to provide deterministic communication under different traffic and deployment assumptions.

These mechanisms regulate when and how frames are transmitted, allowing the network to provide guarantees such as bounded delay, jitter, and controlled bandwidth allocation. In the MCQA dataset of TSNBench, we covered the basics of different TSN mechanisms, including TAS, CBS, ATS, CQF, and CBS. The MCQAs are theoretical in nature and cover the basic understanding of the mechanisms without going into their mathematical or analytical details. In contrast, for the open-ended mechanisms, we evaluate the capability of the models to perform numerical analysis, formulate mathematical equations, and find the WCD values for the flows in the network. For this, we selected two TSN mechanisms: CBS and CQF. The WCD values of the flows using the CBS mechanism are calculated using NC analysis, which is mathematically complex. Therefore, we also evaluate the CQF mechanism as a simpler mechanism. The WCD values of the flows using the CQF mechanism can be directly calculated using the routing of the flow and the cycle duration. The detailed working mechanism and architecture of CQF and CBS are described in detail in the Appendix 9 and 10. The theory of NC is further explained along with the mathematical equations in Appendix 8.

Network Calculus (NC) is a theory for calculating worst-case bounds in communication networks based on min-plus algebra. Its basic paradigm involves two operators: convolution ⊗ \otimes

(f ⊗ g) ​ (t) = inf 0 ≤ s ≤ t { f ​ (t − s) + g ​ (s) }, (f\!\otimes\!g)(t)\!=\!\inf_{0\leq s\leq t}\{\!f(t\!-\!s)\!+\!g(s)\!\}, (6) and deconvolution ⊘ \oslash,

(f ⊘ g) ​ (t) = sup s ≥ 0 { f ​ (t + s) − g ​ (s) }. (f\!\oslash\!g)(t)\!=\!\sup_{s\geq 0}\{f(t\!+\!s)\!-\!g(s)\!\}. (7)

Based on this algebra, the arrival curve and the service curve are constructed to describe the maximum arrival traffic data and the minimum service capability over any time interval, respectively. In the hybrid TSN/TAS+CBS architecture, the service for ET traffic is constrained not only by the bandwidth reservation, but also by high-priority TT traffic. We adopt the state-of-the-art network calculus model (Zhao et al., 2021, 2024) to ensure deadline guarantees for ET flows with an arbitrary number of SR classes in the TSN/TAS+CBS architecture. Since, in our open-end CBS questions, we do not have any TAS mechanism, we use the TSN/TAS+CBS architecture without the TAS mechanism in it with only CBS mechanism for the AVB flows in the network.

As described in (Zhao et al., 2024), the service curve β ​ (t) \beta(t) is for constraining the minimum service capabilities, satisfying ℛ ∗ ​ (t) ≥ (ℛ ⊗ β) ​ (t). \mathcal{R}^{*}(t)\geq\left(\mathcal{R}\otimes\beta\right)(t). (8) The function ℛ ​ (t) \mathcal{R}(t) (resp. ℛ ∗ ​ (t) \mathcal{R}^{*}(t)) is the input (resp. output) cumulative function counting the total data bits of the flow that arrive at (resp. departure from) the server up to time t t. A typical example of a service curve is the rate-latency form, β R, T ​ (t) = R ​ [ t − T ] + \beta_{R,T}(t)=R[t-T]^{+} (9) with the service rate R R and latency T T. The notation [ x ] + [x]^{+} equals x x if x ≥ 0 x\geq 0, and 0 otherwise.

In the hybrid TSN/TAS+CBS architecture, the CBS service curve (Zhao et al., 2021) for the arbitrary SR Class M i M_{i} (i ∈ [ 1, N S ​ R ] i\in[1,N_{SR}]) with the impact of TT traffic at the output port h h is, β M i h ​ (t) = i ​ d ​ S ​ l M i h ​ [ t − α T ​ A ​ S h ​ (t) C − c M i h, max i ​ d ​ S ​ l M i h ] ↑ +, \beta^{h}_{M_{i}}(t)=idSl^{h}_{M_{i}}\left[t-\frac{\alpha_{TAS}^{h}(t)}{C}-\frac{c_{M_{i}}^{h,\max}}{idSl^{h}_{M_{i}}}\right]^{+}_{\uparrow}, (10) where c M i h, max c_{M_{i}}^{h,\max} is the credit upper bound for SR Class M i M_{i}, c M i h, max = i ​ d ​ S ​ l M i h ⋅ ∑ j = 1 i − 1 c M j h, min − l > i h, max ∑ j = 1 i − 1 i ​ d ​ S ​ l M j h − C, c_{M_{i}}^{h,\max}=idSl^{h}_{M_{i}}\cdot\frac{\sum_{j=1}^{i-1}c_{M_{j}}^{h,\min}-l^{h,\max}_{>i}}{\sum_{j=1}^{i-1}idSl^{h}_{M_{j}}-C}, (11) where l > i h, max = max j > i ⁡ { l M j h, max, l B ​ E h, max } l^{h,\max}_{>i}=\max_{j>i}\{l^{h,\max}_{M_{j}},l^{h,\max}_{BE}\} is the maximum frame size with priority lower than Class M i M_{i} at h h, l M j h, max l^{h,\max}_{M_{j}} is the maximum frame size of Class M i M_{i} at h h, and c M i h, min c_{M_{i}}^{h,\min} is the lower credit bound of SR Class M i M_{i}, c M i h, min = s ​ d ​ S ​ l M i h ⋅ l M i h, max C. c_{M_{i}}^{h,\min}=sdSl^{h}_{M_{i}}\cdot\frac{l^{h,\max}_{M_{i}}}{C}. (12) α T ​ A ​ S h ​ (t) \alpha_{TAS}^{h}(t) in Eq. (10) is the arrival curve of TT traffic scheduled by GCL.

The arrival curve α ​ (t) \alpha(t) is for constraining the arrival process of the flow, satisfying ℛ ​ (t) ≤ (ℛ ⊗ α) ​ (t). \mathcal{R}(t)\leq\left(\mathcal{R}\otimes\alpha\right)(t). (13) A typical example of an arrival curve is the burst-rate form,

α ​ (t) = b + ρ ⋅ t, \alpha(t)=b+\rho\cdot t, (14)

for t > 0 t>0 and 0 otherwise, with the parameters b b as the maximum burst tolerance and ρ \rho as the long-term rate of the flow.

For each ET flow f f at its source ES h 0 h_{0}, the arrival curve can be modeled as,

α f h 0 ​ (t) = b f h 0 + ρ f h 0 ​ t, \alpha_{f}^{h_{0}}(t)=b_{f}^{h_{0}}+\rho_{f}^{h_{0}}t, (15) where b f h 0 = l f b_{f}^{h_{0}}=l_{f}, and ρ f h 0 = l f / P f \rho_{f}^{h_{0}}=l_{f}/P_{f}. The arrival curve of flow f f at intermediate node h h is the output arrival curve of f f departing from the server h − h^{-},

α f h ​ (t) = α f h − ⊘ δ D f h − ​ (t), \alpha_{f}^{h}(t)=\alpha_{f}^{h^{-}}\oslash\delta_{D_{f}^{h^{-}}}(t), (16)

where D f h − D_{f}^{h^{-}} is the latency upper bound of flow f f queuing at server h − h^{-}, and δ D ​ (t) \delta_{D}(t) is the pure-delay function.

The aggregate arrival curve for ET flows of SR Class M i M_{i} at h h is obtained by summing the arrival curves of individual flows. It also incorporates the link shaping curve and the CBS shaping curve to improve the tightness of the analysis results. α M i h ​ (t) = ∑ h − ∈ ℋ ∑ f ∈ ℱ M i h −, h α f h ​ (t) ∧ σ l ​ i ​ n ​ k h −, h ​ (t) ∧ σ M i h −, h ​ (t), \alpha^{h}_{M_{i}}\!(t)\!=\!\!\sum_{h^{-}\in\mathcal{H}}\sum_{f\in\mathcal{F}_{M_{i}}^{h^{-}\!,h}}\!\!\!\!\!\!\alpha^{h}_{f}(t)\!\wedge\!\sigma_{link}^{h^{-}\!,h}(t)\!\wedge\!\sigma_{M_{i}}^{h^{-}\!,h}(t), (17)

where x ∧ y = min ⁡ { x, y } x\wedge y=\min\{x,y\}, σ l ​ i ​ n ​ k h −, h ​ (t) \sigma_{link}^{h^{-}\!,h}(t) is the link shaping curve from the preceding output h − h^{-} to the current output port h h: σ l ​ i ​ n ​ k h −, h ​ (t) = C ​ t + l M i h −, h, max, \sigma_{link}^{h^{-}\!,h}(t)=Ct+l_{M_{i}}^{h^{-}\!,h,\max}, (18) considering the packetization impact of the maximum frame size l M i h −, h, max l_{M_{i}}^{h^{-}\!,h,\max} of flows with Class M i M_{i} from h − h^{-} to h h. σ M i h −, h ​ (t) \sigma_{M_{i}}^{h^{-}\!,h}(t) is the CBS shaping curve of Class M i M_{i} from h − h^{-} to h h: σ M i h −, h ​ (t) = i ​ d ​ S ​ l M i h − ​ [ t − β T ​ A ​ S h − ​ (t) C + c M i h −, max − c M i h −, min i ​ d ​ S ​ l M i h − ] + l M i h −, h, max, \sigma_{M_{i}}^{h^{-}\!,h}\!(t)\!=\!idSl^{h^{-}}_{M_{i}}\!\left[\!t\!-\!\frac{\beta^{h^{-}}_{TAS}(t)}{C}\!+\!\frac{c_{M_{i}}^{h^{-}\!,\max}\!-\!c_{M_{i}}^{h^{-}\!,\min}}{idSl^{h^{-}}_{M_{i}}}\!\right]\!+\!l_{M_{i}}^{h^{-}\!,h,\max}, (19) β T ​ A ​ S h ​ (t) \beta^{h}_{TAS}(t) represents the minimum service supplied to TT traffic on the output port h h.

With NC-based Total Flow Analysis (TFA), the worst-case delay upper bound D f h D_{f}^{h} for flow f ∈ ℱ M i h f\in\mathcal{F}_{M_{i}}^{h} at h h equals the worst-case delay upper bound D M i h D_{M_{i}}^{h} for all flows with the same priority M i M_{i} aggregating at h h, D f h = D M i h = h ​ D ​ e ​ v ​ (α M i h, β M i h) = sup t ≥ 0 { inf { τ ≥ 0 ∣ α M i h ​ (t) ≤ β M i h ​ (t + τ) } } D_{f}^{h}\!=\!D_{M_{i}}^{h}\!=\!hDev(\alpha_{M_{i}}^{h},\beta_{M_{i}}^{h})=\!\sup_{t\geq 0}\left\{\inf\left\{\tau\!\geq\!0\mid\alpha^{h}_{M_{i}}\!(t)\leq\beta^{h}_{M_{i}}\!(t\!+\!\tau)\right\}\right\}\\ (20)

where α M i h ​ (t) \alpha^{h}_{M_{i}}(t) is the arrival curve of aggregate flows of Class M i M_{i} from Eq. (17), and β M i h ​ (t) \beta^{h}_{M_{i}}(t) is the service curve for Class M i M_{i} from Eq. (10). The upper bound of the worst-case end-to-end delay for the flow f f is then obtained by summing the per-port latency bounds along its route.

Credit-Based Shaper (CBS) is a TSN mechanism designed to prevent starvation of lower-priority traffic while guaranteeing a reserved portion of bandwidth for higher-priority queues, thereby providing reliability through bounded end-to-end delays. Traffic assigned to queues using CBS is typically referred to as Audio Video Bridging (AVB) traffic.

Here, we build on the description from (Bujosa Mateu, 2024). In CBS, each AVB queue is associated with a credit value. This credit increases over time when a frame is waiting to be transmitted or when the credit is negative, and decreases while a frame is being transmitted. Moreover, if the credit is positive and there are no AVB frames waiting to be transmitted, the credit is immediately reset to 0. The rates at which credit is increased and decreased are defined by the parameters idleSlope and sendSlope, respectively. Each queue implementing CBS is configured with its own idleSlope and sendSlope values, which determine its allocated bandwidth share. In particular, the bandwidth reserved for a queue is expressed as Eq. (21). A queue is eligible for transmission only when its credit is zero or positive.

Reserved ​ BW = idleSlope idleSlope + sendSlope ⋅ B ​ W \mathrm{Reserved\;BW}=\frac{\textit{idleSlope}}{\textit{idleSlope}+\textit{sendSlope}}\cdot BW (21)

Consider the example illustrated in Figure 11, which includes two AVB queues and one Best Effort (BE) queue. Frames 1 and 4 are assigned to the higher-priority AVB queue, while frames 2 and 3 belong to the lower-priority AVB queue and the BE queue, respectively.

At time T0, both AVB queues are eligible for transmission. Due to strict priority scheduling, the higher-priority AVB queue (priority 2) is selected, and frame 1 is transmitted. During this transmission, its credit decreases, while the credit of the lower-priority AVB queue increases because it is waiting.

At time T1, the higher-priority AVB queue has accumulated negative credit and is therefore no longer eligible for transmission. As a result, the lower-priority AVB queue is selected, and frame 2 is transmitted, even though a higher-priority frame (frame 4) is waiting. During this time, the lower-priority queue’s credit decreases, while the higher-priority queue’s credit recovers.

By time T2, both AVB queues have negative credit, making them ineligible for transmission. Consequently, the BE queue is selected, and frame 3 is transmitted, despite the presence of a higher-priority AVB frame waiting.

Finally, at time T3, the credit of the higher-priority AVB queue has recovered to zero, making it eligible again. Therefore, frame 4 is transmitted.

Cyclic Queuing and Forwarding (CQF) (Debnath et al., 2025b) is a TSN shaping mechanism which uses a single cycle duration, denoted as T T, across the entire network. T T is the minimum scheduling unit where we put the TSN flows. Furthermore, T T defines the granularity of the end-to-end delay of the flows in the network. The unit of T T is in μ \mu s in TSNBench. In a TSN switch, every egress port in the network has eight queues. TSN flows are stored in the queues depending on its priority. In CQF, for each egress port, two queues are used: an even queue and an odd queue. Figure 14 shows the basic working diagram of CQF with two queues (even and odd). As shown in Figure 14, CQF works by employing two queues, let’s say, Q 8 Q_{8} and Q 7 Q_{7} for TT flows by operating them in a ping-pong manner where Q 7 Q_{7} receives and Q 8 Q_{8} transmits at the first cycle slot (T 1 T_{1}). During the second cycle slot (T 2 T_{2}), Q 8 Q_{8} receives and Q 7 Q_{7} transmits. Selecting or allotting a cycle slot for a flow means selecting the cycle slot number (within the hyperperiod H H) and the queue for the flow.

In the CQF evaluation of TSNBench, we provide the cycle duration (T \mathrm{T}) and network-specific delays to the model as input through the prompt.

WCD CQF: The worst case end-to-end delay of the TT flows in the CQF network is quantified as follows: Max ​ Delay = f i. ϕ + (SW num + 1) ⋅ T + ξ, \mathrm{Max\;Delay}=f_{i}.\phi+(\mathrm{SW_{num}}+1)\cdot\mathrm{T}+\xi, (22) where f i ⋅ ϕ f_{i}\cdot\phi is the offset of the flow f i f_{i} in μ \mu s, SW num \mathrm{SW_{num}} is the total number of switches in the route of the TT flow, T \mathrm{T} is the cycle duration in μ \mu s, and ξ \xi denotes the network specific delays: processing delay, propagation delay, and time synchronization error (sync error \mathrm{sync_{error}}).

To maintain the same standards across all human reviewers, we use the following rules to evaluate the MCQA dataset. There are four possible options for every question in the MCQA dataset.

1. Accept: i. Technically correct. ii. Clearly worded and self-contained. iii. Unambiguous options. iv. Accurate and sufficient explanation. v. The correct answer is actually the correct answer. 2. Reject: i. Incorrect or misleading. ii. Poorly constructed beyond revision. iii. Irrelevant to TSN. iv. Incomplete information. v. Too paper-dependent. vi. Duplicate questions. 3. Revise: i. Minor issues in grammar, clarity, or wording. ii. Options need improvement. iii. Explanation needs refinement. 4. Doubtful: i. Paper-specific or uncertain about the correctness of the question. ii. Explanation seems questionable. iii. Needs further clarification. For a doubtful multiple-choice question, we read the research paper and re-evaluate the question. Afterward, the decision can be accept, reject, or revise; if it is still doubtful, we send it to another expert reviewer for a consensus-based group decision.

Accept: i. Technically correct. ii. Clearly worded and self-contained. iii. Unambiguous options. iv. Accurate and sufficient explanation. v. The correct answer is actually the correct answer.

Technically correct.

Clearly worded and self-contained.

Unambiguous options.

Accurate and sufficient explanation.

The correct answer is actually the correct answer.

Reject: i. Incorrect or misleading. ii. Poorly constructed beyond revision. iii. Irrelevant to TSN. iv. Incomplete information. v. Too paper-dependent. vi. Duplicate questions.

Incorrect or misleading.

Poorly constructed beyond revision.

Irrelevant to TSN.

Incomplete information.

Too paper-dependent.

Duplicate questions.

Revise: i. Minor issues in grammar, clarity, or wording. ii. Options need improvement. iii. Explanation needs refinement.

Minor issues in grammar, clarity, or wording.

Options need improvement.

Explanation needs refinement.

Doubtful: i. Paper-specific or uncertain about the correctness of the question. ii. Explanation seems questionable. iii. Needs further clarification. For a doubtful multiple-choice question, we read the research paper and re-evaluate the question. Afterward, the decision can be accept, reject, or revise; if it is still doubtful, we send it to another expert reviewer for a consensus-based group decision.

Paper-specific or uncertain about the correctness of the question.

Explanation seems questionable.

Needs further clarification.

Key principles followed while reviewing the dataset: We ensured that the MCQAs were technically accurate and aligned with TSN fundamentals. We avoided tricky questions and preferred clarity over complexity. The same set of rules was given to all expert reviewers who worked on this dataset and served as human judges. After the review, 185 questions were revised by the domain experts, as shown below in Table 5.

We present three representative sample questions from our MCQA dataset below.

Q1 TSN Keyword What does TAS stand for in TSN traffic management? A. Transmission Access Scheduler B. Traffic Analysis System C. Time-Aware Shaper D. Traffic Admission Service Correct Answer: C

Q2 Research Paper In a Cyclic Queuing and Forwarding (CQF) network what fundamental limitation would prevent effective fault tolerance using Frame Replication and Elimination for Reliability (FRER) in a linear topology where each switch has maximum transmission unit (MTU) sized frames frequently queued? A. CQF’s ping-pong queue switching would create timing conflicts with FRER’s frame elimination mechanism. B. EMI interference would corrupt both original and replicated frames equally, making spatial redundancy ineffective. C. FRER cannot detect bit errors caused by EMI since it lacks Cyclic Redundancy Check (CRC) verification capabilities. D. Linear topologies cannot provide the disjoint paths required for FRER’s spatial redundancy approach, forcing expensive hardware additions. Correct Answer: D

Q3 Research Paper What fundamental challenge makes the Time Aware Shaper (TAS) implementation complex despite its ability to provide guaranteed end-to-end delays? A. The requirement to synchronize all network devices to a common time reference. B. The need to maintain separate queues for each traffic class simultaneously. C. The difficulty in estimating worst-case transmission times for variable-length frames. D. The synthesis of the gate control list, which is an NP-complete problem. Correct Answer: D

For the CBS and CQF mechanisms, two different approaches are used for WCD calculation. NC is used to calculate the CBS WCD, whereas an analytical mathematical calculation is used to find the WCD for the CQF mechanism. Since these two mechanisms work differently, we design prompts tailored to each mechanism.

Role: We start by defining the role of the model: “You are an expert Time-Sensitive Networking (TSN) orchestrator.” We inject three network inputs: (i) network topology, (ii) TSN flow information, and (iii) the routes of the flows. We use the prompt-as-program (Reynolds and McDonell, 2021) approach to separate the network topology, flow information, and flow routes. All of these are provided in text format. However, to evaluate different topologies, flows, and routes, we separate them from the prompt logic. This ensures that the prompt remains the same across different network topologies and parameters.

Constants: To correctly calculate the WCD, information about the network parameters is required. To prevent the model from assuming these values and to keep the constant values consistent across all models, we provide this information in the prompt.

Constants for CBS open-ended questions: B ​ a ​ n ​ d ​ w ​ i ​ d ​ t ​ h = 100 Bandwidth=100 Mbps, P ​ r ​ o ​ p ​ a ​ g ​ a ​ t ​ i ​ o ​ n ​ d ​ e ​ l ​ a ​ y = 1 ​ μ Propagation\ delay=1\,\mu s, S ​ w ​ i ​ t ​ c ​ h ​ i ​ n ​ g ​ d ​ e ​ l ​ a ​ y = 1 ​ μ Switching\ delay=1\,\mu s, T ​ i ​ m ​ e ​ s ​ y ​ n ​ c ​ h ​ r ​ o ​ n ​ i ​ z ​ a ​ t ​ i ​ o ​ n ​ e ​ r ​ r ​ o ​ r = 1 ​ μ Time\ synchronization\ error=1\,\mu s, The switches of the network are cut-through switches, I ​ d ​ l ​ e ​ S ​ l ​ o ​ p ​ e IdleSlope = 75 % =75\%

By controlling these network parameters, we directly mitigate hallucinations and assumptions about numerical values.

Architecture Restriction: TSN supports multiple architectures that affect the Quality of Service (QoS) and the WCD of the flows. The prompt restricts the model to using only one TSN mechanism through the following directive.

For the CBS mechanism, we use:

TSN Mechanism: Only Credit-Based Shaper (CBS, IEEE 802.1Qav) is allowed; All flows are AVB Class A, PCP = 6, using queue 6 only.

For the CQF mechanism, we use:

TSN Mechanism: Only Cyclic Queuing and Forwarding (CQF, IEEE 802.1Qch) is allowed; All flows are TT, PCP = 7, using queue 7 (odd) and 6 (even) only.

Our reasoning is that letting the model select the TSN architecture or mechanism is a separate benchmarking problem, where the model is evaluated on architecture design performance. In TSNBench, our goal is to benchmark LLMs in TSN. Without an explicit restriction, the model may select an incorrect or inappropriate mechanism, producing a hallucinated architecture that does not satisfy the QoS requirements of the flows. This restriction forces the model to use a single solution space. It further ensures that the WCDs provided by different models are not caused by architectural faults or mechanism selection ambiguity, but rather by calculation and implementation errors within the specified mechanism.

Structured Output: We instruct the model through the prompt to provide the output strictly in JSON format (Yang et al., 2026).

For the open-ended questions, there are three variable entries: network topology, flow information, and flow routing. We use the K-shortest path algorithm to determine the routes of the flows. The routes are then directly provided to the models as input for further evaluation.

Network Topologies Used: For the open-ended questions, we selected three different topologies to evaluate the models: a one-switch topology, a medium-mesh topology, and an industrial ring topology. Figures 15, 16, and 17 represent the one-switch, medium-mesh, and ring topologies used in TSNBench, respectively.

Flow parameters: We show the flow information used in TSNBench as follows.

Flow Information TC1_flows.txt 0,node2_1,node5_2,2500,709,965 1,node5_4,node3_2,2500,610,825 2,node0_4,node0_1,1000,786,887 3,node2_3,node4_3,2500,1088,1233 4,node0_4,node3_3,1000,1015,488 5,node0_4,node0_1,2500,926,501...

Ground Truth WCD Values The ground-truth WCD values of the flows for all open-ended test cases for the CBS mechanism are calculated using a verified NC tool (Zhao et al., 2018; Debnath et al., 2025c; Gavriluţ and Pop, 2020). For the WCD of the CQF mechanism, we use the mathematical equation given in Eq. 22.

We evaluate both open-source and closed-source state-of-the-art LLMs on TSNBench. A detailed list of the models, along with their model numbers and snapshots, is given in Table 6. This ensures that the results are reproducible by the community.

We evaluate the models under two different configurations: (i) default temperature settings (0.7) and (ii) temperature set to 0.0, for both MCQA and open-ended questions. As in safety-critical networks, we want to ensure deterministic results. Therefore, we evaluate whether LLMs can provide consistent results when the temperature is set to 0.0. For models that do not support the temperature parameter, we use the default temperature for evaluation.

Table 7 provides the accuracy and average consistency of the models for the MCQA dataset under the default temperature and temperature set to 0.0. Average consistency represents the ability of the model to provide the same results across three runs.

The cost and latency of a model are important evaluation parameters for the research community. Spending a large amount of money on benchmark evaluation is a real bottleneck for research groups. Moreover, not all models can be evaluated locally. Table 8 presents the cost and latency of the TSNBench MCQA and open-ended questions. Evaluating MCQA is relatively much cheaper than evaluating open-ended questions.

We provide MAE and MAPE evaluations for the open-ended questions. A sample calculation is given as follows:

MAE and MAPE calculation example: Consider a model evaluated on three test cases (TCs). These three TCs may have different topologies, different flows and flow parameters, and different routes. For each TC, we have the ground-truth and predicted WCD values shown in Table 9. The ground truth is calculated using an NC solver for CBS and a mathematical equation for CQF.

Per-TC MAE: Suppose TC1, TC2, and TC3 contain three, two, and three flows, respectively. { f 1, f 2, f 3 } ∈ T ​ C ​ 1; \displaystyle\{f_{1},f_{2},f_{3}\}\in TC1; { f 1, f 2 } ∈ T ​ C ​ 2; \displaystyle\{f_{1},f_{2}\}\in TC2; { f 1, f 2, f 3 } ∈ T ​ C ​ 3; \displaystyle\{f_{1},f_{2},f_{3}\}\in TC3; Let Γ ​ (f 0) \Gamma(f_{0}) denote the absolute error of flow f 0 f_{0} in TC1, β ​ (f 0) \beta(f_{0}) denote the predicted WCD of flow f 0 f_{0} given by the LLM model, and Ω ​ (f 0) \Omega(f_{0}) denote the ground truth of flow f 0 f_{0}. We calculate Γ ​ (f 0) \Gamma(f_{0}) as follows: Γ ​ (f 0) = | β ​ (f 0) − Ω ​ (f 0) | \displaystyle\Gamma(f_{0})=|\beta(f_{0})-\Omega(f_{0})| In the given example, let Γ ​ (f 0) \Gamma(f_{0}) = 12, Γ ​ (f 1) \Gamma(f_{1}) = 30, and Γ ​ (f 2) \Gamma(f_{2}) = 10 for TC1. Similarly, for TC2, Γ ​ (f 0) \Gamma(f_{0}) = 8 and Γ ​ (f 1) \Gamma(f_{1}) = 45 and for TC3, Γ ​ (f 0) \Gamma(f_{0}) = 20, Γ ​ (f 1) \Gamma(f_{1}) = 15, and Γ ​ (f 2) \Gamma(f_{2}) = 0. We calculate the MAE for TC1, TC2, and TC3 represented as MAE TC1 \text{MAE}_{\text{TC1}}, MAE TC2 \text{MAE}_{\text{TC2}}, and MAE TC3 \text{MAE}_{\text{TC3}} as follows: MAE TC1 \displaystyle\text{MAE}_{\text{TC1}} = (12 + 30 + 10) / 3 = 17.3 ​ μ ​ s \displaystyle=(12+30+10)/3=17.3~\mu\text{s} MAE TC2 \displaystyle\text{MAE}_{\text{TC2}} = (8 + 45) / 2 = 26.5 ​ μ ​ s \displaystyle=(8+45)/2=26.5~\mu\text{s} MAE TC3 \displaystyle\text{MAE}_{\text{TC3}} = (20 + 15 + 0) / 3 = 11.7 ​ μ ​ s \displaystyle=(20+15+0)/3=11.7~\mu\text{s}

For every model, we have 100 test cases, and the final MAE is averaged across all test cases (in this example 3 test cases) and is represented as: MAE = (17.3 + 26.5 + 11.7) / 3 = 18.5 ​ μ ​ s \text{MAE}=(17.3+26.5+11.7)/3=18.5~\mu\text{s} The per-flow MAPE denoted as α ​ (f 0) \alpha(f_{0}) is calculated as follows: α ​ (f 0) = | β ​ (f 0) − Ω ​ (f 0) | Ω ​ (f 0) × 100 \displaystyle\alpha(f_{0})=\frac{|\beta(f_{0})-\Omega(f_{0})|}{\Omega(f_{0})}\times 100 For TC1, we calculate the MAPE as follows: MAPE TC1 \displaystyle\text{MAPE}_{\text{TC1}} = α ​ (f 0) + α ​ (f 1) + α ​ (f 2) 3 = 8.7 % \displaystyle=\frac{\alpha(f_{0})+\alpha(f_{1})+\alpha(f_{2})}{3}=8.7\% Similarly, the MAPE for TC2 and TC3 is given as follows: MAPE TC2 \displaystyle\text{MAPE}_{\text{TC2}} = 11.5 % \displaystyle=11.5\% MAPE TC3 \displaystyle\text{MAPE}_{\text{TC3}} = 3.7 % \displaystyle=3.7\% The final MAPE for each model is averaged across the 3 test cases: MAPE = (8.7 + 11.5 + 3.7) / 3 = 8.0 % \text{MAPE}=(8.7+11.5+3.7)/3=8.0\% In TSNBench, all test cases contributes equally towards the model performance irrespective of the number of flows in the network. As per the network architecture, all flows are equally critical and needs the same preference. This ensures that for each network scenario all the flows are weighted equally.

Test Case: TC1 TSN mechanism: CBS You are an expert Time-Sensitive Networking (TSN) orchestrator. Your task is to calculate the worst case delay (WCD) for each TSN flow. Input: Network Topology (T ​ C ​ 1 ​ _ ​ t ​ o ​ p ​ o. t ​ x ​ t TC1\_topo.txt) Flow Information (T ​ C ​ 1 ​ _ ​ f ​ l ​ o ​ w ​ s. t ​ x ​ t TC1\_flows.txt) Routing of the Flow (T ​ C ​ 1 ​ _ ​ r ​ o ​ u ​ t ​ e. t ​ x ​ t TC1\_route.txt) Constant: Bandwidth link = 100 Mbps; Propagation delay = 1 μ \mu s; Switching delay = 1 μ \mu s; Time synchronization error = 1 μ \mu s; The switches of the network are cut-through switches. TSN Mechanism: Only Credit-Based Shaper (CBS, IEEE 802.1Qav) is allowed; All flows are AVB Class A, PCP = 6, using queue 6 only. Task: 1. Map each egress port’s queues and collect the set of flows traversing from that port, using the given topology, flows, and route of the flow. 2. For each egress port, use the given IdleSlope and then compute the SendSlope. 3. For each flow, construct an arrival curve from its frame size and periodicity. 4. For each port, derive a lower-bounded CBS service curve. 5. Calculate the worst case delay (WCD) in microseconds (μ \mu s) for each flow using Network Calculus method. 6. Provide the confidence score between 0.0 and 1.0 from your answers. 1.0 means mathematically or procedurally provable from given info with zero ambiguity. 0.0 means zero confidence.

Map each egress port’s queues and collect the set of flows traversing from that port, using the given topology, flows, and route of the flow.

For each egress port, use the given IdleSlope and then compute the SendSlope.

For each flow, construct an arrival curve from its frame size and periodicity.

For each port, derive a lower-bounded CBS service curve.

Calculate the worst case delay (WCD) in microseconds (μ \mu s) for each flow using Network Calculus method.

Provide the confidence score between 0.0 and 1.0 from your answers. 1.0 means mathematically or procedurally provable from given info with zero ambiguity. 0.0 means zero confidence.

Grok 4.1 Fast (Non-Reasoning) output: F0: 1452.0, F1: 1124.0, F2: 678.0, F3: 1234.0, F4: 1567.0, F5: 689.0, F6: 456.0, F7: 789.0, F8: 567.0, F9: 589.0, F10: 678.0, F11: 645.0, F12: 1123.0, F13: 789.0, F14: 456.0, F15: 678.0, F16: 789.0, F17: 1567.0, F18: 456.0, F19: 1234.0 Model Reasoning and Explanation Output: Parameters used: send ​ _ ​ slope \mathrm{send\_slope}: 75 % × 100 ​ Mbps = 75 ​ Mbps = 75000 ​ bits / μ ​ s 75\%\times 100~\mathrm{Mbps}=75~\mathrm{Mbps}=75000~\mathrm{bits}/\mu\mathrm{s} frame ​ _ ​ overhead \mathrm{frame\_overhead}: 24 bytes (Ethernet header + FCS), mtu \mathrm{mtu}: 1522 bytes (max frame size), queue \mathrm{queue}: 6 (PCP=6, AVB Class A), cbs ​ _ ​ credit ​ _ ​ max \mathrm{cbs\_credit\_max}: MTU × 8 = 12176 \mathrm{MTU}\times 8=12176 bits WCD equation given by the model: α ​ (t) = L max Period + t Period (arrival ​ curve) \alpha(t)=\frac{L_{\max}}{\mathrm{Period}}+\frac{t}{\mathrm{Period}}\quad\mathrm{(arrival\ curve)} β ​ (t) = R send × t − C (CBS ​ service ​ curve) \beta(t)=R_{\mathrm{send}}\times t-C\quad\mathrm{(CBS\ service\ curve)} W ​ C ​ D = h × (prop ​ _ ​ delay + switch ​ _ ​ delay + sync ​ _ ​ error) + max ⁡ (α ∩ β) WCD=h\times\left(\mathrm{prop\_delay}+\mathrm{switch\_delay}+\mathrm{sync\_error}\right)+\max(\alpha\cap\beta) SendSlope = IdleSlope × Bandwidth = 0.75 × 100 ​ Mbps \mathrm{SendSlope}=\mathrm{IdleSlope}\times\mathrm{Bandwidth}=0.75\times 100~\mathrm{Mbps} R send = SendSlope = 75 ​ Mbps R_{\mathrm{send}}=\mathrm{SendSlope}=75~\mathrm{Mbps} C = credit max = MTU × 8 ​ bits C=\mathrm{credit}_{\max}=\mathrm{MTU}\times 8~\mathrm{bits} h = hop ​ count ​ from ​ path h=\mathrm{hop\ count\ from\ path} Per-flow WCD: W ​ C ​ D F i = ∑ h ∈ path ​ (F i) (d prop h + d switch h + d sync h) + max t ≥ 0 ⁡ (α F i ​ (t) ∩ β port ​ (t)) WCD_{F_{i}}=\sum_{h\in\mathrm{path}(F_{i})}\left(d_{\mathrm{prop}}^{h}+d_{\mathrm{switch}}^{h}+d_{\mathrm{sync}}^{h}\right)+\max_{t\geq 0}\left(\alpha_{F_{i}}(t)\cap\beta_{\mathrm{port}}(t)\right)

Grok 4.1 Fast (Non-Reasoning) output:

F0: 1452.0, F1: 1124.0, F2: 678.0, F3: 1234.0, F4: 1567.0, F5: 689.0, F6: 456.0, F7: 789.0, F8: 567.0, F9: 589.0, F10: 678.0, F11: 645.0, F12: 1123.0, F13: 789.0, F14: 456.0, F15: 678.0, F16: 789.0, F17: 1567.0, F18: 456.0, F19: 1234.0

Run 2 of the model (Grok 4.1 Fast (Non-Reasoning)): WCD equation given by the model during run 2: Frame size: L i \displaystyle\text{Frame size:}\quad L_{i} = payload ​ _ ​ i + 24 ​ bytes, \displaystyle=\mathrm{payload}\_i+4~\mathrm{bytes}, Transmission time: T x i \displaystyle\text{Transmission time:}\quad T_{x_{i}} = 8 ​ L i R, \displaystyle=\frac{8L_{i}}{R}, Arrival curve: α i ​ (t) \displaystyle\text{Arrival curve:}\quad\alpha_{i}(t) = L i p i ⋅ t, \displaystyle=\frac{L_{i}}{p_{i}}\cdot t, CBS service curve: β ​ (t) \displaystyle\text{CBS service curve:}\quad\beta(t) = min ⁡ { α h i ​ (t), sendSlope ⋅ t + credit max }, \displaystyle=\min\left\{\alpha_{h_{i}}(t),\;\mathrm{sendSlope}\cdot t+\mathrm{credit}_{\max}\right\}, Worst-case delay: W ​ C ​ D i \displaystyle\text{Worst-case delay:}\quad WCD_{i} = max h ∈ path ​ ∑ (T prop, h + T switch, h + T queue, h) + T sync, \displaystyle=\max_{h\in\mathrm{path}}\sum\left(T_{\mathrm{prop},h}+T_{\mathrm{switch},h}+T_{\mathrm{queue},h}\right)+T_{\mathrm{sync}}, Queueing delay: T queue, h \displaystyle\text{Queueing delay:}\quad T_{\mathrm{queue},h} = L i α ​ C + burst ​ _ ​ interference, \displaystyle=\frac{L_{i}}{\alpha C}+\mathrm{burst}\_{\mathrm{interference}}, Send slope: sendSlope \displaystyle\text{Send slope:}\quad\mathrm{sendSlope} = idleSlope ⋅ R − idleSlope ⋅ R = α ​ R ​ (1 − 2 ​ α); \displaystyle=\mathrm{idleSlope}\cdot R-\mathrm{idleSlope}\cdot R=\alpha R(1-2\alpha); Per-flow WCD: W ​ C ​ D i \displaystyle WCD_{i} = hop ​ _ ​ count i ⋅ (1 + 1) + max ​ _ ​ queue ​ _ ​ delay + 1; \displaystyle=\mathrm{hop\_count}_{i}\cdot(1+1)+\mathrm{max\_queue\_delay}+1; Expert Explanation: The model uses different equations across different runs even with temperature set to 0.0. When analyzing the first run, the model makes several fundamental errors in evaluating the interference/blocking experienced by the queue under analysis, i.e., both the arrival curve and CBS service curve. First, it assumes the wrong maximum burst size and does not consider the variant output arrival curve at subsequent hops along the path. Second, it assumes a non-zero maximum credit, which is possible in this scenario due to the non-preemption frame of lower priority of non-CBS traffic. However, the model applies the wrong maximum credit to the CBS service curve, incorrectly understanding the relation of the corresponding rate (75%). Furthermore, in the final WCD calculation, the model considers only the arrival curve of the frame under analysis. This is incorrect: the proper approach requires using the aggregate arrival curve of all frames transmitted through the queue. It is also not clear whether the m ​ a ​ x ​ (α ∩ β) max(\alpha\cap\beta) operation represents the maximum horizontal deviation between α \alpha and β \beta. The output arrival curve is not considered in the following hops’ calculation. On the other hand, the second run exhibits even more severe hallucinations, producing calculations of the wrong arrival curve without considering any burst size, and the wrong CBS service curve considering the wrong service rate (sendSlope) and wrong relation between the corresponding rate and the maximum credit, as well as without considering the output arrival curve either etc. GPT-5 output: F0: 714.65, F1: 821.79, F2: 171.48, F3: 687.17, F4: 907.26, F5: 171.48, F6: 183.9, F7: 453.91, F8: 611.08, F9: 181.3, F10: 611.08, F11: 321.35, F12: 1124.12, F13: 483.67, F14: 172.43, F15: 489.26, F16: 521.33, F17: 918.73, F18: 181.3, F19: 624.12

Run 2 of the model (Grok 4.1 Fast (Non-Reasoning)):

WCD equation given by the model during run 2: Frame size: L i \displaystyle\text{Frame size:}\quad L_{i} = payload ​ _ ​ i + 24 ​ bytes, \displaystyle=\mathrm{payload}\_i+4~\mathrm{bytes}, Transmission time: T x i \displaystyle\text{Transmission time:}\quad T_{x_{i}} = 8 ​ L i R, \displaystyle=\frac{8L_{i}}{R}, Arrival curve: α i ​ (t) \displaystyle\text{Arrival curve:}\quad\alpha_{i}(t) = L i p i ⋅ t, \displaystyle=\frac{L_{i}}{p_{i}}\cdot t, CBS service curve: β ​ (t) \displaystyle\text{CBS service curve:}\quad\beta(t) = min ⁡ { α h i ​ (t), sendSlope ⋅ t + credit max }, \displaystyle=\min\left\{\alpha_{h_{i}}(t),\;\mathrm{sendSlope}\cdot t+\mathrm{credit}_{\max}\right\}, Worst-case delay: W ​ C ​ D i \displaystyle\text{Worst-case delay:}\quad WCD_{i} = max h ∈ path ​ ∑ (T prop, h + T switch, h + T queue, h) + T sync, \displaystyle=\max_{h\in\mathrm{path}}\sum\left(T_{\mathrm{prop},h}+T_{\mathrm{switch},h}+T_{\mathrm{queue},h}\right)+T_{\mathrm{sync}}, Queueing delay: T queue, h \displaystyle\text{Queueing delay:}\quad T_{\mathrm{queue},h} = L i α ​ C + burst ​ _ ​ interference, \displaystyle=\frac{L_{i}}{\alpha C}+\mathrm{burst}\_{\mathrm{interference}}, Send slope: sendSlope \displaystyle\text{Send slope:}\quad\mathrm{sendSlope} = idleSlope ⋅ R − idleSlope ⋅ R = α ​ R ​ (1 − 2 ​ α); \displaystyle=\mathrm{idleSlope}\cdot R-\mathrm{idleSlope}\cdot R=\alpha R(1-2\alpha); Per-flow WCD: W ​ C ​ D i \displaystyle WCD_{i} = hop ​ _ ​ count i ⋅ (1 + 1) + max ​ _ ​ queue ​ _ ​ delay + 1; \displaystyle=\mathrm{hop\_count}_{i}\cdot(1+1)+\mathrm{max\_queue\_delay}+1;

Expert Explanation:

The model uses different equations across different runs even with temperature set to 0.0. When analyzing the first run, the model makes several fundamental errors in evaluating the interference/blocking experienced by the queue under analysis, i.e., both the arrival curve and CBS service curve. First, it assumes the wrong maximum burst size and does not consider the variant output arrival curve at subsequent hops along the path. Second, it assumes a non-zero maximum credit, which is possible in this scenario due to the non-preemption frame of lower priority of non-CBS traffic. However, the model applies the wrong maximum credit to the CBS service curve, incorrectly understanding the relation of the corresponding rate (75%). Furthermore, in the final WCD calculation, the model considers only the arrival curve of the frame under analysis. This is incorrect: the proper approach requires using the aggregate arrival curve of all frames transmitted through the queue. It is also not clear whether the m ​ a ​ x ​ (α ∩ β) max(\alpha\cap\beta) operation represents the maximum horizontal deviation between α \alpha and β \beta. The output arrival curve is not considered in the following hops’ calculation. On the other hand, the second run exhibits even more severe hallucinations, producing calculations of the wrong arrival curve without considering any burst size, and the wrong CBS service curve considering the wrong service rate (sendSlope) and wrong relation between the corresponding rate and the maximum credit, as well as without considering the output arrival curve either etc.

GPT-5 output:

F0: 714.65, F1: 821.79, F2: 171.48, F3: 687.17, F4: 907.26, F5: 171.48, F6: 183.9, F7: 453.91, F8: 611.08, F9: 181.3, F10: 611.08, F11: 321.35, F12: 1124.12, F13: 483.67, F14: 172.43, F15: 489.26, F16: 521.33, F17: 918.73, F18: 181.3, F19: 624.12

Model Reasoning and Explanation Output: sendSlope \displaystyle\mathrm{sendSlope} = − 25 ​ Mbps, \displaystyle=-5~\mathrm{Mbps}, Arrival curve: b \displaystyle\text{Arrival curve:}\quad b = ∑ ∀ F i | share ​ output ​ port L i ​ bits, \displaystyle=\sum_{\mathrm{\forall F_{i}|{\ share\ output\ port}}}L_{i}~\mathrm{bits}, r \displaystyle r = ∑ ∀ F i | share ​ output ​ port L i p i ​ bits / μ ​ s, \displaystyle=\sum_{\mathrm{\forall F_{i}|{\ share\ output\ port}}}\frac{L_{i}}{p_{i}}~\mathrm{bits}/\mu\mathrm{s}, α ​ (t) \displaystyle\alpha(t) = b + r ​ t, \displaystyle=b+rt, T tech, total \displaystyle T_{\mathrm{tech,total}} = T prop + T switching + T sync \displaystyle=T_{\mathrm{prop}}+T_{\mathrm{switching}}+T_{\mathrm{sync}} = h ​ o ​ p ​ s × 1 ​ μ ​ s + (h ​ o ​ p ​ s − 1) × 1 ​ μ ​ s + (h ​ o ​ p ​ s − 1) × 1 ​ μ ​ s = 13 ​ μ ​ s, \displaystyle=hops\times 1~\mu\mathrm{s}+(hops-1)\times 1~\mu\mathrm{s}+(hops-1)\times 1~\mu\mathrm{s}=3~\mu\mathrm{s}, W ​ C ​ D total \displaystyle WCD_{\mathrm{total}} = W ​ C ​ D sum + T tech, total \displaystyle=WCD_{\mathrm{sum}}+T_{\mathrm{tech,total}} = 701.65 ​ μ ​ s + 13 ​ μ ​ s = 714.65 ​ μ ​ s. \displaystyle=0165~\mu\mathrm{s}+3~\mu\mathrm{s}=1465~\mu\mathrm{s}. Expert Explanation: GPT uses the correct arrival and service curves. Its main error is that it fails to include the output port (output arrival curves) in the analysis. Ministral 3 8B output: F0: 1250.0, F1: 1100.0, F2: 500.0, F3: 1500.0, F4: 600.0, F5: 1100.0, F6: 2500.0, F7: 1200.0, F8: 500.0, F9: 500.0, F10: 2000.0, F11: 500.0, F12: 1000.0, F13: 1000.0, F14: 2000.0, F15: 500.0, F16: 1500.0, F17: 1000.0, F18: 500.0, F19: 2000.0 Model Reasoning and Explanation Output: Bandwidth: R \displaystyle\text{Bandwidth:}\quad R = 100 ​ Mbps = 12.5 ​ MB / s, \displaystyle=00~\mathrm{Mbps}=25~\mathrm{MB/s}, Propagation delay: T prop \displaystyle\text{Propagation delay:}\quad T_{\mathrm{prop}} = 1 ​ μ ​ s per hop, \displaystyle=1~\mu\mathrm{s}\quad\text{per hop}, Switching delay: T switch \displaystyle\text{Switching delay:}\quad T_{\mathrm{switch}} = 1 ​ μ ​ s per switch, \displaystyle=1~\mu\mathrm{s}\quad\text{per switch}, Time synchronization error: T sync \displaystyle\text{Time synchronization error:}\quad T_{\mathrm{sync}} = 1 ​ μ ​ s, \displaystyle=1~\mu\mathrm{s}, IdleSlope: IdleSlope \displaystyle\text{IdleSlope:}\quad\mathrm{IdleSlope} = 0.75 ​ R = 75 ​ Mbps = 9.375 ​ MB / s, \displaystyle=75R=5~\mathrm{Mbps}=375~\mathrm{MB/s}, Cut-through switch: T buffer \displaystyle\text{Cut-through switch:}\quad T_{\mathrm{buffer}} = 0, \displaystyle=0, Payload overhead: L overhead \displaystyle\text{Payload overhead:}\quad L_{\mathrm{overhead}} = 18 ​ B + 4 ​ B = 22 ​ B, \displaystyle=8~\mathrm{B}+4~\mathrm{B}=2~\mathrm{B}, Maximum frame size: L max \displaystyle\text{Maximum frame size:}\quad L_{\max} = 1433 ​ B + 22 ​ B = 1455 ​ B, \displaystyle=433~\mathrm{B}+2~\mathrm{B}=455~\mathrm{B}, Maximum transmission time: T tx, max \displaystyle\text{Maximum transmission time:}\quad T_{\mathrm{tx,max}} = 1455 × 8 100 ​ Mbps = 9.24 ​ μ ​ s. \displaystyle=\frac{1455\times 8}{100~\mathrm{Mbps}}=24~\mu\mathrm{s}.

Model Reasoning and Explanation Output: sendSlope \displaystyle\mathrm{sendSlope} = − 25 ​ Mbps, \displaystyle=-5~\mathrm{Mbps}, Arrival curve: b \displaystyle\text{Arrival curve:}\quad b = ∑ ∀ F i | share ​ output ​ port L i ​ bits, \displaystyle=\sum_{\mathrm{\forall F_{i}|{\ share\ output\ port}}}L_{i}~\mathrm{bits}, r \displaystyle r = ∑ ∀ F i | share ​ output ​ port L i p i ​ bits / μ ​ s, \displaystyle=\sum_{\mathrm{\forall F_{i}|{\ share\ output\ port}}}\frac{L_{i}}{p_{i}}~\mathrm{bits}/\mu\mathrm{s}, α ​ (t) \displaystyle\alpha(t) = b + r ​ t, \displaystyle=b+rt, T tech, total \displaystyle T_{\mathrm{tech,total}} = T prop + T switching + T sync \displaystyle=T_{\mathrm{prop}}+T_{\mathrm{switching}}+T_{\mathrm{sync}} = h ​ o ​ p ​ s × 1 ​ μ ​ s + (h ​ o ​ p ​ s − 1) × 1 ​ μ ​ s + (h ​ o ​ p ​ s − 1) × 1 ​ μ ​ s = 13 ​ μ ​ s, \displaystyle=hops\times 1~\mu\mathrm{s}+(hops-1)\times 1~\mu\mathrm{s}+(hops-1)\times 1~\mu\mathrm{s}=3~\mu\mathrm{s}, W ​ C ​ D total \displaystyle WCD_{\mathrm{total}} = W ​ C ​ D sum + T tech, total \displaystyle=WCD_{\mathrm{sum}}+T_{\mathrm{tech,total}} = 701.65 ​ μ ​ s + 13 ​ μ ​ s = 714.65 ​ μ ​ s. \displaystyle=0165~\mu\mathrm{s}+3~\mu\mathrm{s}=1465~\mu\mathrm{s}.

Expert Explanation:

GPT uses the correct arrival and service curves. Its main error is that it fails to include the output port (output arrival curves) in the analysis.

Ministral 3 8B output:

F0: 1250.0, F1: 1100.0, F2: 500.0, F3: 1500.0, F4: 600.0, F5: 1100.0, F6: 2500.0, F7: 1200.0, F8: 500.0, F9: 500.0, F10: 2000.0, F11: 500.0, F12: 1000.0, F13: 1000.0, F14: 2000.0, F15: 500.0, F16: 1500.0, F17: 1000.0, F18: 500.0, F19: 2000.0

SendSlope \displaystyle\mathrm{SendSlope} = IdleSlope ​ (1 − IdleSlope) \displaystyle=\mathrm{IdleSlope}\,(1-\mathrm{IdleSlope}) = 9.375 ​ MB / s × 0.25 \displaystyle=375~\mathrm{MB/s}\times 25 = 2.34375 ​ MB / s, \displaystyle=34375~\mathrm{MB/s}, Burst size: B burst, i \displaystyle\text{Burst size:}\quad B_{\mathrm{burst},i} = SendSlope ⋅ p i, \displaystyle=\mathrm{SendSlope}\cdot p_{i}, CBS service curve: S ​ (t) \displaystyle\text{CBS service curve:}\quad S(t) = SendSlope ​ (t − B burst SendSlope), t ≥ B burst SendSlope. \displaystyle=\mathrm{SendSlope}\left(t-\frac{B_{\mathrm{burst}}}{\mathrm{SendSlope}}\right),\quad t\geq\frac{B_{\mathrm{burst}}}{\mathrm{SendSlope}}. WCD Calculation: Arrival curve for flow F x: R (t) \displaystyle\text{Arrival curve for flow }F_{x}:\quad R(t) = min ⁡ { B x, S x ​ t }, \displaystyle=\min\left\{B_{x},\;S_{x}t\right\}, B x \displaystyle B_{x} = L payload, x + L overhead, \displaystyle=L_{\mathrm{payload},x}+L_{\mathrm{overhead}}, S x \displaystyle S_{x} = 8 ​ L payload, x p x, \displaystyle=\frac{8L_{\mathrm{payload},x}}{p_{x}}, Service curve for each port: S ​ (t) \displaystyle\text{Service curve for each port:}\quad S(t) = SendSlope ​ (t − B burst SendSlope), t ≥ B burst SendSlope, \displaystyle=\mathrm{SendSlope}\left(t-\frac{B_{\mathrm{burst}}}{\mathrm{SendSlope}}\right),\quad t\geq\frac{B_{\mathrm{burst}}}{\mathrm{SendSlope}}, W ​ C ​ D \displaystyle WCD = max ⁡ (0, B burst − SendSlope ⋅ P ​ e ​ r ​ i ​ o ​ d SendSlope) \displaystyle=\max\left(0,\;\frac{B_{\mathrm{burst}}-\mathrm{SendSlope}\cdot Period}{\mathrm{SendSlope}}\right) + T prop + T switch + T sync. \displaystyle\quad+T_{\mathrm{prop}}+T_{\mathrm{switch}}+T_{\mathrm{sync}}. Expert Explanation: Even though it is a small model, this agent delivers a poor result. Similar to what was observed with Grok 4.1 Fast (Non-Reasoning), it fails to account for the arrival curves of flows and the wrong CBS service curve. In addition, it analyzes only a single queue, instead of computing the delay at the output port of each device along the end-to-end path. Moreover, it introduces unit inconsistencies such as dividing the frame size expressed in bytes by a link speed given in bits/ μ \mu s, which leads to incorrect results.

SendSlope \displaystyle\mathrm{SendSlope} = IdleSlope ​ (1 − IdleSlope) \displaystyle=\mathrm{IdleSlope}\,(1-\mathrm{IdleSlope}) = 9.375 ​ MB / s × 0.25 \displaystyle=375~\mathrm{MB/s}\times 25 = 2.34375 ​ MB / s, \displaystyle=34375~\mathrm{MB/s}, Burst size: B burst, i \displaystyle\text{Burst size:}\quad B_{\mathrm{burst},i} = SendSlope ⋅ p i, \displaystyle=\mathrm{SendSlope}\cdot p_{i}, CBS service curve: S ​ (t) \displaystyle\text{CBS service curve:}\quad S(t) = SendSlope ​ (t − B burst SendSlope), t ≥ B burst SendSlope. \displaystyle=\mathrm{SendSlope}\left(t-\frac{B_{\mathrm{burst}}}{\mathrm{SendSlope}}\right),\quad t\geq\frac{B_{\mathrm{burst}}}{\mathrm{SendSlope}}. WCD Calculation: Arrival curve for flow F x: R (t) \displaystyle\text{Arrival curve for flow }F_{x}:\quad R(t) = min ⁡ { B x, S x ​ t }, \displaystyle=\min\left\{B_{x},\;S_{x}t\right\}, B x \displaystyle B_{x} = L payload, x + L overhead, \displaystyle=L_{\mathrm{payload},x}+L_{\mathrm{overhead}}, S x \displaystyle S_{x} = 8 ​ L payload, x p x, \displaystyle=\frac{8L_{\mathrm{payload},x}}{p_{x}}, Service curve for each port: S ​ (t) \displaystyle\text{Service curve for each port:}\quad S(t) = SendSlope ​ (t − B burst SendSlope), t ≥ B burst SendSlope, \displaystyle=\mathrm{SendSlope}\left(t-\frac{B_{\mathrm{burst}}}{\mathrm{SendSlope}}\right),\quad t\geq\frac{B_{\mathrm{burst}}}{\mathrm{SendSlope}}, W ​ C ​ D \displaystyle WCD = max ⁡ (0, B burst − SendSlope ⋅ P ​ e ​ r ​ i ​ o ​ d SendSlope) \displaystyle=\max\left(0,\;\frac{B_{\mathrm{burst}}-\mathrm{SendSlope}\cdot Period}{\mathrm{SendSlope}}\right) + T prop + T switch + T sync. \displaystyle\quad+T_{\mathrm{prop}}+T_{\mathrm{switch}}+T_{\mathrm{sync}}.

Expert Explanation:

Even though it is a small model, this agent delivers a poor result. Similar to what was observed with Grok 4.1 Fast (Non-Reasoning), it fails to account for the arrival curves of flows and the wrong CBS service curve. In addition, it analyzes only a single queue, instead of computing the delay at the output port of each device along the end-to-end path. Moreover, it introduces unit inconsistencies such as dividing the frame size expressed in bytes by a link speed given in bits/ μ \mu s, which leads to incorrect results.

Test Case: TC1 TSN mechanism: CQF You are an expert Time-Sensitive Networking (TSN) orchestrator. Your task is to calculate the worst case delay (WCD) for each TSN flow. Input: Network Topology (T ​ C ​ 1 ​ _ ​ t ​ o ​ p ​ o. t ​ x ​ t TC1\_topo.txt) Flow Information (T ​ C ​ 1 ​ _ ​ f ​ l ​ o ​ w ​ s. t ​ x ​ t TC1\_flows.txt) Routing of the Flow (T ​ C ​ 1 ​ _ ​ r ​ o ​ u ​ t ​ e. t ​ x ​ t TC1\_route.txt) Constant: Bandwidth link = 100 Mbps; Propagation delay = 1 μ \mu s; Switching delay = 1 μ \mu s; Time synchronization error = 1 μ \mu s; The switches of the network are cut-through switches. TSN Mechanism: Only Cyclic Queuing and Forwarding (CQF, IEEE 802.1Qch) is allowed; All flows are TT, PCP = 7, using queue 7 (odd) and 6 (even) only. Task: 1. Map each egress port’s queues and collect the set of flows traversing that port, using the given topology, flows, and route of the flow. 2. For the entire network, use the given cycle duration and compute the Hypercycle. 3. For each flow, set the offset or the start time of the flow from the sending node as 0. 4. Calculate the worst case delay (WCD) in microseconds (μ \mu s) for each flow. 5. Provide the confidence score between 0.0 and 1.0 from your answers. 1.0 means mathematically or procedurally provable from given info with zero ambiguity. 0.0 means zero confidence.

Map each egress port’s queues and collect the set of flows traversing that port, using the given topology, flows, and route of the flow.

For the entire network, use the given cycle duration and compute the Hypercycle.

For each flow, set the offset or the start time of the flow from the sending node as 0.

Calculate the worst case delay (WCD) in microseconds (μ \mu s) for each flow.

Provide the confidence score between 0.0 and 1.0 from your answers. 1.0 means mathematically or procedurally provable from given info with zero ambiguity. 0.0 means zero confidence.

Claude Sonnet’s output: F0: 257.72, F1: 206.8, F2: 105.096, F3: 218.704, F4: 253.904, F5: 104.008, F6: 103.808, F7: 153.52, F8: 119.408, F9: 119.752, F10: 122.896, F11: 118.464, F12: 222.04, F13: 156.552, F14: 103.808, F15: 156.576, F16: 155.68, F17: 256.616, F18: 101.696, F19: 208.328 Model Reasoning and Explanation Output: Cycle duration: = 50 ​ μ ​ s, \displaystyle\text{Cycle duration:}=0~\mu\mathrm{s}, LCM of all periods and cycle: LCM ​ (1000, 2500, 5000, 50) \displaystyle\text{LCM of all periods and cycle:}\quad\mathrm{LCM}(000,500,000,0) = 5000 ​ μ ​ s, \displaystyle=000~\mu\mathrm{s}, Hypercycle: = 5000 ​ μ ​ s. \displaystyle\text{Hypercycle:}=000~\mu\mathrm{s}. WCD equation: W ​ C ​ D \displaystyle\text{WCD equation:}\quad WCD = ∑ hops (T trans + T prop + T sw + T sync) + T buffering. \displaystyle=\sum_{\mathrm{hops}}\left(T_{\mathrm{trans}}+T_{\mathrm{prop}}+T_{\mathrm{sw}}+T_{\mathrm{sync}}\right)+T_{\mathrm{buffering}}. T trans \displaystyle T_{\mathrm{trans}} = (Payload + Ethernet ​ _ ​ overhead) ⋅ 8 R, \displaystyle=\frac{\left(\mathrm{Payload}+\mathrm{Ethernet\_overhead}\right)\cdot 8}{R}, T prop \displaystyle T_{\mathrm{prop}} = 1 ​ μ ​ s per link, \displaystyle=1~\mu\mathrm{s}\quad\text{per link}, T sw \displaystyle T_{\mathrm{sw}} = 1 ​ μ ​ s per switch, \displaystyle=1~\mu\mathrm{s}\quad\text{per switch}, T sync \displaystyle T_{\mathrm{sync}} = 1 ​ μ ​ s per hop, \displaystyle=1~\mu\mathrm{s}\quad\text{per hop}, T buffering \displaystyle T_{\mathrm{buffering}} = (hop ​ _ ​ count − 1) ⋅ T cycle \displaystyle=\left(\mathrm{hop\_count}-1\right)\cdot T_{\mathrm{cycle}} In CQF, frames may wait up to one cycle duration at each intermediate switch if they arrive just after the transmission window closes. For cut-through with CQF, worst case adds (N − 1) ∗ cycle ​ _ ​ duration (N-1)*\mathrm{cycle\_duration} where N N is hop count. Hop count = 5, \displaystyle=5, Payload_bytes = 965 ​ bytes, \displaystyle=65~\mathrm{bytes}, Frame_size_bytes = 1003 ​ bytes, \displaystyle=003~\mathrm{bytes}, Transmission_delay = 80.24 ​ μ ​ s, \displaystyle=024~\mu\mathrm{s}, Total propagation delay = 5 ​ μ ​ s, \displaystyle=5~\mu\mathrm{s}, Total switching delay = 4 ​ μ ​ s, \displaystyle=4~\mu\mathrm{s}, Total synchronization error = 5 ​ μ ​ s, \displaystyle=5~\mu\mathrm{s}, CQF buffering = 200 ​ μ ​ s, \displaystyle=00~\mu\mathrm{s}, WCD = 257.72 ​ μ ​ s. \displaystyle=5772~\mu\mathrm{s}. Ground truth WCD:: Route hop count of ​ f 1 \displaystyle\text{Route hop count of }f_{1}: 4, T = 50 ​ μ ​ s, \displaystyle:4,T=0~\mu\mathrm{s}, W ​ C ​ D f 1 \displaystyle WCD_{f_{1}} = f 1 ⋅ ϕ + (S ​ W num + 1) ​ T + ξ \displaystyle=f_{1}\cdot\phi+(SW_{\mathrm{num}}+1)T+\xi = (3 + 1) × 50 + 5 \displaystyle=(3+1)\times 0+5 = 205 ​ μ ​ s. \displaystyle=05~\mu\mathrm{s}. Expert Explanation: The model knows the working mechanism of CQF. However, the model calculated the wrong Hypercycle by including cycle duration along with the periods. Furthermore, the CQF WCD explanation given by the model is wrong.

Claude Sonnet’s output:

F0: 257.72, F1: 206.8, F2: 105.096, F3: 218.704, F4: 253.904, F5: 104.008, F6: 103.808, F7: 153.52, F8: 119.408, F9: 119.752, F10: 122.896, F11: 118.464, F12: 222.04, F13: 156.552, F14: 103.808, F15: 156.576, F16: 155.68, F17: 256.616, F18: 101.696, F19: 208.328

Model Reasoning and Explanation Output: Cycle duration: = 50 ​ μ ​ s, \displaystyle\text{Cycle duration:}=0~\mu\mathrm{s}, LCM of all periods and cycle: LCM ​ (1000, 2500, 5000, 50) \displaystyle\text{LCM of all periods and cycle:}\quad\mathrm{LCM}(000,500,000,0) = 5000 ​ μ ​ s, \displaystyle=000~\mu\mathrm{s}, Hypercycle: = 5000 ​ μ ​ s. \displaystyle\text{Hypercycle:}=000~\mu\mathrm{s}. WCD equation: W ​ C ​ D \displaystyle\text{WCD equation:}\quad WCD = ∑ hops (T trans + T prop + T sw + T sync) + T buffering. \displaystyle=\sum_{\mathrm{hops}}\left(T_{\mathrm{trans}}+T_{\mathrm{prop}}+T_{\mathrm{sw}}+T_{\mathrm{sync}}\right)+T_{\mathrm{buffering}}. T trans \displaystyle T_{\mathrm{trans}} = (Payload + Ethernet ​ _ ​ overhead) ⋅ 8 R, \displaystyle=\frac{\left(\mathrm{Payload}+\mathrm{Ethernet\_overhead}\right)\cdot 8}{R}, T prop \displaystyle T_{\mathrm{prop}} = 1 ​ μ ​ s per link, \displaystyle=1~\mu\mathrm{s}\quad\text{per link}, T sw \displaystyle T_{\mathrm{sw}} = 1 ​ μ ​ s per switch, \displaystyle=1~\mu\mathrm{s}\quad\text{per switch}, T sync \displaystyle T_{\mathrm{sync}} = 1 ​ μ ​ s per hop, \displaystyle=1~\mu\mathrm{s}\quad\text{per hop}, T buffering \displaystyle T_{\mathrm{buffering}} = (hop ​ _ ​ count − 1) ⋅ T cycle \displaystyle=\left(\mathrm{hop\_count}-1\right)\cdot T_{\mathrm{cycle}} In CQF, frames may wait up to one cycle duration at each intermediate switch if they arrive just after the transmission window closes. For cut-through with CQF, worst case adds (N − 1) ∗ cycle ​ _ ​ duration (N-1)*\mathrm{cycle\_duration} where N N is hop count. Hop count = 5, \displaystyle=5, Payload_bytes = 965 ​ bytes, \displaystyle=65~\mathrm{bytes}, Frame_size_bytes = 1003 ​ bytes, \displaystyle=003~\mathrm{bytes}, Transmission_delay = 80.24 ​ μ ​ s, \displaystyle=024~\mu\mathrm{s}, Total propagation delay = 5 ​ μ ​ s, \displaystyle=5~\mu\mathrm{s}, Total switching delay = 4 ​ μ ​ s, \displaystyle=4~\mu\mathrm{s}, Total synchronization error = 5 ​ μ ​ s, \displaystyle=5~\mu\mathrm{s}, CQF buffering = 200 ​ μ ​ s, \displaystyle=00~\mu\mathrm{s}, WCD = 257.72 ​ μ ​ s. \displaystyle=5772~\mu\mathrm{s}.

Ground truth WCD::

Route hop count of ​ f 1 \displaystyle\text{Route hop count of }f_{1}: 4, T = 50 ​ μ ​ s, \displaystyle:4,T=0~\mu\mathrm{s}, W ​ C ​ D f 1 \displaystyle WCD_{f_{1}} = f 1 ⋅ ϕ + (S ​ W num + 1) ​ T + ξ \displaystyle=f_{1}\cdot\phi+(SW_{\mathrm{num}}+1)T+\xi = (3 + 1) × 50 + 5 \displaystyle=(3+1)\times 0+5 = 205 ​ μ ​ s. \displaystyle=05~\mu\mathrm{s}.

Expert Explanation:

The model knows the working mechanism of CQF. However, the model calculated the wrong Hypercycle by including cycle duration along with the periods. Furthermore, the CQF WCD explanation given by the model is wrong.

GPT-4o output: F0: 260.0, F1: 210.0, F2: 60.0, F3: 210.0, F4: 260.0, F5: 60.0, F6: 60.0, F7: 110.0, F8: 60.0, F9: 60.0, F10: 60.0, F11: 60.0, F12: 210.0, F13: 110.0, F14: 60.0, F15: 110.0, F16: 110.0, F17: 260.0, F18: 60.0, F19: 210.0 Model Reasoning and Explanation Output: Hypercycle = 100 ​ μ ​ s. \displaystyle\text{Hypercycle}=00~\mu\mathrm{s}. WCD = (hop ​ _ ​ count × (cycle ​ _ ​ duration + T switch + T prop + T sync)) + Payload Bandwidth \mathrm{WCD}=\left(\mathrm{hop\_count}\times\left(\mathrm{cycle\_duration}+T_{\mathrm{switch}}+T_{\mathrm{prop}}+T_{\mathrm{sync}}\right)\right)+\frac{\mathrm{Payload}}{\mathrm{Bandwidth}} Each flow’s path, cycle duration, hypercycle, and CQF parameters are detailed, along with the equation used for WCD calculation. No missing inputs are identified, but assumptions are made based on typical network configurations. Expert Explanation: The Hypercycle calculated by the model is wrong, moreover the WCD equation used by the model has an extra wrong component Payload Bandwidth \frac{\mathrm{Payload}}{\mathrm{Bandwidth}}. One of the most interesting finding is all the models provide very different WCD equations across different runs and different test cases. Ministral 3 8B output: F0: 1160.0, F1: 1020.0, F2: 100.0, F3: 1160.0, F4: 1260.0, F5: 100.0, F6: 100.0, F7: 100.0, F8: 100.0, F9: 100.0, F10: 100.0, F11: 100.0, F12: 1160.0, F13: 100.0, F14: 100.0, F15: 100.0, F16: 100.0, F17: 1260.0, F18: 100.0, F19: 1160.0 Model Reasoning and Explanation Output: Hypercycle: H \displaystyle\text{Hypercycle:}\quad H = 10000 ​ μ ​ s, \displaystyle=0000~\mu\mathrm{s}, Max frame size = 1526 ​ bytes = 1500 + 26 ​ bytes, \displaystyle=526~\mathrm{bytes}=500+6~\mathrm{bytes}, Max transmission time = 12.208 ​ μ ​ s, \displaystyle=2208~\mu\mathrm{s}, Max queuing delay = 48 ​ μ ​ s, \displaystyle=8~\mu\mathrm{s}, W ​ C ​ D \displaystyle WCD = h ​ (T prop + T switch + T sync) \displaystyle=h\left(T_{\mathrm{prop}}+T_{\mathrm{switch}}+T_{\mathrm{sync}}\right) + T tx, max ​ ⌈ payload max ​ _ ​ frame ​ _ ​ size ⌉ \displaystyle\quad+T_{\mathrm{tx,max}}\left\lceil\frac{\mathrm{payload}}{\mathrm{max\_frame\_size}}\right\rceil + T queue, max ​ (h − 1), \displaystyle\quad+T_{\mathrm{queue,max}}(h-1),

GPT-4o output:

F0: 260.0, F1: 210.0, F2: 60.0, F3: 210.0, F4: 260.0, F5: 60.0, F6: 60.0, F7: 110.0, F8: 60.0, F9: 60.0, F10: 60.0, F11: 60.0, F12: 210.0, F13: 110.0, F14: 60.0, F15: 110.0, F16: 110.0, F17: 260.0, F18: 60.0, F19: 210.0

Model Reasoning and Explanation Output: Hypercycle = 100 ​ μ ​ s. \displaystyle\text{Hypercycle}=00~\mu\mathrm{s}. WCD = (hop ​ _ ​ count × (cycle ​ _ ​ duration + T switch + T prop + T sync)) + Payload Bandwidth \mathrm{WCD}=\left(\mathrm{hop\_count}\times\left(\mathrm{cycle\_duration}+T_{\mathrm{switch}}+T_{\mathrm{prop}}+T_{\mathrm{sync}}\right)\right)+\frac{\mathrm{Payload}}{\mathrm{Bandwidth}} Each flow’s path, cycle duration, hypercycle, and CQF parameters are detailed, along with the equation used for WCD calculation. No missing inputs are identified, but assumptions are made based on typical network configurations.

Expert Explanation:

The Hypercycle calculated by the model is wrong, moreover the WCD equation used by the model has an extra wrong component Payload Bandwidth \frac{\mathrm{Payload}}{\mathrm{Bandwidth}}. One of the most interesting finding is all the models provide very different WCD equations across different runs and different test cases.

Ministral 3 8B output:

F0: 1160.0, F1: 1020.0, F2: 100.0, F3: 1160.0, F4: 1260.0, F5: 100.0, F6: 100.0, F7: 100.0, F8: 100.0, F9: 100.0, F10: 100.0, F11: 100.0, F12: 1160.0, F13: 100.0, F14: 100.0, F15: 100.0, F16: 100.0, F17: 1260.0, F18: 100.0, F19: 1160.0 Model Reasoning and Explanation Output: Hypercycle: H \displaystyle\text{Hypercycle:}\quad H = 10000 ​ μ ​ s, \displaystyle=0000~\mu\mathrm{s}, Max frame size = 1526 ​ bytes = 1500 + 26 ​ bytes, \displaystyle=526~\mathrm{bytes}=500+6~\mathrm{bytes}, Max transmission time = 12.208 ​ μ ​ s, \displaystyle=2208~\mu\mathrm{s}, Max queuing delay = 48 ​ μ ​ s, \displaystyle=8~\mu\mathrm{s}, W ​ C ​ D \displaystyle WCD = h ​ (T prop + T switch + T sync) \displaystyle=h\left(T_{\mathrm{prop}}+T_{\mathrm{switch}}+T_{\mathrm{sync}}\right) + T tx, max ​ ⌈ payload max ​ _ ​ frame ​ _ ​ size ⌉ \displaystyle\quad+T_{\mathrm{tx,max}}\left\lceil\frac{\mathrm{payload}}{\mathrm{max\_frame\_size}}\right\rceil + T queue, max ​ (h − 1), \displaystyle\quad+T_{\mathrm{queue,max}}(h-1),

Hop count: h \displaystyle\text{Hop count:}\quad h = 5, \displaystyle=5, Propagation + switching + sync = 5 × (1 + 1 + 1) = 15 ​ μ ​ s, \displaystyle=5\times(1+1+1)=5~\mu\mathrm{s}, Transmission time = 12.208 × ⌈ 2500 1526 ⌉ = 12.208 × 2 = 24.416 ​ μ ​ s, \displaystyle=2208\times\left\lceil\frac{2500}{1526}\right\rceil=2208\times 2=4416~\mu\mathrm{s}, Queueing delay = 48 × (5 − 1) = 192 ​ μ ​ s, \displaystyle=8\times(5-1)=92~\mu\mathrm{s}, Total = \displaystyle\text{Total}= 15 + 24.416 + 192 = 231.416 ​ μ ​ s, \displaystyle 5+4416+92=31416~\mu\mathrm{s}, Adjusted worst-case = 1160 ​ μ ​ s. \displaystyle=160~\mu\mathrm{s}. Expert Explanation: The WCD equation provided by the model is wrong. Even though the model takes into consideration the number of hops present in the route, the delays accumulated across each hop and also calculates the hop count. However, the model misses the most crucial part of the WCD equation which is the cycle duration. Furthermore, the two components of the WCD equation (T tx, max ​ ⌈ payload max ​ _ ​ frame ​ _ ​ size ⌉ T_{\mathrm{tx,max}}\left\lceil\frac{\mathrm{payload}}{\mathrm{max\_frame\_size}}\right\rceil) and (T queue, max ​ (h − 1) T_{\mathrm{queue,max}}(h-1)) considered by the model is entirely hallucinated. These two components are mainly contributing to the large WCD values of this model.

Hop count: h \displaystyle\text{Hop count:}\quad h = 5, \displaystyle=5, Propagation + switching + sync = 5 × (1 + 1 + 1) = 15 ​ μ ​ s, \displaystyle=5\times(1+1+1)=5~\mu\mathrm{s}, Transmission time = 12.208 × ⌈ 2500 1526 ⌉ = 12.208 × 2 = 24.416 ​ μ ​ s, \displaystyle=2208\times\left\lceil\frac{2500}{1526}\right\rceil=2208\times 2=4416~\mu\mathrm{s}, Queueing delay = 48 × (5 − 1) = 192 ​ μ ​ s, \displaystyle=8\times(5-1)=92~\mu\mathrm{s}, Total = \displaystyle\text{Total}= 15 + 24.416 + 192 = 231.416 ​ μ ​ s, \displaystyle 5+4416+92=31416~\mu\mathrm{s}, Adjusted worst-case = 1160 ​ μ ​ s. \displaystyle=160~\mu\mathrm{s}.

Expert Explanation:

The WCD equation provided by the model is wrong. Even though the model takes into consideration the number of hops present in the route, the delays accumulated across each hop and also calculates the hop count. However, the model misses the most crucial part of the WCD equation which is the cycle duration. Furthermore, the two components of the WCD equation (T tx, max ​ ⌈ payload max ​ _ ​ frame ​ _ ​ size ⌉ T_{\mathrm{tx,max}}\left\lceil\frac{\mathrm{payload}}{\mathrm{max\_frame\_size}}\right\rceil) and (T queue, max ​ (h − 1) T_{\mathrm{queue,max}}(h-1)) considered by the model is entirely hallucinated. These two components are mainly contributing to the large WCD values of this model.

To understand the nature of WCD computation failures, we identify five distinct failure modes observed across models and mechanisms.

The model returns WCD = 0 for all flows, producing a structurally valid JSON response but with no computational content. This failure mode affects GPT-4o and DeepSeek-V3.2 (Non-thinking) on CBS, and Llama 3.2 1B across all test cases for CBS and CQF. This suggests these models recognize the output format requirement but cannot engage with the underlying NC computation or any reasoning behind the WCD calculation.

The model produces valid WCD values for fewer than 80% of flows in a given TC, resulting in incomplete coverage. This affects Mistral Large 3 on CBS and Llama 3.3 on CBS, suggesting these models lose track of flow indexing in large topologies.

The model cannot process the full open-ended prompt due to context window limitations or API timeout. This affects Qwen3 8B (API timeout across all TCs) and Llama 3.2 1B (context limit exceeded), confirming that small models are structurally unsuited for TSN open-end evaluation.

The model returns an empty response for all open-ended test cases, regardless of network topology or flow count. This failure mode exclusively affects DeepSeek-V3.2 (Thinking), which produces no output, neither WCD values nor intermediate reasoning, across all evaluated topologies, including one-switch, medium-mesh, and ring configurations, and across all flows, for both CBS and CQF mechanisms.