Validation & claims
cufemlab labels every analysis with exactly what its result means and how far you can trust it. This page defines the validation tiers, lists which of the 15 live analysis types fall in each, and states precisely what "internally validated, external passport not yet attached" is - and is not.
The newest type, cufem3d_gpu_field_analysis, is experimental: GPU-accelerated 3-D field analysis is available on compatible GPU workers, and runtime metadata reports the actual execution backend and fallback state. Every result carries an analytic cross-check against the magnetized-sphere reference, and an optional CPU/GPU numerical parity check runs when run_reference_check is enabled (off by default); the type has no external validation passport, and it is not an independently validated 3-D torque prediction.
The six tiers
A result carries one of six tier labels. The tier tells you what the number was checked against and what an engineer is entitled to conclude from it.
Externally cross-checked
The result ships an independent oracle card: the same case was solved by a separate, external reference and the two agree within a stated tolerance. This is the strongest tier we offer.
Internally validated
The result is verified against the method's own analytic solution or a conservation identity (method of manufactured solutions, convergence order, energy balance). It is not yet accompanied by an external passport.
Experimental surrogate
A learned model that predicts a field. It is an in-distribution surrogate and self-validating demo, scored each run against the real solver on the same input. It is not a general-purpose neural operator.
Experimental
A new execution path: every result carries an analytic cross-check against the magnetized-sphere reference, and a CPU/GPU numerical parity check runs when the optional run_reference_check parameter is enabled (off by default), but carrying no external validation passport. Results report validation_card.status = not_validated. Only cufem3d_gpu_field_analysis. It is the GPU execution lane for the same 3-D edge-element field solve as the CPU utility, and it is not an independently validated 3-D torque prediction. It is fail-closed: with no compatible GPU the job fails rather than silently running on CPU.
Engineering estimate
A reduced-order or coefficient-based calculation intended for design-space ranking and early sizing, not for sign-off. Useful and repeatable, but not a validated field solution.
Utility
A convenience or reporting action - comparisons, signed report generation, quick demo checks. These produce artifacts and summaries, not a physics verdict to be validated.
Where the 15 live types fall
Fifteen public analysis types: two externally cross-checked, five internally validated, one experimental (the GPU 3-D lane), one experimental surrogate, two engineering estimates and four utilities. Two plus five plus one plus one plus two plus four is fifteen. The scope column below marks each type's tier.
- cogging_torque_movingband - flagship; cross-checked against FEMM and GetDP
- cufem3d_field_validation - 3-D field validated against an analytic sphere
- thermal_conduction_steady - MMS L2 order ~1.93; energy-balance relative 3.7e-14 vs 1e-6 tol
- thermal_conduction_transient - theta-method; Crank-Nicolson order 2 confirmed
- eddy_thermal_coupled - skin-effect Joule loss to thermal chain; spatial order 1.998
- core_loss_estimate - Bertotti 3-term separation; k_c closed form to 0.05%
- coreloss_thermal_coupled - core-loss density to thermal response; energy chain
- field_to_field_neural_operator - in-distribution FNO surrogate / self-validating demo
- cufem3d_gpu_field_analysis - GPU 3-D field lane. Analytic magnetized-sphere reference every run, plus an optional CPU/GPU parity check (run_reference_check, off by default), and no external validation passport: results report validation_card.status = not_validated. It is not an independently validated 3-D torque prediction, and it is fail-closed.
- pmsm_operating_torque
- iron_loss_estimate
- material_comparison
- signed_report_generation
- demo_motor_quick_check
- cogging_sweep_2d (legacy)
What "internally validated, external passport not yet attached" means
All five new physics types return result.v1 with validation_card.status = "not_validated" and claim_allowed_public = false. That wording is precise and deliberate. Read it exactly:
What it does mean
- The solver was checked against its own governing physics: a manufactured analytic solution, a measured convergence order, or a conservation/energy identity, and it passed within tolerance.
- The method is self-consistent and reproducible - deterministic runs, HMAC-signed evidence.
- The numbers on this page (orders ~1.93, 1.998, 2; energy balance 3.7e-14) come from those internal checks.
What it does NOT mean
- It is not accompanied by an external oracle card - no independent outside tool or third-party reference is bundled with the result yet.
- It is not "certified" or "third-party validated". Those words never apply to a not_validated card.
- claim_allowed_public = false means the platform will not let the result be published as an externally validated claim.
Internal card vs external oracle card
The difference is the source of truth the result is checked against:
- Truth source = the method's own analytic solution or conservation identity.
- Answers: "Does the solver solve the equation it says it solves, at the expected order?"
- Bundled today with all five new physics types.
- Truth source = an independent outside tool (FEMM, GetDP) or a closed-form analytic reference.
- Answers: "Does an unrelated reference agree with this result on the same case?"
- Bundled today only with cogging_torque_movingband and cufem3d_field_validation.
An external passport for the new physics types is a planned follow-on. Until it is attached, the honest statement is: "validated internally against its analytic / energy identity; external validation passport not yet attached."
The experimental surrogate, stated plainly
field_to_field_neural_operator is a Fourier Neural Operator (FNO2d, 4 spectral layers). It is an in-distribution surrogate and self-validating demo. Each run draws an in-distribution conductivity field from an integer conductivity_seed, predicts the temperature field, and scores itself against the real solver on that same field (self_validation_rel_l2).
- Held-out rel-L2 is 0.68% on the training distribution. Out of distribution it degrades to 5.8%.
- It beats a linear baseline (9.8%) and a POD-DeepONet (11.8%) on that distribution.
- It is not a general-purpose neural operator. Arbitrary custom-field or CAD input is a planned follow-on, not shipped.
Data-rights note: TEAM-7 and H(curl)
Internal validation of our H(curl) edge-element and eddy-current solver line exists, including a TEAM-7 style benchmark run inside the engine. However, third-party TEAM-7 mesh and reference data is not published or bundled with cufemlab. We do not ship, redistribute, or quote outside reference datasets we do not have the rights to distribute. The internal validation is ours; the external benchmark data behind it stays with its owners.
Reading a result card in the API
Every job result is result.v1. The validation_card is where the tier and status live. Fetch it and read it before you rely on a number.
import os
from cufemlab_client import Client
client = Client(api_key=os.environ["CUFEMLAB_API_KEY"]) # default base https://app.cufemlab.secrotec.nl
project = client.create_project(name="thermal-check")
job = client.analyze(
project_id=project.id,
analysis_type="thermal_conduction_steady",
input_params={}, # sensible defaults for every parameter
gpu=True,
)
job.wait(poll_interval=10)
result = job.result()
print(result.verdict) # pass / fail
print(result.value) # result_summary.value
# validation_card.status == "not_validated" for the five new physics types:
# internally validated, external passport not yet attached.
Summary
- Two types are externally cross-checked (Tier 1) and ship an independent oracle card.
- Five new physics types are internally validated (Tier 2) - correct by their own analytic and energy identities, external passport not yet attached (validation_card = not_validated).
- One type is an experimental in-distribution surrogate (Tier 3), not a general-purpose neural operator. One is the experimental GPU 3-D field lane (cufem3d_gpu_field_analysis, Tier 4) — cross-checked against an analytic sphere every run, with an optional CPU/GPU parity check (run_reference_check, off by default), and no external passport.
- Two types are engineering estimates (Tier 5); four are utilities (Tier 6).
- TEAM-7 / H(curl) internal validation exists; third-party benchmark data is not published or bundled.
- cufemlab makes no accuracy or speed claim against any commercial tool.