bead.serving

Structural capabilities, factory registration, and conformance checks for externally trained, read-only predictors.

Capability protocols

protocols

Structural capabilities for read-only model serving.

The protocols in this module describe independent serving capabilities. A consumer can request only the operations it uses, while a provider can implement the protocols structurally without inheriting from a bead base class. In particular, probabilistic predictors do not need to implement training, mixed effects, language-model scoring, or embeddings.

NLIProbabilities = Mapping[str, float]

Three-way natural language inference probabilities.

The conformance suite requires exactly three keys: "entailment", "neutral", and "contradiction". Values must be finite probabilities that sum to one.

ModelIdentity

Bases: Protocol

Identity metadata published by a serving model.

Attributes:

Name Type Description
model_name str

Stable name of the model family or artifact.
model_version str

Version of the model or artifact.

model_name: str property

Return the stable model name.

Returns:

Type Description
str

Stable model name.

model_version: str property

Return the model or artifact version.

Returns:

Type Description
str

Model or artifact version.

TaskCompatibility

Bases: Protocol

Task types and item structures accepted by a predictor.

Attributes:

Name Type Description
supported_task_types Collection[TaskType]

Collection of task types accepted by the predictor.

supported_task_types: Collection[TaskType] property

Return the task types accepted by the predictor.

Returns:

Type Description
Collection[TaskType]

Accepted task types.

validate_item_compatibility(item: Item, item_template: ItemTemplate) -> None

Validate that an item can be scored under its template.

Parameters:

Name Type Description Default
item Item

Item to validate.

 required
item_template ItemTemplate

Template that defines the item's task and structure.

 required

Raises:

Type Description
ValueError

If the item cannot be scored by the predictor.

ProbabilisticPredictor

Bases: Protocol

Read-only probability prediction with declared column semantics.

Column i of the returned matrix corresponds to response_encoding.labels[i]. Implementations return one row per item, with finite probabilities in [0, 1] that sum to one.

response_encoding: ResponseEncoding property

Return the response encoding that names probability columns.

Returns:

Type Description
ResponseEncoding

Ordered labels and scale type for the probability matrix.

predict_proba(items: list[Item], participant_ids: list[str] | None = None) -> np.ndarray

Return a probability matrix with shape (items, levels).

Parameters:

Name Type Description Default
items list[Item]

Items to score.

 required
participant_ids list[str] | None

Optional participant identifier for each item.

 None

Returns:

Type Description
ndarray

Probability matrix with one row per item and one column per response-encoding label.

PersistablePredictor

Bases: Protocol

Native-artifact persistence for a serving model.

save(path: str | Path) -> None

Save the predictor's native artifact.

Parameters:

Name Type Description Default
path str | Path

File or directory path for the provider's native artifact.

 required

load(path: str | Path) -> Self classmethod

Load a predictor from a native artifact.

Parameters:

Name Type Description Default
path str | Path

File or directory path containing the provider's native artifact.

 required

Returns:

Type Description
Self

Restored predictor.

NLICapability

Bases: Protocol

Three-way natural language inference scoring.

compute_nli(premise: str, hypothesis: str) -> NLIProbabilities

Return entailment, neutral, and contradiction probabilities.

Parameters:

Name Type Description Default
premise str

Premise text.

 required
hypothesis str

Hypothesis text.

 required

Returns:

Type Description
NLIProbabilities

Mapping with exactly the keys "entailment", "neutral", and "contradiction".

ServingPredictor

Bases: ModelIdentity, TaskCompatibility, ProbabilisticPredictor, PersistablePredictor, NLICapability, Protocol

Composition of all read-only serving capabilities.

This convenience protocol is used by the full conformance suite. Consumers that need fewer operations should type against the independent protocols above.

Predictor registry

registry

Factory registry for structurally typed serving predictors.

PredictorFactory = Callable[[str | Path], T]

A callable that loads a predictor from its native artifact.

PredictorRegistry

Map provider names to native-artifact loading factories.

The registry is generic in the capability required by its consumer. It accepts any callable with the factory signature and performs no inheritance or issubclass check.

Examples:

>>> from bead.serving import PredictorRegistry, ServingPredictor
>>> registry = PredictorRegistry[ServingPredictor]()
>>> registry.register("external", ExternalPredictor.load)
>>> predictor = registry.load("external", "model.bin")

register(name: str, factory: PredictorFactory[T]) -> None

Register a loading factory under name.

Parameters:

Name Type Description Default
name str

Provider name used by :meth:load.

 required
factory PredictorFactory[T]

Callable that accepts an artifact path and returns a predictor.

 required

Raises:

Type Description
ValueError

If name is empty.

load(name: str, path: str | Path) -> T

Load a predictor through the factory registered under name.

Parameters:

Name Type Description Default
name str

Registered provider name.

 required
path str | Path

Native artifact path passed to the provider factory.

 required

Returns:

Type Description
T

Predictor returned by the registered factory.

Raises:

Type Description
KeyError

If no factory is registered under name.

list_factories() -> tuple[str, ...]

Return registered factory names in sorted order.

Conformance suite

conformance

Reusable conformance checks for third-party serving predictors.

ConformanceError

Bases: AssertionError

A serving predictor violated its public capability contract.

PredictionCase dataclass

A real bead item paired with the template that defines its task.

NLICase dataclass

A premise-hypothesis pair used to exercise NLI scoring.

ConformanceReport dataclass

Summary returned after every conformance check passes.

run_predictor_conformance(predictor: _ServingCapability, *, prediction_cases: Sequence[PredictionCase], response_encoding: ResponseEncoding, nli_cases: Sequence[NLICase]) -> ConformanceReport

Run the published serving-predictor conformance suite.

The suite verifies structural capability composition, identity, task and item compatibility, ResponseEncoding column semantics, probability shape and normalization, optional participant identifiers, native artifact save/load equality, and three-way NLI probabilities.

Parameters:

Name Type Description Default
predictor ServingPredictor

Structural implementation of all serving capabilities. It does not need to inherit from a bead class.

 required
prediction_cases Sequence[PredictionCase]

Non-empty sequence of real :class:Item and :class:ItemTemplate pairs accepted by the predictor.

 required
response_encoding ResponseEncoding

Encoding whose ordered labels name the columns returned by predict_proba.

 required
nli_cases Sequence[NLICase]

Non-empty sequence of premise-hypothesis pairs.

 required

Returns:

Type Description
ConformanceReport

Summary of the checks that passed.

Raises:

Type Description
ConformanceError

If any capability violates the serving contract.

Examples:

>>> report = run_predictor_conformance(
...     predictor,
...     prediction_cases=[PredictionCase(item, item_template)],
...     response_encoding=encoding,
...     nli_cases=[NLICase("A dog runs.", "An animal moves.")],
... )
>>> report.checks
('identity', 'task_compatibility', 'predict_proba', 'persistence', 'nli')