AvroPrism

dev.constructive.eo.avro.AvroPrism
See theAvroPrism companion object
final class AvroPrism[A] extends Optic[Array[Byte], Array[Byte], A, A, Affine], Dynamic

Optic from the Avro BINARY WIRE FORM to a native type A — the wire bytes are the default carrier:

 AvroPrism[A] <: Optic[Array[Byte], Array[Byte], A, A, Affine]
 type X = (Array[Byte], (Array[Byte], BinarySpan))

Reads locate the focused field's byte span via AvroBinaryCursor and decode only that slice; writes re-encode the focus and splice it in place (three arraycopys, union branch index re-synthesised). The ROOT object is never materialised — neither as a case class nor as a generic record; a record-SHAPED focus is materialised only as that branch's own org.apache.avro.generic.IndexedRecord during the slice decode / re-encode. Mirrors dev.constructive.eo.jsoniter.JsoniterPrism shape-for-shape:

  • Fst[X] = Array[Byte] — original payload; Miss carries it for pass-through (parse failure, path miss, union-branch mismatch, decode failure, unsupported index step).
  • Snd[X] = (Array[Byte], BinarySpan) — payload + located span, so from can splice.

The whole capability-gated extension surface lights up on the bytes — .getOption, .modify, .replace, .foldMap, .andThen, … Drill with the same macro sugar as ever (.field(_.x) / .fields(...) / .union[B] / .at(i) / .each / Dynamic field selection).

'''Two usage modes — pick deliberately:'''

  • '''Layer on an existing codec''': keep decoding to your case class elsewhere; point a prism at the one or two hot-path fields.
  • '''Optics-as-evidence''': the wire Array[Byte] IS the data structure. codecPrism[S] uses AvroCodec[S] as SCHEMA evidence only — the root decode never runs on the byte path; only the drilled leaf's codec decodes its slice. Do NOT look for an avro4s-style whole-record mapping API here: no S is ever materialised. Consuming signatures demand the weakest capability (def validateId[T](idCarrier: T)(using CanGetOption[T, Id]): Boolean), and a drilled-prism given is the evidence that instantiates their T at Array[Byte]. See the docs page's migration recipe for the runnable shape.

'''Field navigation honours the SCHEMA field name (issues #35, #95).''' .field(_.x) — and equally .fields(...), selectDynamic and the .each.field / .each.fields traversal siblings, which all share one resolver — maps the case-class field x to whatever schema field the codec actually emitted for it. Resolution happens ONCE, at prism construction, off the cached schema (zero per-operation cost), by two rungs:

  1. '''By NAME, all-or-nothing.''' If EVERY case field of the parent maps to a DISTINCT schema field — exactly, or uniquely up to _ / - / . and case — then the codec has named the whole correspondence and x's answer is read off that map. Partial or colliding coverage is treated as no signal at all and the rung abstains for every field, because one lucky name match on a schema whose OTHER columns are legacy is how a working call site gets re-aimed at the wrong column.
  2. '''By DECLARATION POSITION''' — the i-th case field is the i-th schema field. This is where a name transform lands (a kindlings snake-case config, a custom transformFieldNames, a vulcan per-field override map), because a transform REMOVES the literal Scala name by construction, so rung 1 cannot have fired.

'''The positional rung is only right when the codec's schema is positionally 1:1 with the case class.''' Kindlings-derived codecs are, by construction. A hand-written or vulcan.Codec field list need not be: a COMPUTED/derived schema column, a dropped field, or a reordered field list all break it. The name rung recovers most of that population; what it cannot recover is a field list that both renames beyond recognition AND reorders (equal arity, no name hit) — that resolves by position, silently, and is wrong. Two more shapes stay wrong for the same reason: a schema column that BEARS a case field's name but HOLDS a different value (a derived public id, a stale legacy column), and two columns whose names normalise alike. Navigate all of them with AvroPrism.fieldNamed("schema_name"), which bypasses resolution entirely and is itself checked against the schema; ResolutionResidualSpec pins each shape's exact behaviour.

Behaviour change against 0.15.1, for the release notes: a hand-written codec that PERMUTES the Scala names (writes case field a into a schema field literally named b, and vice versa) resolved correctly by position and now resolves by name, i.e. wrongly. No name transform can produce that shape. Map keys are data, not schema-named fields, and keep their literal key.

Two sibling surfaces, one mechanism each (deliberately NOT duplicated here):

  • record — the IndexedRecord-carried optic (AvroRecordPrism) with the Ior-bearing diagnostic surface (get / modify / place / transfer + *Unsafe) over IndexedRecord | Array[Byte] | String input. Use it when you hold parsed records or need accumulated AvroFailure diagnostics.
  • sliceBytes / graftBytes — the encoded-fragment surface for hashing / shipping / splicing a field's raw encoding across payloads without decoding the focus at all.

Storage decomposition: an AvroPrism[A] holds an AvroFocus (Leaf vs Fields) and a cached root schema; the byte walk uses the focus's path, the slice decode uses its codec.

'''Laws & preconditions''' (normative):

  • The Optional laws hold '''up to canonical re-encoding of the focused slice''': modify(identity) re-encodes the focus, so a payload whose focused slice used non-canonical (but spec-legal) encodings — non-minimal varints, byte-sized array blocks — comes back canonicalised. Byte-for-byte identity holds for payloads from conformant writers (apache-avro's own encoders included); put-get and put-put hold unconditionally.
  • '''Writes require a decodable current focus''': the Affine to decodes eagerly, so .replace onto a span whose current value doesn't decode as A — or a .union[B] focus sitting on a different runtime branch — is a Miss pass-through. graftBytes is the decode-free write (and the only one that can SWITCH union branches).
  • '''A write that fails to ENCODE the new value is a silent pass-through''' (from is total by type — there is nowhere in Optic.from to put the failure). The payload comes back byte-uncanonicalised but otherwise UNCHANGED, and the call reports success. This is deliberate-but-unsatisfying and is pinned by AvroWriteCorrectnessSpec; the record face's AvroRecordPrism *Ior members are where such a failure is visible (as AvroFailure.DecodeFailed — the encode is funnelled through the same decodeOrFail seam). See docs/research/2026-09-22-exception-audit.md (class C) for why the fix is a failure-typed write carrier rather than a local throw — tracked as issue #117.
  • '''The payload must be encoded under exactly this prism's reader schema.''' This is about PAYLOAD drift — a name absent from the READER schema is a construction-time refusal on both .field and .fieldNamed, not a runtime miss. The byte walk performs no writer/reader schema resolution: structurally drifted payloads Miss silently, and a same-typed field REORDER between writer and reader is undetectable from the bytes — the walk reads the wrong field with full confidence. Confluent-framed payloads are handled by composing ConfluentWire.confluent (a byte Prism that strips the header, resolves the writer schema, and fingerprint-gates) BEFORE this optic — confluent.andThen(thisWalk); past a fingerprint mismatch a mixed-schema topic still needs a resolving decode (the record face with the right schema per payload).
  • Dynamic field sugar is shadowed by real members: an Avro field named like a member of this class (record, field, at, union, each, fields, …) must be drilled with the explicit .field(_.record) form.

Attributes

Companion
object
Source
AvroPrism.scala
Graph
Supertypes
trait Dynamic
trait Optic[Array[Byte], Array[Byte], A, A, Affine]
class Object
trait Matchable
class Any

Members list

Grouped members

Operations

inline def cross[C, D](o: Optic[Array[Byte], Array[Byte], C, D, Affine])(using Accessor[Affine], ReverseAccessor[Affine]): Optic[A, A, C, D, Affine]

Build-then-observe across the build-output ⇄ read-input seam, preserving structure, on a shared carrier F. Flip self (it must be reversible — Accessor[F] and ReverseAccessor[F], i.e. an Iso or Review over Direct) so it reads T from B, then andThen that under the same carrier. The result is the full Optic[B, A, C, D, F], not a collapsed getter: its read capability follows the carrier (.get for Direct), and self's read focus A survives as the composite's write-back focus.

Build-then-observe across the build-output ⇄ read-input seam, preserving structure, on a shared carrier F. Flip self (it must be reversible — Accessor[F] and ReverseAccessor[F], i.e. an Iso or Review over Direct) so it reads T from B, then andThen that under the same carrier. The result is the full Optic[B, A, C, D, F], not a collapsed getter: its read capability follows the carrier (.get for Direct), and self's read focus A survives as the composite's write-back focus.

This is exactly self.reverse.andThen(that). The motivating case is ana.cross(cata): a Review (the unfold) crossed with a getter on the built S (the fold) — a (materializing) hylomorphism whose .get reads the folded value. When that sits on a different carrier (a Prism, a Fold, …), the cross-carrier cross overload below is selected instead.

Seam: that's source is self's T and its back-type is self's S.

Attributes

Inherited from:
Optic
Source
Optic.scala

Type members

Types

type X = (Array[Byte], (Array[Byte], BinarySpan))

Structural leftover: Fst[X] is the original payload (Miss pass-through), Snd[X] the payload plus the located span so from can splice without re-walking.

Structural leftover: Fst[X] is the original payload (Miss pass-through), Snd[X] the payload plus the located span so from can splice without re-walking.

Attributes

Source
AvroPrism.scala

Value members

Concrete methods

transparent inline def at(i: Int): Any
Extension method from AvroPrism

.at(i) — drill into the i-th array element / map entry.

.at(i) — drill into the i-th array element / map entry.

Attributes

Source
AvroPrism.scala
transparent inline def each: Any
Extension method from AvroPrism

.each — split into an AvroTraversal over the iterated array.

.each — split into an AvroTraversal over the iterated array.

Attributes

Source
AvroPrism.scala
transparent inline def field[B](inline selector: A => B)(using codecB: AvroCodec[B]): AvroPrism[B]
Extension method from AvroPrism

.field(_.x) — drill via selector lambda.

.field(_.x) — drill via selector lambda.

Attributes

Source
AvroPrism.scala
def fieldNamed[B](schemaName: String)(using codecB: AvroCodec[B]): AvroPrism[B]
Extension method from AvroPrism

.fieldNamed[B]("schema_name") — drill by the EXPLICIT schema field name, bypassing resolution entirely. The escape hatch for a hand-written codec the resolver cannot read (a field list that both renames beyond recognition and reorders, a column bearing another field's name, two columns normalising alike); the common (derived / order-preserving / name-transformed) codecs need .field(_.x) instead, which resolves the name for you.

.fieldNamed[B]("schema_name") — drill by the EXPLICIT schema field name, bypassing resolution entirely. The escape hatch for a hand-written codec the resolver cannot read (a field list that both renames beyond recognition and reorders, a column bearing another field's name, two columns normalising alike); the common (derived / order-preserving / name-transformed) codecs need .field(_.x) instead, which resolves the name for you.

The name is CHECKED against the schema it will be looked up in, at construction (issue #95): a name the record does not carry throws rather than Missing silently at runtime. A MAP parent is carved out — .fieldNamed is also how a map KEY is addressed, and an absent key is data, not a mistake. To feature-detect a RECORD field, ask the schema: codec.schema.getField(name).

Attributes

Source
AvroPrism.scala
transparent inline def fields(inline selectors: A => Any*): Any
Extension method from AvroPrism

.fields(_.a, _.b, ...) — focus a NamedTuple over selected fields.

.fields(_.a, _.b, ...) — focus a NamedTuple over selected fields.

The AvroCodec for the synthesised NamedTuple is summoned at the call site; with no hand-written given in scope it auto-derives through kindlings. The 22-selector ceiling is gone: up to 22 selectors the focus is spelled TupleN and hearth builds it with that tuple's constructor; at 23 and above the focus type IS a *: cons chain, which hearth ≥ 0.4.2 builds through Tuple.fromArray instead. (Both spellings are needed — hearth's constructor branch below 23 cannot build a cons chain, and there is no TupleN above 22.)

That is not the same as "unbounded". .fields selects from a case class, so '''254 selectors is a hard, permanent ceiling''' — the JVM caps a parameter list at 254 slots and a 255-field case class does not compile at all. Well below that, the binding limit is the compiler thread's stack, because the derivation recurses per field: measured, -Xss1m (the JVM default) tops out around 32 selectors, -Xss4m around 150, and -Xss8m — what this repo's .jvmopts sets — reaches 254. A downstream build gets none of that automatically, so raise -Xss there before concluding a cover is too wide. Compile time is the third cost: the derivation grows superlinearly in arity, so a wide cover may also want a higher -Xmacro-settings:avroDerivation.timeout=30s — the unit suffix is required, a bare integer is silently ignored. (Issue #96.)

Attributes

Source
AvroPrism.scala
def from(aff: Affine[X, A]): Array[Byte]

Re-encode the focus and splice it over the located span; a Miss passes the payload through unchanged.

Re-encode the focus and splice it over the located span; a Miss passes the payload through unchanged.

Attributes

Source
AvroPrism.scala
def graftBytes(bytes: Array[Byte], fragment: Array[Byte]): Either[AvroFailure, Array[Byte]]

Graft an encoded fragment into a binary payload at the focused field — splice bytes in place of the field's current encoding, decode-free: prefix copy + (when the prism focuses a union branch: the zigzag-encoded index of the prism's focused branch, re-synthesised from the path — the payload's current branch is discarded, so grafting can SWITCH branches) + the fragment bytes + suffix copy.

Graft an encoded fragment into a binary payload at the focused field — splice bytes in place of the field's current encoding, decode-free: prefix copy + (when the prism focuses a union branch: the zigzag-encoded index of the prism's focused branch, re-synthesised from the path — the payload's current branch is discarded, so grafting can SWITCH branches) + the fragment bytes + suffix copy.

'''NO SCHEMA VALIDATION.''' The fragment bytes are spliced verbatim — this method does NOT check that fragment is a valid encoding of the focused field's schema. A fragment encoded under a different (or drifted) schema produces a payload that is silently corrupt until something decodes it. The caller owns schema-fingerprint checks (compare the writer schema / Confluent schema id of the fragment's source against the receiving field's schema) BEFORE grafting — the ConfluentWire.graftGated / ConfluentWire.graftResolving factories are the fingerprint-gated wrappers that do exactly that.

The located AvroFailure is a bare Left (the graft locate yields exactly one failure, nothing to accumulate); the spliced payload a Right.

Attributes

Source
AvroPrism.scala
def graftBytesUnsafe(bytes: Array[Byte], fragment: Array[Byte]): Array[Byte]

Silent counterpart to graftBytes — input bytes pass through unchanged on any failure. Same NO-SCHEMA-VALIDATION caveat as graftBytes.

Silent counterpart to graftBytes — input bytes pass through unchanged on any failure. Same NO-SCHEMA-VALIDATION caveat as graftBytes.

Attributes

Source
AvroPrism.scala

The IndexedRecord-carried counterpart of this prism — same focus, but Optic[IndexedRecord, IndexedRecord, A, A, Affine] plus the Ior-bearing diagnostic surface. Drill here on the byte prism, then flip at the end:

The IndexedRecord-carried counterpart of this prism — same focus, but Optic[IndexedRecord, IndexedRecord, A, A, Affine] plus the Ior-bearing diagnostic surface. Drill here on the byte prism, then flip at the end:

 codecPrism[Person].field(_.name).record.modify(_.toUpperCase)(record)

Attributes

Source
AvroPrism.scala
transparent inline def selectDynamic(inline name: String): Any

Dynamic field sugar — codecPrism[Person].name lowers to codecPrism[Person].field(_.name) via the macro. Shadowed by real members (record, field, at, …); use the explicit .field(_.x) form for an Avro field named like one of them.

Dynamic field sugar — codecPrism[Person].name lowers to codecPrism[Person].field(_.name) via the macro. Shadowed by real members (record, field, at, …); use the explicit .field(_.x) form for an Avro field named like one of them.

Attributes

Source
AvroPrism.scala
def sliceBytes(bytes: Array[Byte]): Either[AvroFailure, AvroFragment]

Slice the focused field's encoded value bytes out of a binary payload.

Slice the focused field's encoded value bytes out of a binary payload.

The returned AvroFragment carries the value bytes (union branch index STRIPPED when the prism focuses a union branch), the resolved field / branch schema, and the branch ordinal. The runtime branch must match the prism's focused branch — a payload sitting on a different branch surfaces AvroFailure.UnionResolutionFailed.

Array-index steps are unsupported (AvroFailure.UnsupportedSpanStep); for a .fields(...) prism the span addressed is the PARENT record enclosing the selected fields (the selected fields themselves are not contiguous).

The located AvroFailure is a bare Left (the span locate yields exactly one failure, nothing to accumulate); the fragment a Right.

Attributes

Source
AvroPrism.scala
def sliceBytesUnsafe(bytes: Array[Byte]): Option[AvroFragment]

Silent counterpart to sliceBytes — None on any failure.

Silent counterpart to sliceBytes — None on any failure.

Attributes

Source
AvroPrism.scala
def to(bytes: Array[Byte]): Affine[X, A]

Locate + slice-decode the focus — Hit carries the span for from, Miss the payload.

Locate + slice-decode the focus — Hit carries the span for from, Miss the payload.

Attributes

Source
AvroPrism.scala
transparent inline def union[Branch]: Any
Extension method from AvroPrism

.union[Branch] — drill into a union alternative by branch type.

.union[Branch] — drill into a union alternative by branch type.

Attributes

Source
AvroPrism.scala

Inherited methods

def andThen[C, IB, G[_, _]](inner: Optic[A, Unit, C, IB, G])(using rc: ReadCompose[Affine, G]): rc.Out[Array[Byte], C]

ANY outer ∘ read-only inner — the inner is honestly one-way (T = Unit: a Getter, AffineFold, or Fold), so only the two READ sides matter and the composite collapses to the read-only join of their strengths via compose.ReadCompose (Getter / PickFold / ForgetFold).

ANY outer ∘ read-only inner — the inner is honestly one-way (T = Unit: a Getter, AffineFold, or Fold), so only the two READ sides matter and the composite collapses to the read-only join of their strengths via compose.ReadCompose (Getter / PickFold / ForgetFold).

A trait member (not an extension in the companion) deliberately: once a receiver is statically one of the fused concrete classes, its andThen member overloads enter resolution and Scala 3 never falls back to extension methods when they all fail — the collapse must be in the member overload set to be reachable without an expected-type ascription.

Only the inner's T is pinned to Unit; its B stays free (IB) even though read-only inners always have B = Unit. That keeps this overload strictly LESS specific than the same-carrier andThen above (which accepts every argument this one does whenever B = Unit at the receiver), so a same-carrier read-only ∘ read-only call resolves unambiguously to the AssociativeFunctor path and this one fires exactly on the cross-seam cells the generic member cannot type.

Attributes

Inherited from:
Optic
Source
Optic.scala
inline def andThen[C, D](o: Optic[A, A, C, D, Affine]): Optic[Array[Byte], Array[Byte], C, D, Affine]

Compose with another optic under the shared carrier F. Requires AssociativeFunctor[F]. Cross-carrier composition (Lens → Optional, Lens → Traversal, …) goes through the Morph-summoning overload of this same method.

Compose with another optic under the shared carrier F. Requires AssociativeFunctor[F]. Cross-carrier composition (Lens → Optional, Lens → Traversal, …) goes through the Morph-summoning overload of this same method.

Attributes

Example
case class Address(street: String)
case class Person(address: Address)
val streetLens = lens[Person](_.address).andThen(lens[Address](_.street))
Inherited from:
Optic
Source
Optic.scala