codegen

The SDK ships four protoc plugins that generate running Go code from your proto definitions. You invoke them through buf generate. Install them with:

go install github.com/infobloxopen/devedge-sdk/cmd/protoc-gen-svc@latest
go install github.com/infobloxopen/devedge-sdk/cmd/protoc-gen-storage@latest
go install github.com/infobloxopen/devedge-sdk/cmd/protoc-gen-ent@latest
go install github.com/infobloxopen/devedge-sdk/cmd/protoc-gen-devedge-authz@latest

protoc-gen-devedge-authz

This plugin reads (infoblox.authz.v1.rule) method annotations and emits a []authz.MethodRule table as a checked-in file. The result is equivalent to what authzpb.RulesFromGlobal() would produce by reflection, but is verified at compile time.

The generated Register<Service> (see protoc-gen-svc below) calls server.AddRules(<Service>AuthzRules...) for you, so a service’s rules are contributed to the server at registration time. server.Config.Rules is an optional additive override — you can merge extra rules in when you need to, but the generated path does not require it. The boot-time completeness gate runs at server.Serve over the accumulated rule set (fail-closed); see server → Serve.

The plugin emits one <Service>AuthzRules variable per service declaration, plus an AllAuthzRules that concatenates them for the whole file:

// Generated:
var FooServiceAuthzRules = []authz.MethodRule{ /* ... */ }
var FooSummaryServiceAuthzRules = []authz.MethodRule{ /* ... */ }
var AllAuthzRules = slices.Concat(FooServiceAuthzRules, FooSummaryServiceAuthzRules)

Register each service (or use the …WithRepository one-liner) and its rules are contributed automatically — there is nothing to pass to Config.Rules. AllAuthzRules is useful when you want the whole file’s rules in one reference, for example to seed a permission catalog.

protoc-gen-svc

This plugin generates the server-package wiring for each service. For a WidgetService over a Widget resource it emits:

• Register<Service>(s *server.Server, impl <Service>Server) error — records the service’s methods on the server, contributes <Service>AuthzRules via server.AddRules, registers impl on the gRPC server, and wires the HTTP/JSON gateway. The boot-time authz completeness gate runs later, at server.Serve, over the accumulated rule set (fail-closed).

• A generated default CRUD handler, <Service>CRUDHandler. It embeds Unimplemented<Service>Server and holds a persistence.Repository[*<Resource>, string], with one method per detected AIP standard method delegating to the repository:

  Detected RPC shape	Generated body
  Create<R> (request carries the resource)	repo.Create(ctx, req.Get<R>())
  Get<R> / Delete<R> (request keyed by id or an AIP-122 name)	repo.Get/repo.Delete — when keyed by name, parses it via Parse<R>Name
  List<R>s (paging + optional filter/order_by/show_deleted)	repo.List(persistence.ListOptions{...})
  Update<R> (resource + update_mask)	repo.Update(res.Id, res, update_mask...)
  Undelete<R> (soft-delete resources)	repo.Undelete

  Detection is by request/response shape, not name alone, and tolerates extra optional fields. An RPC matching no standard shape (for example an AIP-136 custom method) is left Unimplemented and is never silently mis-handled. Tenant stamping is the repository’s job — it stamps account_id from context on create. The interceptor chain applies read_mask/field_mask, so the handler does neither.

• New<Service>Handler(repo) *<Service>CRUDHandler — returns the default handler so you can embed or wrap it before registering via Register<Service>.

• Register<Service>WithRepository(s *server.Server, repo) error — the one-call CRUD path (New<Service>Handler + Register<Service>). A pure-CRUD service needs nothing else:

  repo := myv1.NewWidgetRepository(db) // or NewWidgetEntRepository(client)
  if err := myv1.RegisterWidgetServiceWithRepository(s, repo); err != nil { /* ... */ }
  // ... s.Serve(ctx)

Overriding a method (escape hatch)

The generated handler is marked DO NOT EDIT. To add custom or non-CRUD logic, embed <Service>CRUDHandler in your own type and redefine only the methods you change. The remaining methods still come from the generated defaults, and any custom RPC the generator left Unimplemented is now served. Regenerating codegen does not disturb your override because it lives in your file:

type handler struct {
    myv1.WidgetServiceCRUDHandler // Get/List/Update/Delete come from here
}

// Override Create to add custom logic; everything else is the generated default.
func (h *handler) CreateWidget(ctx context.Context, req *myv1.CreateWidgetRequest) (*myv1.Widget, error) {
    if req.Widget.Id == "" { req.Widget.Id = uuid.New().String() }
    return h.Repo.Create(ctx, req.Widget)
}

// ArchiveWidget is an AIP-136 custom method the generated handler left Unimplemented.
func (h *handler) ArchiveWidget(ctx context.Context, req *myv1.ArchiveWidgetRequest) (*myv1.ArchiveWidgetResponse, error) { /* ... */ }

h := &handler{}
h.WidgetServiceCRUDHandler.Repo = repo
_ = myv1.RegisterWidgetService(s, h) // plain Register, with your overriding handler

For a fully custom (non-CRUD) service, implement the bare <Service>Server interface and use Register<Service>(s, custom) directly.

In a scaffolded service the s *server.Server you register on is app.Server, the shared server a servicekit module is handed. Add a custom method or second resource walks this override through the module and host.

Cross-service references + guaranteed BatchGet

A UI view usually spans domains served by different microservices — an asset and its region, a key and its user. To make a service federatable, declare a cross-service reference once and the framework guarantees the referenced target can be batch-fetched, so a composition layer resolves N references in one round trip (no N+1). This is the substrate a link-expansion consumer (REST ?expand=) builds on; it does not itself expand links.

Declare the reference on the scalar foreign-key field with the standard google.api.resource_reference (AIP-124) — there is no devedge-specific annotation:

import "google/api/resource.proto";

message Asset {
  option (google.api.resource) = { type: "asset.example.com/Asset" pattern: "assets/{asset}" };

  string id        = 1;
  // Cross-service reference: names a resource served by another microservice.
  string region_id = 2 [(google.api.resource_reference) = { type: "region.example.com/Region" }];
}

type is the target’s AIP-122 resource type (matching the target’s google.api.resource.type). Types are globally unique, so the target module/endpoint is resolved from the catalog — you do not annotate the module.

Emitted metadata (<Service>References). For each annotated field, protoc-gen-svc emits a DO NOT EDIT table of reference.Reference{ FieldName, FKField, TargetType, Cardinality } — the same emission style as <Service>AuthzRules. A composition layer reads it to know what to resolve and where:

var AssetServiceReferences = []reference.Reference{
    { FieldName: "RegionId", FKField: "region_id", TargetType: "region.example.com/Region", Cardinality: reference.One },
}

Guaranteed BatchGet on referenced targets (referenced ⇒ batch-fetchable). When a service’s proto declares an AIP-137 BatchGet<R> RPC (request carries repeated string ids/names, response the repeated resource, read rule), protoc-gen-svc generates the handler — it delegates to the repository’s BatchGet (AIP-137), so the guarantee holds by construction. Because the batch path is required, that service’s generated <Service>CRUDHandler.Repo is a persistence.BatchRepository[*<R>, string] (not the plain Repository); the generated ent/GORM/Memory repositories already satisfy it. This is automatic at initial scaffold; if you add a BatchGet<R> RPC to an existing resource, regenerating widens those generated signatures, so you must widen the hand-owned module.go/main.go repository type to persistence.BatchRepository[*<R>, string] and switch its constructor to New<R>EntBatchRepository / New<R>GormBatchRepository. BatchGet flows through the same fail-closed authz interceptor (verb read) and read_mask middleware (AIP-157) as Get/List; row-level Obligations apply. It is not a projection or privilege escape hatch.

Fail-loud — never a silent N+1. A reference whose target type does not serve BatchGet fails, loud, at two gates:

• Codegen (primary). A reference target’s Repo is a BatchRepository, so passing a non-batch repository is a compile error before a binary exists.
• Registration / Serve (backstop). The generated Register<Service> records the service’s references (RecordReferences) and, if it serves BatchGet, declares its resource a batch target (RecordBatchTarget). At Serve, AssertReferenceTargets fails closed if any recorded reference names a target type with no registered BatchGet on the server — catching cross-repo / version skew that local codegen cannot see. When the target is served by a different process (split-microservice federation), the reference source declares it with Server.RecordExternalReferenceTarget("<domain>/<Target>") so the gate passes without a local BatchGet; the gateway resolves it (see graphql-federation).

Write-boundary invariant (metadata only). A reference is metadata, not a Go edge. The region_id field stays a scalar foreign key: there is no traversable Go accessor from Asset into Region, no ent/GORM edge, and no cascade. Reads compose above the services (a composition layer resolves references); writes always route to the owning resource’s service. This mirrors the deliberately edge-less infoblox.ddd.v1.references invariant.

Resolving without an N+1 (the reference package). reference.Load is the DataLoader-style primitive: given N parents naming M distinct targets, it dedups the foreign keys and issues one BatchGet of the distinct set through a reference.ReferenceResolver. A StaticResolver maps a target type to an in-process BatchGet-capable client for tests and single-binary compositions; a catalog-backed resolver (dialing services) arrives with link-expansion.

resolver := reference.NewStaticResolver()
resolver.Register("region.example.com/Region", regionBatchClient) // implements reference.BatchGetter[*Region]

// N assets -> ONE BatchGet of the distinct region ids.
byID, err := reference.Load[*regionv1.Region](ctx, resolver, ref, assets,
    func(a *assetv1.Asset) []string { return []string{a.GetRegionId()} },
    func(r *regionv1.Region) string { return r.GetId() },
)

protoc-gen-storage

This plugin generates a GORM-backed Repository for each message. For a message named APIKey it emits:

• APIKeyModel — the GORM model. Standard columns plus ETag, CreatedAt, UpdatedAt — and, only for resources that opt into soft-delete (see below), a DeletedAt column of type gorm.DeletedAt. The APIKey example opts in, so its model has DeletedAt. The ETag column is populated only when the resource declares an etag field (see ETag below).
• toModel_APIKey / fromModel_APIKey — converters. They skip repeated fields (TODO: JSONB), nested messages (TODO: serialization), and secret fields.
• APIKeyRepository + NewAPIKeyRepository — Get, List, Create, Update, Delete, satisfying persistence.Repository[*APIKey, string] (a compile-time var _ check is emitted).

Soft-delete

Soft-delete is opt-in per resource, following AIP-148/149. A message becomes soft-deletable when it declares a server-managed delete_time timestamp:

google.protobuf.Timestamp delete_time = 7 [(google.api.field_behavior) = OUTPUT_ONLY];

The field name must be delete_time, the type google.protobuf.Timestamp, and it must be OUTPUT_ONLY — the framework sets it, not the client. When present, protoc-gen-storage emits the full soft-delete shape:

• a DeletedAt column (gorm.DeletedAt, indexed) on the model;
• Delete performs a soft delete (stamps deleted_at) instead of a physical row delete;
• List excludes soft-deleted rows unless ListOptions.ShowDeleted is set;
• Undelete (AIP-149) clears deleted_at and restores the row.

Without a delete_time field a resource is hard-delete by default: Delete physically removes the row, List has no deleted-row filter, and Undelete is a stub that always returns persistence.ErrNotFound so the Repository interface stays uniform. Adding the delete_time field is the only switch — there is no separate annotation or option.

Soft-delete with unique fields and the dialect option

When a resource is both soft-deletable and has a per-tenant unique field (for example source_ref), the per-tenant composite unique constraint (account_id, <field>) must not let a soft-deleted row keep the key reserved. You must be able to create a fresh resource with the same value after the old one is soft-deleted, and Undelete must correctly fail with a conflict if the key was taken in the meantime. The codegen handles this automatically, using a strategy chosen by the dialect plugin option (passed to both protoc-gen-ent and protoc-gen-storage):

Set the option in buf.gen.yaml to match your production database (dev on SQLite still works either way):

plugins:
  - local: protoc-gen-storage
    opt: [module=…, dialect=mysql]   # omit for postgres/sqlite (the default)
  - local: protoc-gen-ent
    opt: [dialect=mysql]

The behavior is identical across backends and dialects; only the physical schema differs. A per-tenant unique field on a resource without soft-delete is unaffected — it gets a plain composite unique index.

A sibling marker, google.protobuf.Timestamp expire_time = N [(google.api.field_behavior) = OUTPUT_ONLY], additionally emits a PurgeExpired(ctx, before) method that hard-deletes rows past their TTL. expire_time is OUTPUT_ONLY, so your Create handler stamps it (for example b.ExpireTime = timestamppb.New(time.Now().Add(ttl)) before repo.Create). The generated toModel carries it to the expire_time column, and PurgeExpired(ctx, time.Now()) removes rows whose stamp has passed.

ETag (AIP-154 / optimistic concurrency)

Declare a server-managed ETag field on the resource:

string etag = N [(google.api.field_behavior) = OUTPUT_ONLY];

protoc-gen-storage then stamps a fresh opaque token on every Create and Update, and surfaces it on every read (fromModel copies the stored etag column onto the proto). A Get therefore returns a stable token a client echoes as If-Match; the token changes on the next write, so a stale If-Match is rejected with a 412 (see middleware → ETag and preconditions for the handler pattern). The token is opaque — clients must not parse it.

On the ent backend you get the same behavior. protoc-gen-ent adds the generated entrepo.EtagMixin, which supplies the etag column and a mutation hook that stamps a fresh etag.New() token on every Create and Update automatically, with no consumer code on the write path. The generated fromEnt<Resource> projection surfaces it on reads (p.Etag = e.Etag), so the AIP-154 round-trip works with no consumer code on either path.

The If-Match precondition comparison follows the same handler pattern on both backends (see middleware → ETag and preconditions).

Secret fields

A field marked (infoblox.field.v1.opts) = {secret: true} does not get a plaintext column. Instead protoc-gen-storage emits two columns:

KeyValueHash   string `gorm:"column:key_value_hash;index"` // for lookup
KeyValueCipher string `gorm:"column:key_value_cipher"`       // for recovery

The constructor then requires an Encryptor:

func NewAPIKeyRepository(db *gorm.DB, enc secret.Encryptor) *APIKeyRepository

Create/Update hash and encrypt the value. A LookupBy<Field>Hash method is emitted for each secret field so you can find a record by the hash of a presented value.

Tenant scoping

If a message has an account_id field, every Get/List/Update/Delete/LookupBy*Hash query adds an account_id = ? clause from middleware.TenantIDFromContext(ctx):

tenantID := middleware.TenantIDFromContext(ctx)
q := r.db.WithContext(ctx).Where("id = ?", key)
if tenantID != "" {
    q = q.Where("account_id = ?", tenantID)
}

protoc-gen-ent

This plugin generates an ent schema for each resource message (a message with an id field). Run go generate on the directory the schema landed in to produce the type-safe ent client. With the scaffold’s out: gen setting the command is go generate ./gen/ent; use go generate ./ent only if you set out: ..

The ent shape enforces tenant scoping and secret-field handling through ent’s privacy layer and hooks (applied by a generated mixin), so the invariants hold even for ad-hoc graph traversals — not just CRUD.

A message is treated as a resource when it declares an id field — the same rule protoc-gen-storage uses. Request/response wrappers and other transport types without an id (for example a consumer-declared LRO Operation) are skipped and receive no ent schema or batch wrapper.

protoc-gen-ent generates the ent schema (under ent/schema/), a batch wrapper, and the repository adapter itself (<resource>_repo.ent.go), so you write no hand-written ent wiring. The generated New<Resource>EntRepository fills the six entrepo.EntRepository[T, K] closures (Get/List/Create/Update/Delete/Undelete) over the ent client, with:

• tenant guards on mutations;
• persistence.ConstraintError classification;
• ent.IsNotFound → persistence.ErrNotFound mapping;
• the secret hash/cipher block;
• AIP-160 filtering from the generated <Resource>EntColumns maps;
• a conditional set for belongs_to foreign keys;
• the fromEnt<Resource> projection.

A LookupBy<Field>Hash is emitted per secret field. Construct the repository with:

repo := apikeyv1.NewAPIKeyEntRepository(client, enc) // persistence.Repository[*APIKey, string]

Customization

For computed or derived values that the generator cannot infer, register the generated hooks from your own regen-safe file. The adapter calls them when set, so re-running codegen does not disturb your custom logic:

func init() {
    apikeyv1.FromEntAPIKeyCustom  = func(e *ent.APIKey, p *apikeyv1.APIKey)     { /* read projection */ }
    apikeyv1.ToEntAPIKeyOnCreate  = func(p *apikeyv1.APIKey, b *ent.APIKeyCreate)   { /* extra write columns */ }
    apikeyv1.ToEntAPIKeyOnUpdate  = func(p *apikeyv1.APIKey, u *ent.APIKeyUpdateOne) { /* extra write columns */ }
}

Fail-closed field coverage

If a resource field has no deterministic mapping — a nested non-relationship message, a repeated non-relationship field, an enum, or a non-string map — generation fails and names the field and the remedy, rather than silently dropping it. To resolve this, make the field a relationship, give it a scalar storage type, or mark it OUTPUT_ONLY.

Multi-surface

The (infoblox.storage.v1.model) message option (from github.com/infobloxopen/apis/proto/infoblox/storage/v1) binds a resource message to a backing storage model so several API surfaces can project one stored entity. When absent or equal to the message’s own name, the message is an ordinary single-table resource. When the value names a different message, this message is a surface (a projection) over that owner’s model:

import "infoblox/storage/v1/storage.proto";

message Coupon { string id = 1; string account_id = 2; string code = 3; /* … */ }

message CouponSummary {                                 // a read projection over Coupon
  option (infoblox.storage.v1.model) = "Coupon";
  string id = 1; string account_id = 2; string code = 3; // a subset of Coupon's fields
}

Both protoc-gen-ent and protoc-gen-storage then emit, for the surface, a New<Surface>…Repository plus a projection over the owner’s type — and no table of its own (no ent schema, no GORM model struct). One model can have N repositories/projections. Mutation guards (tenant scope, soft-delete, undelete) follow the owner; the written/projected columns follow the surface’s own fields.

An owner and its surfaces are fully generated on both backends with no hand-written adapter. The apikey fixture’s APIKeySummary surface (a tenant-scoped projection over APIKey) round-trips owner-write → surface-read on ent and GORM with no hand-written persistence code (testdata/apikey/.../multisurface_test.go).

Resource names on ent (AIP-122)

When a resource carries a (google.api.resource) pattern and an OUTPUT_ONLY name field, the ent backend derives name from id and never stores it. The generated ent schema omits the name column, the adapter never writes it, and the generated fromEnt<R> projection recomputes it on every read via Format<R>Name(e.ID). A Get, List, Create, or Update response always carries the resource name with no consumer code.

The plugin also emits the same package-level helpers the GORM backend does:

const <R>NamePattern = "<resources>/{<resource>}"
func Format<R>Name(id string) string          // id → "resources/abc123"
func Parse<R>Name(name string) (string, error) // "resources/abc123" → id

Relationships in ent

For a parent/child pair — a has_many on the parent and a belongs_to on the child (the natural AIP shape) — the plugin emits a single ent edge as a proper inverse pair, referencing each related schema by its singular type:

// parent: Fleet has_many Vehicle
func (Fleet) Edges() []ent.Edge {
    return []ent.Edge{
        edge.To("vehicles", Vehicle.Type),
    }
}

// child: Vehicle belongs_to Fleet, with a scalar fleet_id FK
func (Vehicle) Fields() []ent.Field {
    return []ent.Field{
        field.String("id").StorageKey("id").Immutable(),
        field.String("fleet_id").Optional(),
    }
}
func (Vehicle) Edges() []ent.Edge {
    return []ent.Edge{
        edge.From("fleet", Fleet.Type).Ref("vehicles").Unique().Field("fleet_id"),
    }
}

The child’s .Field("fleet_id") binds the edge’s foreign key to the scalar fleet_id column, so ent generates a single SetFleetID setter — the scalar field and the association share one column rather than colliding. The FK stays a first-class, queryable scalar. A belongs_to whose parent does not declare a reciprocal has_many is emitted as a self-contained edge.To(...).Unique(), also binding the scalar FK when present.

A complete, buildable two-resource example (schema, ent client, wiring, and an edge-traversal test) lives in testdata/fleet/.

import (
    "database/sql"
    "database/sql/driver"

    _ "modernc.org/sqlite" // registers driver name "sqlite"
)

func init() {
    for _, n := range sql.Drivers() {
        if n == "sqlite3" {
            return // already present (e.g. mattn/go-sqlite3 pulled in transitively)
        }
    }
    db, _ := sql.Open("sqlite", ":memory:")
    drv := db.Driver()
    _ = db.Close()
    sql.Register("sqlite3", drv.(driver.Driver))
}

// client := enttest.Open(t, "sqlite3", "file:ent?mode=memory&_pragma=foreign_keys(1)")

import _ "<your-module>/ent/runtime" // installs mixin validators + tenant/soft-delete interceptors

Configuring buf.gen.yaml

version: v2
plugins:
  - local: protoc-gen-go
  - local: protoc-gen-go-grpc
  - local: protoc-gen-devedge-authz
  - local: protoc-gen-svc
  - local: protoc-gen-storage
  - local: protoc-gen-grpc-gateway
  - local: protoc-gen-ent

A real service uses one storage backend — protoc-gen-storage (GORM) or protoc-gen-ent, not both (ent replaces storage). The combined list above shows all available plugins; the devedge-sdk new service scaffold emits one or the other.

See Define a service for a complete configured example.