Persistence

Stateful Functions treats state as a first class citizen and so all stateful functions can easily define state that is automatically made fault tolerant by the runtime.

Defining a Persistent Values

All stateful functions may contain state by merely defining one or more PersistedValue fields. A PersistedValue is defined by its name and the class of the type that it stores. The data is always scoped to a specific function type and identifier. Below is a stateful function that greets users based on the number of times they have been seen.

Warning

All PersistedValue fields must be marked with an @Persisted annotation or they will not be made fault tolerant by the runtime.

package com.ververica.statefun.docs;

import com.ververica.statefun.sdk.Context;
import com.ververica.statefun.sdk.FunctionType;
import com.ververica.statefun.sdk.StatefulFunction;
import com.ververica.statefun.sdk.annotations.Persisted;
import com.ververica.statefun.sdk.state.PersistedValue;

public class FnUserGreeter implements StatefulFunction {

  public static FunctionType TYPE = new FunctionType("ververica", "greeter");

  @Persisted
  private final PersistedValue<Integer> count = PersistedValue.of("count", Integer.class);

  public void invoke(Context context, Object input) {
    String userId = context.self().id();
    int seen = count.getOrDefault(0);

    switch (seen) {
      case 0:
        System.out.println(String.format("Hello %s!", userId));
        break;
      case 1:
        System.out.println("Hello Again!");
        break;
      case 2:
        System.out.println("Third time is the charm :)");
        break;
      default:
        System.out.println(String.format("Hello for the %d-th time", seen + 1));
    }

    count.set(seen + 1);
  }
}

Persisted value comes with the right primitive methods to build powerful stateful applications. Calling PersistedValue#get will return the current value of an object stored in state, or null if nothing is set. Conversely, PersistedValue#set will update the value in state and PersistedValue#clear will delete the value from state.

Supported Types

Stateful Functions applications are typically designed to run indefinitely or for long periods of time. As with all long-running services, the applications need to be updated to adapt to changing requirements. This goes the same for data schemas that the applications work against; they evolve along with the application. That is why the system limits the types that can be stored inside a PersistedValue to all Java primitives and complex types that support well defined schema migration semantics.

Note

Schema evolution is supported naturally with protobuf and json, and the project is working on connecting it to Flink’s schema evolution capabilities.

POJO types

Stateful Functions recognizes data types as a POJO type if the following conditions are fulfilled:

• The class is public and standalone (no non-static inner class)
• The class has a public no-argument constructor
• All non-static, non-transient fields in the class (and all superclasses) are either public (and non-final) or have a public getter- and a setter- method that follows the Java beans naming conventions for getters and setters.

Apache Avro

Stateful Functions can store any Apache Avro class and fully supports evolving schema of Avro type state, as long as the schema change is considered compatible by Avro’s rules for schema resolution.

Protocol Buffers

Stateful Functions can store any Protocol Buffer class and fully supports schema evolution as long as the schema change is considered compatible by ProtoBuff’s rules for schema evolution.

Json

Stateful Functions can store any object that serializes as JSON.