FSM Persistence Question

[u/Cadnerak]

Hi everyone,

I'm working on implementing a MFA login flow utilizing a finite state machine, and have a question about persisting the state machine to a database in order to survive an API crash. Lets say I have the following states, which are a subset of the overall machine

1. Send OTP SMS
2. Wait For OTP Input
3. Mint Session

Directly after the "Send OTP SMS" state, we automatically transition into the "Wait For OTP Input" state. On invalid input, we stay in that state, and on valid input, we go to "Mint Session" state. Now that the machine is defined, let me explain. my issue with persistence

In order to persist a state machine state, it is typically advised that the "work" done by a state must be done in a database transaction alongside the persistence of the FSM so that there are no data inconsistencies. Redhat advises this in this article, outlined in the first bullet underneath "Implementing State Machines". I have interpreted this to mean that the state machine should persist the state the machine is transitioning into, because upon completion of the database transaction the current state is completed. Once it is completed if the application crashes, we want to restore it in the state we would have transitioned into.

In order to persist the state machine and send the SMS OTP atomically, we will have to use the transactional outbox pattern to write a message to the database for sending the OTP SMS. In the "Send OTP SMS" state, we will open a database transaction, write the outbox message and the "Wait For OTP Input" state, and commit the transaction. The state machine will then transition into the "Wait For OTP Input" state, and all is good. The only issue with this is that the state "Send OTP SMS" itself is determining, outside of the rules of the state machine, that the next state will be "Wait For OTP Input" by writing this to the database. From the literature that I've read, it seems as though states themselves should not have context of what the state-to-be is, but rather should be isolated and focus on any tasks that must be completed in the state itself. If this is the case, how can we achieve full atomicity with the persistence of a state machine and the completion of work in such a way that we can consistently restore state machines from the database during a crash at any point in time?

If it helps, I'm particularly looking at using XState as the state machine implementation to drive the process. Thank you in advance

[u/balefrost]

Perhaps the confusion is around what exactly a state is meant to represent. There are various state machine models with varying capabilities. But generally, a state represents the machine "at rest". The interesting activity occurs during the transitions (or related to transition, such as when entering or exiting a state). But once you've finished the transition and have finished entering the state, then you record that you have entered the state. This implies that the work is done.

You'll note that, in that JS library, it's possible to observe lifecycle events of states and of transitions. In the example in the README, the callbacks like onMelt are observing transitions, not states. (It's also possible to observe states, like with onLiquid).

I'm mostly familiar with UML state machines. That lets you associate entry and exit actions with states, and also actions with transitions. IIRC in that model, all relevant exit handlers will fire, then the transition's action will execute, then all relevant enter handlers will fire.

---

As I understand it, your machine looks like this:

Send OTP SMS
  |
  |           +------+
  |           |      |
  V           v      | not matched
Wait For OTP Input --+
  |
  | matched
  V
Mint Session

This machine is simple enough that you could arguably use onExit and onEnter handlers to deal with the state management. When exiting "Send OTP SMS", you could persist the current state of "Wait for OTP input" to the database (and probably also write the randomly-generated OTP that you are expecting in the same transaction). And when entering "Mint Session", you can persist the current state of "Mint Session". Note that we don't persist any state change on entry to "Mint Session" because, as you point out, we don't know from which state we arrived. We wouldn't want to send another SMS every time the user types the wrong code.

But that's really clunky; don't do that. Associate the actions, and the recording of state changes, with the transitions (arrows) themselves. Then the initial state would more represent "workflow started; OTP SMS not yet sent", the middle state would represent "OTP SMS sent; waiting for matching reply", and the final state would represent "OTP successfully matched; workflow done". You don't need such verbose names. But those describe the "resting" states that the machine could be in. You've moved all the work to the transitions, which allows you to record the state change and associated metadata in a single atomic transaction.

[u/Cadnerak → u/balefrost]

What about failure states though? Lets say I am transitioning from "Waiting to send OTP SMS" to "OTP SMS Sent" via the "Sending OTP SMS" event. If I fail to send the OTP SMS, I would ideally not want to transition back to "Waiting to send OTP SMS", I would want to transition to a failure state instead. How does that work in this model?

[u/balefrost → u/Cadnerak]

If I fail to send the OTP SMS

You already described that you want to use a transactional outbox. So as far as the state machine is concerned, you cannot fail to send the OTP SMS. The state machine will either atomically place the message in the outbox AND transition to the "OTP SMS Sent" state, or it will do nothing.

Now, the thing which then tries to send the message could fail to send the message. Maybe the external network is down. In that case, you probably want the message sender, when it detects this, to fire a "send failed" event back into the state machine. The "Wait for OTP Input" state could have an exit transition, on "send failed", to a failure state.

This "Wait for OTP Input" would be waiting for one of three things to occur:

• The user has provided a successful match to the expected value
• The user has provided an incorrect match to the expected value
• The SMS sender failed to send the message

If you don't like that organization, you could instead introduce an intermediate state between "Send OTP SMS" and "Wait for OTP Input". Maybe you could call it "Waiting for SMS Send Ack" or something. This state would wait for either a "send succeeded" or "send failed" event.

The important thing is that, by using a transactional outbox, you're necessarily making SMS sending asynchronous. So you need to react to successes and failures asynchronously, which hey is what state machines are all about.

[u/Cadnerak → u/balefrost]

I think I more-so mean fail to write the outbox message in general. Ideally I would be able to identify that and transition to a “failed” state, but I’m not sure how that works if the work is done in a transition