How can I validate test coverage for a state machine with invocation steps

[CodenameLiam]

I'm following along with the docs (https://stately.ai/docs/xstate-test and https://stately.ai/docs/testing), trying to understand how I can validate that I have full test coverage across my state machine

My understanding of createTestModel(machine).getShortestPaths() from the xstate/graph package is that it will traverse all paths on the graph, however it doesn't support invocations

I could use the Arrange, Act, Assert pattern described in https://stately.ai/docs/testing, but then I'm not sure how I'd deduce whether all paths are successfully being traversed

Could I please get some guidance on how to validate test coverage on a given state machine definition

[CodenameLiam]

Ok so I went a bit mad and implemented something that kind of works, but just keep in mind that this isn't super battle tested

I model every transition as a string from a source state to a target state with an appropriate event/guard (if there is one)

type SourceStateName = string;
type TargetStateName = string;
type EventName = string;
type GuardName = string;

type TransitionKey = `${SourceStateName}.${EventName}${GuardName} -> ${TargetStateName}`;

Then I get every possible transition in a given machine

const collectAllTransitionKeys = (machine: AnyStateMachine) => {
    const possibleTransitions = new Set<TransitionKey>();

    Object.values(machine.root.states).forEach((state) => {
        collectTransitionKeysFromNode(state, possibleTransitions);
    });

    return possibleTransitions;
};

const collectTransitionKeysFromNode = (node: AnyStateNode, possibleTransitions: Set<TransitionKey>) => {
    node.transitions.forEach((transitions) => {
        transitions.forEach((transition) => {
            buildTransitionKeys(transition).forEach((key) => possibleTransitions.add(key));
        });
    });

    node.always?.forEach((transition) => {
        buildTransitionKeys(transition).forEach((key) => possibleTransitions.add(key));
    });

    if (node.states) {
        Object.values(node.states).forEach((childState) => {
            collectTransitionKeysFromNode(childState, possibleTransitions);
        });
    }
};

I wrote an inspector to track transitions for an actor and stored them in the same format

function createStateMachineCoverageTracker(machine: AnyStateMachine) {
    const visitedTransitions = new Set<TransitionKey>();
    const expectedTransitions = collectAllTransitionKeys(machine);

    const recordTransition = (event: InspectionEvent) => {
        if (event.type === '@xstate.microstep') {
            event._transitions?.forEach((t) => {
                buildTransitionKeys(t).forEach((key) => visitedTransitions.add(key));
            });
        }
    };

    return {
        recordTransition,
        getCoverageReport: () => ({
            actual: Array.from(visitedTransitions).sort(),
            expected: Array.from(expectedTransitions).sort()
        })
    };
}

And then I wrote a simple helper function to validate full test coverage at the end of a given test suite

function withFullCoverage<TMachine extends AnyStateMachine>(
    machine: TMachine,
    testSuite: (runtime: ReturnType<typeof createStateMachineTestRuntime>) => void
): EmptyFunction {
    return () => {
        const runtime = createStateMachineTestRuntime(machine);

        testSuite(runtime);

        test('[coverage] all state transitions are exercised', () => {
            const { actual, expected } = runtime.getCoverageReport();
            expect(actual).toEqual(expected);
        });
    };
}

And so for for a simple machine like...

const machine = setup({}).createMachine({
    initial: 'inactive',
    states: {
        inactive: {
            on: { toggle: { target: 'active' } }
        },
        active: {
            on: { toggle: { target: 'inactive' } }
        }
    }
});

This test

describe(
    'some actor',
    withFullCoverage(machine, (runtime) => {
        const sut = runtime.getActor();

        test('happy path', () => {
            sut.send({ type: 'toggle' });
        });
    })
);

Fails with the result

Error: expect(received).toEqual(expected) // deep equality

- Expected  - 1
+ Received  + 0

  Array [
-   "active.toggle -> inactive",
    "inactive.toggle -> active",
  ]

Because we never transition from active back to inactive via the toggle event :)

Below is the rest of the code I used to build the above

function getGuardName(guard?: UnknownGuard): string {
    if (!guard) return '';

    // Case 1: Named string guard (e.g., 'myNamedGuard')
    if (typeof guard === 'string') {
        return `[${guard}]`;
    }

    // Case 2: Guard object with `type` key (e.g., { type: 'myNamedGuard' })
    if (typeof guard === 'object' && 'type' in guard) {
        const type = (guard as { type?: unknown }).type;
        if (typeof type === 'string') return `[${type}]`;
    }

    // Fallback: unnamed inline guard
    return '[inlineguard]';
}

const buildTransitionKeys = (transition: AnyTransitionDefinition): TransitionKey[] => {
    const sourceStateName = transition.source.key;
    const eventName = !!transition.eventType ? transition.eventType : 'always';
    const guardName = getGuardName(transition.guard);
    const targetStates = transition.target ?? [];

    return targetStates.map((target) => {
        const targetStateName = target.key;
        return `${sourceStateName}.${eventName}${guardName} -> ${targetStateName}` as TransitionKey;
    });
};

type ProvideInput<T> = T extends {
    provide(impl: infer U): any;
}
    ? U
    : never;

function createStateMachineTestRuntime<TMachine extends AnyStateMachine>(
    machine: TMachine,
    opts?: ActorOptions<AnyActorLogic>
) {
    const { getCoverageReport, recordTransition } = createStateMachineCoverageTracker(machine);
    const options: ActorOptions<AnyActorLogic> = { ...opts, inspect: recordTransition };

    let actor = createActor(machine, options);
    actor.start();

    const getActor = () =>
        new Proxy({} as Actor<TMachine>, {
            get(_, key) {
                return Reflect.get(actor, key as keyof Actor<TMachine>);
            }
        });

    const restart = () => {
        actor.stop();
        actor = createActor(machine, options);
        actor.start();
    };

    const provide = (impl: ProvideInput<TMachine>) => {
        const snapshot = actor.getSnapshot();
        actor.stop();
        actor = createActor(machine.provide(impl), { ...options, snapshot });
        actor.start();
    };

    const waitForState = (targetState: StateValueFrom<TMachine>): Promise<void> => {
        return new Promise((resolve) => {
            const snapshot = actor.getSnapshot();
            if (snapshot.value === targetState) {
                return resolve();
            }

            const subscription = actor.subscribe((snapshot) => {
                if (snapshot.value === targetState) {
                    subscription.unsubscribe();
                    resolve();
                }
            });
        });
    };

    return {
        getActor,
        restart,
        provide,
        waitForState,
        getCoverageReport
    };
}