[FEATURE REQUEST] Awaitable workflows with rollbacks

[willdspd]

Let's say I have a createOrg route in my app, which is called when a user attempts to create a new organisation. In this route, I create an org record in my DB and a Stripe Customer. To ensure both operations succeed or fail as a whole, I would like to use a QStash Workflow. For this, I need the createOrg route to know if the workflow succeeded or failed so I can respond to the user appropriately. Let's say the user is redirected to a payments page after submitting the CreateOrgForm. I need to know the Stripe Customer has been successfully created at this point. I don't think this is currently possible, but I think it's a pretty essential feature. I imagine you could add a rollback callback to context.run, which executes in the background after the Workflow returns an error, to rollback the org record creation in the DB. There may already be a way to solve this, or maybe Workflows aren't even the right tool for this sort of operation?

[CahidArda]

Hi there,

I think workflow can be used in your use case. Consider the following flow:

const userCreated = await context.run("create user in db", async () => {
  const userCreated = await createUserInDb(...)
  return userCreated
})

if (userCreated) {
  var stripeUserCreated = await context.run("create stripe user", async () => {
    const userCreated = await createStripeUser(...)
    return userCreated 
  })
}

if (!stripeUserCreated) {
  await context.run("delete user from database", async () => {
    // delete user
  })
}

A rollback callback is possible, but it may overcomplicate things.

When it comes to showing the status to the user, an endpoint for checking the status of a workflow (which would have the executed steps and their results) is on our roadmap. It should be possible with this endpoint. Alternatively, the progress can be saved somewhere and shown to the user, like we do in our demo app

[willdspd]

Thanks for your response! Correct me if I’m wrong but it sounds like it would be impossible for the createOrg route to know the result of the workflow it triggered, and consequently respond with an error if the workflow failed. Therefore, the only option would be for the createOrg route to respond optimistically with a 202, then via a websocket return the result of the workflow to the client?

[CahidArda]

Yes. That's how workflow can be used at the moment, like a background job.

We could have methods which make it possible to wait until a workflow run finishes so that you can return the correct status to the user.

Can you elaborate more on this use case? I am especially curious about this: if we want to return the correct status to the user, a background job doesn't feel like the right fit. We have to wait for the job to finish.

[willdspd]

The use case is pretty much what I outlined in my hypothetical example, but it’s applicable to any sort of blocking operation that requires a workflow result/error to display to the user. I imaging it would looks something like:

1. createOrg route is called by client
2. createOrg route triggers a workflow
   3.1. If the workflow succeeds on its first try, respond 200 from the createOrg route.
   3.2.1. If a workflow step fails during it’s first try, throw an error in the createOrg route
   3.2.2. Retry the workflow in the background to run failed steps (or rollback steps) if needed