KOMOJU + Shopify nuts & bolts


Before Payments Apps, Shopify used a system called HPSDK. That stands for Hosted Page Software Development Kit. The idea of a “hosted page” is: when I try to pay for a product on your store, I’m brought to a “page” that’s “hosted” by someone other than Shopify.

The new Payments Apps system still uses a hosted page. The primary difference is that it’s built on top of Shopify’s comprehensive Apps system, which was released after the original HPSDK.

Benefits of the new system

We’re not Shopify so we can’t tell you for sure why they choose to do what they do, but we can certainly give you the rundown of what kind of improvements you can expect with this new system.

1. Plug-n-play with OAuth

With the old HPSDK system, the connection between your Shopify store and your KOMOJU account had to be done by hand. This process required the merchant to find their API key on KOMOJU, and then paste it into their Shopify store. This was rather error-prone, and we always had a steady stream of merchants who paste the wrong key, accidentally included a space, etc.

The new Payments Apps system uses the same OAuth setup as the rest of the Shopify Apps ecosystem. This allows merchants to easily discover, add, and remove 3rd party payment providers.

This means that merchants are no longer responsible for ensuring that they don’t accidentally insert test-mode credentials into a live store. Shopify and KOMOJU now handle this automatically based on whether or not the store is a development store, and whether or not the order is a test order.

2. Idempotency + retry semantics

Shopify’s new Payments Apps system enforces a retry policy in case of server-side instability. This means that when KOMOJU attempts to contact Shopify, and a failure occurs on Shopify’s side, KOMOJU must try again. This is enforced for all API requests both inbound and outbound.

Another requirement (again, on both sides), is idempotency. We call an action “idempotent” when performing it multiple times has the same result as performing it only once.

In the above diagram, our “mark order as paid” request successfully reaches Shopify’s servers, but the response didn’t make it back to KOMOJU. As far as Shopify knows, this order is paid. KOMOJU, on the other hand, does not know the state of the order.

Thankfully, because the “mark order as paid” operation is idempotent, KOMOJU can simply try again without worrying about duplicate work. Shopify will kindly return the success response without doing any actual work. The actual work was performed during the previous request. This is nice, especially for refunds where multiple refund attempts could potentially mean “refund 500 yen twice” instead of “refund 500 yen, oops something went wrong, try again”.

The TL;DR is: expect less user-facing errors.

Drawbacks of the new system

As of writing, there are a few wrinkles that still need ironing out. We expect these to be resolved sooner or later, but they’re important to know, as they can have a noticeable effect on both merchants and customers.

1. "pending" order status is missing

This is one feature from the HPSDK that KOMOJU relies on heavily for asynchronous payment methods like Konbini and Bank Transfer.

This is one feature from the HPSDK that KOMOJU relies on heavily for asynchronous payment methods like Konbini and Bank Transfer.

"Authorized" for konbini is not the same as authorized for credit card. Money has not yet been transferred.

The diagram above shows how KOMOJU and Shopify keep track of state as the user makes a Konbini payment.

It’s important to note that this is not the same as the “authorized → captured” flow of a credit card payment. An “authorized” credit card payment basically means the funds already belong to the merchant. The merchant chooses when to capture. For konbini or bank transfer, we are waiting for the customer to do something. The customer chooses when – and if – to capture.

Shopify’s new Payments Apps API currently does not allow us to mark payments as pending. Thus, we are forced into the following flow:

As you can see, Shopify remains in a state of “no payment” until captured. This comes with the following drawbacks:

  • Customers cannot return to the Shopify store webpage while the Shopify order is in a “no payment” state. If the customer returns to Shopify, they will be prompted to pay again.
  • Stock reservation is delayed until capture, severely increasing the probability of oversell. This leads into the next point.

2. Oversells are still possible

This is a common problem for merchants of scarce goods. If your products’ stock regularly drops to 0, you’ve likely experienced an oversell. “Oversell” simply means that your stock drops below 0; you’ve sold more items than you have listed.

Checkout is a multi-step process, and e-commerce platforms need to ask themselves: when do we want to reserve stock? If you’re too early, you’ll have to deal with customers abandoning checkout. If you’re too late, you face a choice between poor UX and oversells.

The above diagram is a (somewhat simplified) illustration of the checkout flow from cart to konbini. Each transition is labeled TN. An e-commerce platform can choose to reserve stock at any given transition. The idea is, reserving stock has the chance to fail. If the customer requests 2 items, and stock is 0 or 1, he or she might see a page like this:

Alternatively, the e-commerce platform may choose to simply oversell the product.

Let’s look at the tradeoffs of each T in figure 6.

  • (T1) Reserving stock here means that the customer can be confident that they get their product. However, the customer might abandon checkout and unintentionally hold up stock. The merchant is burdened with deciding how long to wait before invalidating the customer’s stock.
  • (T2T3) Reserving stock here probably decreases the number of abandoned checkouts that hold up stock, but out-of-stock errors may be frustrating at this point. The customer will likely feel like they were too slow to fill out the form – and they’d be correct.
  • (T4) Reserving stock at this point means you’re too late to show an error page. The ship has already sailed and the convenience store is awaiting cash. We cannot show “out of stock” errors at the konbini. If you reserve stock at this step, you have no choice but to allow oversells. Shopify reserves stock at this step for hosted page payments.

How to handle this

Shopify’s stock reservation policy for hosted page payments leads to the least frustrating UX for customers, but at the expense of some administrative overhead for merchants.

Merchants using hosted page payments for scarce products should always keep a stock buffer.

For example, if you really have 100 items in inventory, set your Shopify stock to something lower, like 90 to allow oversells.

The perfect formula to decide how much of a buffer to keep will depend on the specific merchant’s traffic patterns.

This requires external stock management outside of Shopify, but it will pay off in terms of UX.

Please note that Shopify’s Payments Apps API do not provide any stock information to payment providers. Thus, third-party payment providers like KOMOJU cannot help you avoid oversells.

For Further Reading

Payment methods

All Payment Methods