One of the areas I'm focusing on for Commerce is building a webshop solution that is testable. With a suite of automated unit, functional, and mutational tests, you can make sure that code works as expected at all times. I've written about this before.
Opponents of testing often see it as something that takes a lot of time and effort (which is true), and that makes it expensive to do. An immediate return on investment is also rare, so automated tests are often pushed back or just skipped entirely. While I’ve been convinced about the merit of testing, I too fall into this trap and have yet to achieve the magical 100% test coverage on any sizeable project.
Unfortunately, especially when it comes to e-commerce, the cost of not properly testing every aspect of the code can be very real.
Take the modmore.com billing engine for example. It has been working pretty well for the past three years. I’ve done plenty of manual testing on my local environment to make sure it works as expected when making changes. I just haven't made the time to add unit tests to it, partly because not all the code is as easy to test.
Last weekend I was working on getting the documents in order for the first quarter taxes and shipped it all off to my accountant. Pretty satisfied with the quarterly results, I went on my merry way to do other stuff that didn’t involve looking at numbers and invoices. Until I got an email back from my accountant today about a discrepancy he noticed.
It turns out that there was an issue fetching the current VAT rates for the EU member states from a remote service. Previously we were fetching that over HTTP, but the service started requiring HTTPS, which resulted in a failed connection and the error handling wasn’t sufficient to cause alarms to go off about a critical issue.
Because of this issue, 44 orders which should have incurred 19-21% VAT, ended up being charged a 0% rate. As I still have to pay the VAT that should have been charged, that means I have to pay over €400 in VAT out of own pocket.
Luckily this only affected a relatively small slice of our orders and overall revenue from the past quarter. It won't put us out of business or anything, it just lowers the profit margins for those orders by a considerable amount.
But what would have happened if we had ten times as many clients, and we were talking about a €4000 charge instead? What if it applied to all orders instead of less than 20%? What if my accountant didn’t notice the discrepancy and the problem would only have been discovered after six months? A year? What if a client were to rely on the same code for a webshop with hundreds of thousands of orders each month?
Suddenly spending extra time on automated unit, functional and mutation testing sounds like a great investment. Now if you’ll excuse me, I’ve got some more tests to write for Commerce...