One of the areas I'm focusing on for Commerce is building a webshop solution that is testable. With a suite of automated unit, functional, and mutational tests, you can make sure that code works as expected at all times. I've written about this before.
Opponents of testing often see it as something that takes a lot of time and effort (which is true), and that makes it expensive to do. An immediate return on investment is also rare, so automated tests are often pushed back or just skipped entirely. While I’ve been convinced about the merit of testing, I too fall into this trap and have yet to achieve the magical 100% test coverage on any sizeable project.
Unfortunately, especially when it comes to e-commerce, the cost of not properly testing every aspect of the code can be very real.
Take the modmore.com billing engine for example. It has been working pretty well for the past three years. I’ve done plenty of manual testing on my local environment to make sure it works as expected when making changes. I just haven't made the time to add unit tests to it, partly because not all the code is as easy to test.
Last weekend I was working on getting the documents in order for the first quarter taxes and shipped it all off to my accountant. Pretty satisfied with the quarterly results, I went on my merry way to do other stuff that didn’t involve looking at numbers and invoices. Until I got an email back from my accountant today about a discrepancy he noticed.
It turns out that there was an issue fetching the current VAT rates for the EU member states from a remote service. Previously we were fetching that over HTTP, but the service started requiring HTTPS, which resulted in a failed connection and the error handling wasn’t sufficient to cause alarms to go off about a critical issue.
Because of this issue, 44 orders which should have incurred 19-21% VAT, ended up being charged a 0% rate. As I still have to pay the VAT that should have been charged, that means I have to pay over €400 in VAT out of own pocket.
Look, it's an info box about VAT!
If you aren’t sure what VAT is or how it works, here’s a high level summary of how it works, at least in the Netherlands.
A business charges you the appropriate VAT rate. The rate depends on the type of product and the country. The standard rates in the EU range between 17% (Luxembourg) and 27% (Hungary), but there are reduced rates for special types of goods and services. When selling to customers the price mentioned is typically inclusive VAT, in business-to-business prices are usually exclusive VAT; this may vary per country though.
At the end of the quarter, the business owner or accountant tallies up the amount of VAT they had to charge their clients, and deduct from that the amount of VAT their suppliers charged them. The resulting number is the VAT they have to pay to the tax office. This is why it’s called Value Added Tax; the tax is paid over the value you’ve added (difference between cost and sale price).
At modmore we don't have a lot of direct costs for selling our products, so we end up paying most of the charged VAT to the tax office.
Luckily this only affected a relatively small slice of our orders and overall revenue from the past quarter. It won't put us out of business or anything, it just lowers the profit margins for those orders by a considerable amount.
But what would have happened if we had ten times as many clients, and we were talking about a €4000 charge instead? What if it applied to all orders instead of less than 20%? What if my accountant didn’t notice the discrepancy and the problem would only have been discovered after six months? A year? What if a client were to rely on the same code for a webshop with hundreds of thousands of orders each month?
Suddenly spending extra time on automated unit, functional and mutation testing sounds like a great investment. Now if you’ll excuse me, I’ve got some more tests to write for Commerce...
And very sad about taxes. And very interesting about testing. Could you share you experience about testing (unit and others) in MODX and for MODX components? Not in comments, maybe, but as idea for the next blog post ;)
That's a good idea, and on my list! :)
We're primarily using PHPUnit, and continuous testing with CircleCI. The hardest part I've found with continuous testing is getting it to bootstrap MODX as we need that installed in our tests.
In Commerce we've also got some PHPUnit tests that more resemble functional tests than unit tests (basically mocking the entire order creation rather than just testing one tiny function), so that's the same tool.
As for mutational tests... we're not actually using those yet, and I don't think that's something you'd want to run on every single commit either because it can take quite long. I recently attended a PHP meetup where we got a great introduction of humbug (https://github.com/padraic/humbug) and how mutation tests can help you make sure your other tests are solid. I'll definitely write an article about that too once I played with it some more :)