While working on adding a set of permissions to the ContentBlocks component, Isaac was wondering if there were any good ways of testing them out. The permission system in MODX is not very popular due to its (perceived?) complexity and the need to constantly flush permissions to see a result.

yeah, and the only reliable way i’ve found to test permissions in modx is to have one browser session with the admin changing permissions, and a separate session with the test user. and EVERY SINGLE TIME YOU CHANGE A PERMISSION, you log the test user out and back in ~ Isaac Niebeling

We had a bit of a conversation following that about how the ACLs work.

I had been using the Manage > Flush Your Permissions for refreshing permissions on my limited admin account. I simply made sure the limited user could access that, and that made my testing a lot easier than logging out and in again over and over.

That already seems like one step up, but can we do better?

It turns out that MODX has this thing called Stale Sessions. They've been introduced way back in 2.2.1 and their primary goal is to have a user refresh the "attributes" in its session (which contains a cache of many things about the user, including settings and permissions) the next time it requests a page.

I vaguely remembered that being there, but wasn't quite sure how it worked so I did a search across the MODX codebase to find instances of it, to see where it was called and how it affected the sessions. And that's when I stumbled across the processor for the Manage > Flush Your Permissions menu item.

As the name indicates, the Flush Your Permissions action will to flush the permissions of the user that is logged in. It even warns you that it doesn't affect other user sessions.

But that's a big lie.

The Flush Your Permissions processor isn't restricted to a single user. It tells all users to reload their permissions, through the Stale Sessions feature introduced in 2.2. Seriously, just check the source code if you don't believe me, there's no mention of the current user.

Following this discovery I did some testing with my new set of permissions for ContentBlocks, and indeed flushing the permissions for a full admin user also affected my limited admin user in a different browser.


The commit that introduced this behaviour back in 2012 mentions that it doesn't yet affect anonymous sessions, as those don't have a user record that instructs them to refresh their session, so that's a bit of a caveat if you're working with ACLs for site visitors. But when tweaking client access to the manager, using Flush Your Permissions instead of logging out and back in is going to save you a lot of time.

Can we do better?

So aside from this being a time saver, is there anything else we can do with this information?

Well, one of the reasons I started looking into this was to see if there might be a way to make MODX automatically flush relevant sessions when access policies are changed or added to a user group.

I had concerns about practicality because it could take a long time on large sites, but some tests of the flush permissions feature seem to indicate that it's only a few hundred milliseconds even on sites with thousands of users. That seems totally acceptable for the convenience of not having to flush permissions manually in a lot of cases.

It turns out, the core is already doing this, but only when creating a user group using the access wizard. Not when updating a user group, or when a policy is added, updated or removed.

Sounds like some pull requests may be in order.

One of the areas I'm focusing on for Commerce is building a webshop solution that is testable. With a suite of automated unit, functional, and mutational tests, you can make sure that code works as expected at all times. I've written about this before.

Opponents of testing often see it as something that takes a lot of time and effort (which is true), and that makes it expensive to do. An immediate return on investment is also rare, so automated tests are often pushed back or just skipped entirely. While I’ve been convinced about the merit of testing, I too fall into this trap and have yet to achieve the magical 100% test coverage on any sizeable project.

Unfortunately, especially when it comes to e-commerce, the cost of not properly testing every aspect of the code can be very real.

Take the modmore.com billing engine for example. It has been working pretty well for the past three years. I’ve done plenty of manual testing on my local environment to make sure it works as expected when making changes. I just haven't made the time to add unit tests to it, partly because not all the code is as easy to test.

Last weekend I was working on getting the documents in order for the first quarter taxes and shipped it all off to my accountant. Pretty satisfied with the quarterly results, I went on my merry way to do other stuff that didn’t involve looking at numbers and invoices. Until I got an email back from my accountant today about a discrepancy he noticed.

It turns out that there was an issue fetching the current VAT rates for the EU member states from a remote service. Previously we were fetching that over HTTP, but the service started requiring HTTPS, which resulted in a failed connection and the error handling wasn’t sufficient to cause alarms to go off about a critical issue.

Because of this issue, 44 orders which should have incurred 19-21% VAT, ended up being charged a 0% rate. As I still have to pay the VAT that should have been charged, that means I have to pay over €400 in VAT out of own pocket.

Look, it's an info box about VAT!

If you aren’t sure what VAT is or how it works, here’s a high level summary of how it works, at least in the Netherlands.

A business charges you the appropriate VAT rate. The rate depends on the type of product and the country. The standard rates in the EU range between 17% (Luxembourg) and 27% (Hungary), but there are reduced rates for special types of goods and services. When selling to customers the price mentioned is typically inclusive VAT, in business-to-business prices are usually exclusive VAT; this may vary per country though.

At the end of the quarter, the business owner or accountant tallies up the amount of VAT they had to charge their clients, and deduct from that the amount of VAT their suppliers charged them. The resulting number is the VAT they have to pay to the tax office. This is why it’s called Value Added Tax; the tax is paid over the value you’ve added (difference between cost and sale price).

At modmore we don't have a lot of direct costs for selling our products, so we end up paying most of the charged VAT to the tax office.

Luckily this only affected a relatively small slice of our orders and overall revenue from the past quarter. It won't put us out of business or anything, it just lowers the profit margins for those orders by a considerable amount.

But what would have happened if we had ten times as many clients, and we were talking about a €4000 charge instead? What if it applied to all orders instead of less than 20%? What if my accountant didn’t notice the discrepancy and the problem would only have been discovered after six months? A year? What if a client were to rely on the same code for a webshop with hundreds of thousands of orders each month?

Suddenly spending extra time on automated unit, functional and mutation testing sounds like a great investment. Now if you’ll excuse me, I’ve got some more tests to write for Commerce...

Built out of the need to standardise, Alpacka provides common utilities for developers building extras for MODX. It contains common utilities, varying from parsing chunks to context specific settings and sanitising file names.

Read the full article »

I'm one of the six Core Integrators that have been tasked with merging pull requests on the official MODX repository. Lately I've been slacking a bit (no pun intended) as there's been so much happening at modmore, but this month I'm going to do my best to catch up.

Inspired by yearly initiatives like 24 Pull Requests, I'm doing my own variation on that this year. Rather than contributing a pull request every day until Christmas, I'm going to try to merge (at least!) one pull request every day, starting today. There were 66 open pull requests this morning, so there is plenty of good work by others that deserves to be included in upcoming releases of MODX. Theoretically, in 24 days we'll be down to 42, but maybe some of the other integrators like the idea and help bring it down further, hint hint!

In this blog post, I'll keep a list of which pull requests have been merged on what day. I've lasted updated this post on 2013-07-26 14:06:14 - if that's more than a day ago, be sure to hold me accountable via twitter.

December 1st, 2015 (Tuesday)

Got really excited to start!

Down to 62! 63 because JP sent another pull request!

December 2nd, 2015 (Wednesday)

64 pull requests total today, it's going to be hard to get the number down if we keep getting this much new contributions.

  • #12795: Fix uberbar on desktop - fixed an issue introduced yesterday in #12776 where the uberbar wouldn't show up in 2.5. Fix by JP.
  • #12773: Improve tree usability - thanks to this improvement by Lukas you now have a much larger click target to expand a container in 2.5. Clicking the arrow, icon or empty space will expand or collapse a container, while clicking the name will open it for editing.
  • #12747: Add resource information to OnResourceAutoPublish event - Chris proposed adding some extra information to the OnResourceAutoPublish event, providing info on what resources were affected, which will be in 2.5.

Down to 61 open pull requests!

December 3rd, 2015 (Thursday)

Took a little detour to pull requests targeting 2.4.3 today.

Down to 58. Also sent two pull requests fixing issues introduced in the mobile-friendly improvements in #12776: #12798 (ensuring compatibility with custom manager themes and #12799 (fixing scrolling on large displays - whoops), so back up to 60 open pull requests total now.

December 4th, 2015 (Friday)

December 5th, 2015 (Saturday)

Also sent a really important pull request (merged by Mike within the hour), and a second pull request related to the travis tests as well.

December 6th, 2015 (Sunday)

Oops... missed a day :(

December 7th, 2015 (Monday)

To make it up with you for missing a day, here's a couple more merges into 2.x for the 2.5 release:

After a week of this project, there's now 58 open pull requests. I'm very pleased to see the constant flow of contributions that are being sent to MODX from a range of contributors. At the start of December there were 66 open pull requests, and despite merging 16 of those myself (and several more by other integrators) we've only gone down 8 pull requests in a week. That's equally sad (I was hoping for a larger decline after 1-2hrs of work every day!), but also amazing that we can count on so many contributions from people that genuinely make MODX better.

I've been trying to merge more than one per day as we have so many great contributions pending, and it's really quite a bit of work to stay on top of things. Not every pull request is always ready to get merged, which means that before you get to one that is ready to go you've already spent some time going through the list, reviewing code, figuring out if it's a bug fix or improvement (which can be a very fuzzy line!) to know what branch to merge to, checking for signed CLAs and proper processes, before even getting to the git magic, testing and determining if it's merge-ready. If you'd like to help out make it easier on the volunteering integrations, you can help out with code reviews, making sure people use the right template in pull request (or at the very least provide all relevant information so we don't have to ask) and of course testing out pull requests (especially larger ones) and commenting your findings. That will help integrators spend more time on actually integrating, which will help MODX become better more quickly.

At the moment, it looks like both 2.4.3 and 5.3.0-rc1 could be coming before the end of the year, but there's a lot of pending improvements that would be great to get in. With a bit of help, I'm sure we can get a lot of those merged in to make some really great releases.

December 8th, 2015 (Tuesday)

December 9th, 2015 (Wednesday)

December 10th, 2015 (Thursday)

Down to 53 open pull requests, making progress!

More to come tomorrow!