Hello! Welcome to my humble web presence. I'm Mark Hamstra, 28 years young, wearer of many hats at modmore, resident of Leeuwarden, the Netherlands. Most of my time is spent building and maintaining awesome extras and tools at modmore, but I also love gaming, cooking, and my dogs. More about me.

This site is where I share thoughts, cool projects and other oddities related to MODX, xPDO and ExtJS. I also write about MODX regularly over at MODX.today. Sometimes I post three blogs in a week, sometimes there's nothing new here in a year. Read a random article.


On my article on Caching Guidelines for MODX Revolution, Marc Hinse asked a very good question that I don't want to leave unanswered.. however the answer is slightly longer than a simple comment, so this follow-up tries to go into what he's asking.

This was his comment:

A very good question, and now that the basic caching has been cleared up, I suppose we can move on and dive a bit deeper. The source-order recursive parsing makes it fairly complicated to explain (and I'm not sure if I know all the ins and outs myself either), but I'll give it a shot.

Parser Basics & Tag Order

From what I know, the parser collects all tags and starts executing them from start to finish, and inside out - so first the tag nested all the way down, and the outer most tags last. Everytime it has processed something, it would check that output for any tags and execute those - again "inside out". This is for all cached tags. It waits with processing uncached tags until all cached tags have been executed. So you could say this is the parsing order:

  1. Nested cached tag
  2. Cached tag
  3. Nested uncached tag
  4. Uncached tag
To make it all make more sense, I like to envision it that tags are put in some sort of queue, with the priority being set to any of the 1-4 above where 1 is top priority, and 4 is lowest priority. It's also sorted on source order, so a tag that appears in the header is executed before a tag in the footer - also based on the priority.

Furthermore it keeps this queue and parsing order in different levels. When a tag is found and executed its return value is immediately checked for tags, which are then added to a queue and executed. Any return value of those tags will be checked for tags which are added to a queue and executed. Any return valueof those tags.. well, you get the idea right? It's completely recursive and the parsing order is in effect per level (or scope, or context, whatever you want to call that).

Parsing Tags

Everytime a tag is parsed, the output will be checked to see if there are any tags in there, and if it finds any, they are added to the queue according to their priority and source order. Then it goes back to that queue and executes whatever is left based on its priority and source order.

When all cached tags have been processed, the result of all that (so with all uncached tags in place) is written to the cache for future requests. In other words - any uncached tags will appear as-is in the cache file, and will be picked up and executed on every request.

In Marc's example there's another layer of complexity added with the output modifiers. I think it first looks out for "inner tags", so in your example both [[$chunk1]] and [[$chunk2]] would be parsed before [[*tv]]. In that scenario it could be the following after one round of parsing:

Then in the next "round" of finding tags and adding it to the queue, it would parse the chunks to return your uncached snippet call and to replace the pagetitle tag with the pagetitle. So by now, we have the following:

Now we see that we have a cached TV tag, with uncached snippets in there. We know that cached take priority in the queue, so at this stage we would see the output modifier being executed (as well as other cached calls of course), resulting in an output of (assuming tv = 1):

At that point the cache would be written to file and parsed on every subsequent request. So even when nesting, as long as the outer elements are cached it should only save uncached elements for parsing on every request. This also is true when using an uncached placeholder in a snippet tpl chunk - they would be left in the rest of the output when writing to cache, for processing on every request.

Parsing Order & Conditional Logic

Basically, if you have conditional logic using output filters (or the If snippet for that matter), the "inner" tags will be executed before the outer (conditional) tags. What that means is that when you have the following example:

While this seems fine, it will parse inside-out, meaning that the FirstChildRedirect snippet will always be executed, and you end up being redirected to the first child, and the conditional logic is moot.

Now, knowing that we can nest tags anywhere, we can trick the parser somewhat, and put another pair of double brackets around the conditional logic, and remove them from within the conditional logic:

This will execute the conditional logic first, and will return either "!FirstChildRedirect" or "$video-gallery", which, thanks to the outer pair of two brackets, will form a valid tag which will then get executed after the conditional logic takes place.

This doesn't just stop at the name of the tag to load though, here's an example (from Jason "opengeek" Coward) illustrating just what you could do know that you learned this trick.. First we have a chunk (let's call it "sidebox"), which has the following contents:

And how it's being used (again, assuming the name "sidebox"):

Here's an article to read if you're new to chunk parameters and placeholders, but if you know that - and now know this trick - you may get the feeling there's some powerful applications for stufff like that..

Notes

This is the caching as I understand it at this point of time... There's probably some subtleties missing, but beyond that I'm quite confident it is correct and all. ;) I'm glad that Jason Corward was willing to give this article a read before hitting the final "Publish" button though, and pointing out a few important things. Thanks Jason!

I develop a number of MODX Extras, both open source and bespoke code, which I typically build on the latest stable release. Using a build script the code is transformed into a transport package, which by default ends up in "core/packages" of the MODX installation. This short how-to will explain you how to adjust your build script and project setup and stay in control of your packages.

My Localhost

Now, before I go into detail on how to make this ridiculously easy change, I want to briefly discuss my project set up as that is of course a vital part of staying in control of your data. I've found this works for me, but of course different developers will have different strategies.

All my files are accessible via http://localhost/, ie they are in the "www" folder of WAMP. In there I have a number of MODX installations of various versions. These are for development & testing of stuff I develop and act as a source of documentation as well. On the same level (so immediately under "root") I have a folder called "projects". So to recap, we've got this:

  • www
    • modx-1.0.5
    • modx-2.0.8
    • <...>
    • modx-2.1.3
    • modx-git
    • projects

This allows me to keep my MODX installs totally separate from my project files. Using system settings I then "link" my project into the MODX install I'm developing that project in, and we're set (possible a more extensive blog post on that later!). If I break a MODX install or otherwise screw up, I can simply delete all the files of that install and start over.

In my projects directory I have one directory per project, which in 99% of the cases is also a Git Repository (Like VersionX2, or HandyMan). In there I have a _build, core and assets directory with the right contents - I assume you're familiar with that if you're reading this article.

The Problem..

When I develop a lot of addons, or make a lot of builds, all these are added to the core/packages directory of the MODX install I linked my package to (through a config.core.php file). May be fine if you then want to install the package there, but I don't to keep my links pointing to my own files. And after some time it gets tough to find that one package in between one of the other 100's..

What I want is store packages in a _packages dir in my project Git repository so I instantly know where a package should be (in the projects dir) and as an added bonus the package is also uploaded to Git - giving an easy way to distribute it for people that do not know how / don't want to run the build but do want to check out development.

Aha! The Solution!

In a random moment where I was again browsing into the MODX install dir to get a package, I decided to check out the modPackageBuilder class API docs.

As often is the case, this problem had already been foreseen and there is a very easy way to change the directory the files and transport zip are outputted to. It's so easy, it would have fit in a tweet - but how much fun would that be? So that's why you''ve been reading this blog.

Oh you want to know how now you got this far?

Okay.

Let's say this is part of our regular build script:

That's where it instantiates the modPackageBuilder class. So that's where we'll want to tell it where to output to. We'll be using a public property of the modPackageBuilder for that: $directory. Hah, bet you didn't see that coming!

So let's just make that single small adjustment to our build script that you've been reading this for..

We're simple traversing up two directories (first we're in _build, then we go up to the root, and lastly add _packages to it, putting it in our all new special directory :)

Now hold on.. what about all those files?

Yes - the package builder also creates a lot of files (that's what gets zipped up, actually) and we probably don't feel the need to upload all that to our git repository. To fix that, we'll create a real simple .gitignore file and put that in our _packages directory.

It will ignore all folders, but not files in the root (our transport package).

And that's it folks. :) Thanks for the masterminds Shaun & Jason for thinking of stuff like this ahead of time!

This article has been translated to Russian as well. Know of any other translations? Let me know!

MODX has very granular caching possibilities. Where Evo limits caching to just snippets and has other limits in terms of embedding tags in tags, MODX Revolution does not have these limits.

In MODX Revolution you can cache / uncache any tag you may use. That includes snippets, but also setting tags, placeholders, chunks and even lexicon strings. This article will talk you through some reasons when to call what cached in MODX Revolution (but it will save you most of the inner workings).

It is important to realize that, unless you're doing real advanced stuff and have overriden the core caching, the cache will be cleared when you: save a resource, element and to some extent settings. So basically, whenever your content or structure is changed - your cache will be updated to reflect that.

How do I tell MODX to (not) cache a certain tag?

Simply add in the exclamation mark (!) in front of the token. The token is a dollar sign ($) for Chunks, a plus sign (+) for placeholders, a percentage sign (%) for lexicons and an asterix (*) for resource fields and template variables. Snippets do not have a token.

Examples cached:

[[Wayfinder]]
[[$chunkname]]

Example uncached:

[[!Wayfinder]]
[[!$chunkname]]

Four Guidelines for Caching Strategies

  1. Cache everything you can.
    In general you will want to cache as much as you can - why should MODX retrieve the content from the database on every request, when it hasn't changed? While this may seem logical (and there is rarely any question on wether to use resource fields cached or not), the same applies to other tags. If the output of a snippet doesn't change on every request - why should MODX parse the snippet on every request? It shouldn't.
  2. User-specific output needs to be uncached.
    Anything that outputs user specific information will need to be uncached. Simple - any request could be made by different users, so it wouldn't do any good to load user-specific information from cache. This applies to snippets that interact with the user system (the Login package, a Wishlist component but also Quip for commenting), but also the somewhat less known [[!+modx.user.id]] placeholder.
  3. Content related to URL parameters or POST data needs be uncached.
    If you're using a form which displays information about what was posted, you don't want that to be fixed and cached. So call that uncached as well. If your output is different depending on (non-id/q) URL parameters (for example a search results page), you will want to make sure the related snippet is not cached.
  4. Use custom caching when needed.
    If data (in a snippet) changes before the cache has been cleared, for example if data is retrieved from an external XML source that changes independantly to your MODX Content, you will want to use a custom caching method such as the getCache snippet or write your own caching within the snippet code.

Exceptions

Of course there are exceptions...actually, I could only come up with one so far: (Updated October 26th, came up with another one!)

  • site_url, http_host and related system settings may be called uncached. If you have a certain setup in which one resource+template combination may server multiple urls, you will want to make sure the related system settings are called uncached. One common application for this is your <base href="[[!++site_url]]"> tag for friendly URLs: you will want to be sure your assets are served from the same domain as the user visited. Wether you should force www/no www through htaccess is a whole different discussion, but that is very much related to this and a common cause for issues.
  • Snippets that (may) redirect or forward elsewhere will usually need to be uncached as the result may be empty, but the processing of the snippet does the work to redirect the user.

Common Errors

  1. Calling the "if" snippet uncached, when the subject is a resource field. Most likely caused by the documentation having uncached examples, a lot of people mis-use the If snippet by not caching it when the subject is static for the cache lifetime.
  2. Calling the "Wayfinder" snippet uncached. Please don't! Wayfinder especially can take a good performance hit on your website with larger menus. The only valid reason I can come up with to call Wayfinder uncached is when the output is user specific (member-only area, for example).
  3. Uncaching placeholders in chunks that are used as templates in snippets. Uncached placeholders are processed at the last possible time, and calling them uncached in a template will leave them in tact and they wont get replaced by the snippet, but instead by the core parser long after the snippet is done. (This behavior has seen some changes/bugs in the 2.1.x releases, so it could be that this is different in the latest release - I haven't actually checked in much detail recently.)

What are the errors you have made that you spotted in the above lists? Or do you have other caching tactics or questions? Share below!

Based on a comment from Marc Hinse, a follow-up to this post has been written discussing how nested caching works in MODX Revolution.


In some languages (the Dutch one for sure) on smaller monitors/ browsers (1024px), the top menu in the MODX Manager overflows, pushing the last two or three items down a row, rendering it useless.

I've created a simple plugin to fix that called TopMenuFix, and has been added to the Extras repository

And with simple, I mean really simple. As the menu is simply styled with CSS, all it has to do is override those styles on manager pages. We'll be using the onManagerPageInit event to put some style definitions on the page automatically.

This two line plugin calls the modX::regClientStartupHTMLBlock method, which sounds really complicated but just injects some html into the header of the page. In this case, that is simply this:

And that's all there is to it.

Want this too? No problem! I've wrapped it up in a package using the PackMan addon and it is available from the Package Manager It's called "TopMenuFix". You can also find it on the MODX Website.

The source is on Github and with any bugs or feature requests (I don't expect them.. but.. okay) please also post on the TopMenuFix Repository Issues section.

The manager on my site handles a couple of site. First of all it is, obviously, where I write these articles. Then there was a Dutch translation in the works in a seperate context, as well as another one for the HandyMan website. Every now and then there's somebody who helps me out with these. There's someone translating for me, there's some people who help out on the HandyMan website and then a few more that need access to my main site.

At first I just gave them all full out access, but to provide a little more customization and security for myself I decided to set up a couple of user groups with just the permissions people will need. One of these involved the elements such as chunks and templates. Users working on the HandyMan site don't need access to my main site template or the other way around.

So I set up some access levels. I decided to do it quickly, and set up one "MH Admin" group which has access to the main site context and its elements. That would mean any other user groups (minus the main Administrator one which already had access to everything) only has access to the others. This is, in case you don't know yet, because the security system locks out user groups that do not have the permission in case you set it explicitely. So first everyone's allowed - but when you give one user group explicitely access to it, it will block all the rest. No problem! That's exactly what we want.

Fast forward a month.

It is now June, and a week ago I updated MODX to 2.1.1-pl, and set up a new context related to the development of a new website - this one! Fresh content, fresh layout etc. All good. But suddenly, a week after setting that up, while browsing the site it seems some pages have eaten their template! Nothing shows up, but just the content.

Theory 1: corrupted cache

Okay... odd. Now let's think. What could've caused this? Let's just refresh the cache to make sure we're not just dealing with an incidental corrupt cache or anything. Cache cleared, back to the website, all good. Alright then - back to work!

Theory 2: context settings

Now about an hour later I end up watching my website again, and guess what? Same problem all over again! I didn't settle with clearing the cache this time, but instead kept browsing around in order to see if I can replicate it on the website. At one point I just totally couldn't find it and decided to make a note about it on my homepage just in case it would happen again. So I make the modifications and press the "View" button to check how it looks. But wait! The "View" button doesn't work, but attempts to open a javascript code as webpage. So I went to check the site_url setting in the context, to find it wasn't there.

I never set them on the "web" context for the simple reason that it wasn't needed. The web context runs just like it would be the only one. But, or so I guessed, the fact that I recently added a development context for my new site without settings either (did not want it public yet) might have been the problem. Let's set some settings such as site_url and try again. Yay, it's back up!

... or is it?

What actually was the problem

The problem was that it would not be parsing my templates, chunks and so on. My elements. The same ones that I protected a month earlier in the back-end to make sure only the people I want to be able of accessing could actually see them. After receiving another text message asking if I made my website "Mobile friendly" (yes, that's you Taco!!) it seemed the problem was still there. Aaargh!

Figured it out yet? If you didn't, don't worry - at this stage I didn't either. If you did, where were you when I needed you?! :P

If you remember (or scroll back up) you'll see how I explicitely set my MH Admin user group to be able of accessing these specific categories. What that meant is that any other user groups would not. But that's what we want right? Actually - there's one flaw in there, which may stem from still being used to the Evolution security systems in which you had the division between web and manager users. In Revolution, you do not but instead they can be used for both or just one of them. The user group that caused this was the (anonymous) one - a user group representing people that are not logged in. Site visitors. People who will want to see my templates and other elements.

So because I only explicitely allowed my MH Admin user group to access these elements, they were automatically hidden from other user groups.

Including valuable visitors.

The fix was easy - just give the (anonymous) user group access to these elements as well, using the access control of "Load, list and view". Flush permissions, clear cache and back to the site. Test in three browsers and ask three other people from all over the country to do the same. I am now confident to say the problem has been fixed.

So that's all?

Now there's one more thing that I didn't explain yet, which has seriously made this fault hard to track down. I know this system quite well, and this had me puzzled for days.

One word: Caching.

My site is cached as much as possible, which includes complete resource caching. So whenever a page is requested (by anyone) it is cached. This anyone is often me - whenever I change something I check it out, and click around a bit to check out comments etc. All the resources I visited were added to the cache, and these often were the ones that were most visited as well. So whenever I visit a resource while being logged in, it gets cached with my permissions.

But, and this was the problem, when a resource was not cached it would use the anonymous user group permission and MODX would refuse to display the templates and other elements. As a result it would just dump the content with no errors (it's intended behavior, not something failing) or any clues.

 

So that's what kept me busy over the past period. And I hope this article can prevent you from having to do the same. :)

It's possible that you don't know what the MODX Partner Program is, though it has been getting more and more attention lately. There's not a simple explanation to that as it becomes as much as you put into it, and much of it is constantly in development to offer more benefits to the Partners as well as MODX.

In general it can (and has been) said that it is aimed at developing MODX and your own business by offering resources, a private discussion board and closer contact with the core developers. Depending on the size of your company, there are three levels you can choose from: Professional, Agency and Premier. Where Professional is the cheapest (at $ 500 per year at launch, but as of June $ 50 per month) level, it does come with the advantages mentioned above and also includes a complimentary license for PhpStorm and New Relic, a Partner Listing valued at $ 100 per month and of course access to the partner board. Moving up the levels, the price increases (to $ 5 000 and $ 10 000 for Agency and Premier partners respectively) as well as the benefits such as 5/10 hours of core consults, a MODX integrated helpdesk suite, sales leads and much, much more.

In April I decided to take the plunge, and sign up to become a Professional Partner and see if it is worth it. As you may know, I am a freelance developer and even though the business is running well, at that point $ 500 was still a lot of cash and making the decision to try it out didn't come easy.

Taking the plunge

The first few hours of being an official Professional MODX Partner were mainly spent visiting the private forums, which are packed with other skilled MODXers discussing topics varying from favorite development IDE, development and quality assurance workflows, what they were working on and so on and so on. While the public forums are amazing as it is, there is something about the Partner boards that the public ones just don't have. For example when looking at help requests, it is easier to share bespoke code as the forums are not indexed or otherwise publicized, there is a less demanding atmosphere and they don't just deal with how to use basic MODX properties, but go more into depth challanging the system and the people using it.

Furthermore - there's the scoop. Lots of it. Think about a first look & test run at the new forums, a topic explaining what has changed in 2.1 before the public launch and more. It may be hard to put the value of such forums into money, but for me personally there is a great wealth of information there and lots of dense-packed knowledge from the core team as well as other partners.

A second point that has really attracted me to take the plunge was the MODX Certification. At time of writing this has not yet been fully developed, but it is expected in the third quarter of this year and will provide a huge advantage to show credibility and knowledge. Especially for the smaller companies or freelancers like myself, credibility is often one of the bigger challanges in doing business, and this will be a huge help to get that obstacle out of the way.

Three months later - was it worth it?

Short story: yes.

In the past months I have not only learned a lot about MODX, but also about different tools for development and email marketing, read about the differences between MODX and another system in detail and my business is running like crazy. Joining and actively participating in the program has introduced me to new people and has led to new inquiries - a big bunch of them from other Partners looking for people with my speciality. In June I was also contacted for a promotional activity, which you can see the result from on the Partner Program page.

Everyone will have a different idea of the partner program, and different benefits attract different people. There's designers, developers and decision makers involved but they have one thing in common: they support MODX and have been willing to contribute towards further development of it as a system as well as a company.