On my article on Caching Guidelines for MODX Revolution, Marc Hinse asked a very good question that I don't want to leave unanswered.. however the answer is slightly longer than a simple comment, so this follow-up tries to go into what he's asking.

This was his comment:

Thanks Mark for this guide ;-)

You asked for errors or I think new challenges in building performant cache constructions. What about:

[[*tv:is=`1`:then=`[[$chunk1]]`:else=`[[chunk2]]`*]]

while being chunk1/chunk2 a piece of code like:

[[*pagetitle]]-[[$chunk1-1]]

and chunk1-1

something like

[[!snippet1]]

snippet1 (you expected it) something like

return $_GET['foo']

I think you get what I mean. How is the processing order? How can I be sure to only cache what I want? The Wayfinder example gets a lot of users back on the ground, but what about complex if/then template/chunk structures? A guide with "Look deep nested uncached -> every parent will be uncached" or similar formular would be helpful ;-)

Cheers from germany!
Marc

A very good question, and now that the basic caching has been cleared up, I suppose we can move on and dive a bit deeper. The source-order recursive parsing makes it fairly complicated to explain (and I'm not sure if I know all the ins and outs myself either), but I'll give it a shot.

Parser Basics & Tag Order

From what I know, the parser collects all tags and starts executing them from start to finish, and inside out - so first the tag nested all the way down, and the outer most tags last. Everytime it has processed something, it would check that output for any tags and execute those - again "inside out". This is for all cached tags. It waits with processing uncached tags until all cached tags have been executed. So you could say this is the parsing order:

  1. Nested cached tag
  2. Cached tag
  3. Nested uncached tag
  4. Uncached tag
To make it all make more sense, I like to envision it that tags are put in some sort of queue, with the priority being set to any of the 1-4 above where 1 is top priority, and 4 is lowest priority. It's also sorted on source order, so a tag that appears in the header is executed before a tag in the footer - also based on the priority.

Furthermore it keeps this queue and parsing order in different levels. When a tag is found and executed its return value is immediately checked for tags, which are then added to a queue and executed. Any return value of those tags will be checked for tags which are added to a queue and executed. Any return valueof those tags.. well, you get the idea right? It's completely recursive and the parsing order is in effect per level (or scope, or context, whatever you want to call that).

Parsing Tags

Everytime a tag is parsed, the output will be checked to see if there are any tags in there, and if it finds any, they are added to the queue according to their priority and source order. Then it goes back to that queue and executes whatever is left based on its priority and source order.

When all cached tags have been processed, the result of all that (so with all uncached tags in place) is written to the cache for future requests. In other words - any uncached tags will appear as-is in the cache file, and will be picked up and executed on every request.

In Marc's example there's another layer of complexity added with the output modifiers. I think it first looks out for "inner tags", so in your example both [[$chunk1]] and [[$chunk2]] would be parsed before [[*tv]]. In that scenario it could be the following after one round of parsing:

[[*tv:is=`1`:then=`[[*pagetitle]]-[[$chunk1-1]]`:else=`[[*pagetitle]]-[[$chunk2-1]]`]]

Then in the next "round" of finding tags and adding it to the queue, it would parse the chunks to return your uncached snippet call and to replace the pagetitle tag with the pagetitle. So by now, we have the following:

[[*tv:is=`1`:then=`Welcome to my site!-[[!snippet1]]`:else=`Welcome to my site!-[[!snippet2]]`]]

Now we see that we have a cached TV tag, with uncached snippets in there. We know that cached take priority in the queue, so at this stage we would see the output modifier being executed (as well as other cached calls of course), resulting in an output of (assuming tv = 1):

Welcome to my site!-[[!snippet1]]

At that point the cache would be written to file and parsed on every subsequent request. So even when nesting, as long as the outer elements are cached it should only save uncached elements for parsing on every request. This also is true when using an uncached placeholder in a snippet tpl chunk - they would be left in the rest of the output when writing to cache, for processing on every request.

Parsing Order & Conditional Logic

Basically, if you have conditional logic using output filters (or the If snippet for that matter), the "inner" tags will be executed before the outer (conditional) tags. What that means is that when you have the following example:

[[!getNumber:is=`1`:then=`[[!FrstChildRedirect]]`:else=`[[$video-gallery]]`? &parentId=`[[*id]]`]]]]

While this seems fine, it will parse inside-out, meaning that the FirstChildRedirect snippet will always be executed, and you end up being redirected to the first child, and the conditional logic is moot.

Now, knowing that we can nest tags anywhere, we can trick the parser somewhat, and put another pair of double brackets around the conditional logic, and remove them from within the conditional logic:

[[[[!getNumber:is=`1`:then=`!irstChildRedirect`:else=`$video-gallery`? &parentId=`[[*id]]`]]]]

This will execute the conditional logic first, and will return either "!FirstChildRedirect" or "$video-gallery", which, thanks to the outer pair of two brackets, will form a valid tag which will then get executed after the conditional logic takes place.

This doesn't just stop at the name of the tag to load though, here's an example (from Jason "opengeek" Coward) illustrating just what you could do know that you learned this trick.. First we have a chunk (let's call it "sidebox"), which has the following contents:

<div class="sidebox">
<h1>[[+title]]</h1>
[[[[+element]][[+properties]]]]
</div>

And how it's being used (again, assuming the name "sidebox"):

[[$sidebox?
    &title=`[[getValue? &class=`modResource` &field=`menutitle` &where=`{"id":[[UltimateParent]]}`]]`
    &element=`Wayfinder`
    &properties=`@SidebarMenu? &startId=`[[UltimateParent]]` &level=`1``
]]

Here's an article to read if you're new to chunk parameters and placeholders, but if you know that - and now know this trick - you may get the feeling there's some powerful applications for stufff like that..

Notes

This is the caching as I understand it at this point of time... There's probably some subtleties missing, but beyond that I'm quite confident it is correct and all. ;) I'm glad that Jason Corward was willing to give this article a read before hitting the final "Publish" button though, and pointing out a few important things. Thanks Jason!

This really takes my knowledge of MODX to the next level. Thanks Mark!!!

One question: in the getPage documentation, it's instructed to always call getPage uncached. Can you explain why this is, and the ramifications of ignoring it? Because I notice getPage can be slooooow and it'd be really nice to know when to break the rules re: calling it uncached ;)

I'm sorry I must've hit the button twice. Not trying to spam, I swear! LOL :)

(Removed your double post :P)

getPage uses a "page" URL parameter to display the proper resources (eg only resource 5-10 when on page 2). This is exactly what caching guideline #3 tells you: http://www.markhamstra.com/modx-blog/2011/10/caching-guidelines-for-modx-revolution/

If you call getPage cached you will not be able of seeing different pages... which means you could've just as well not used it in the first place :)

If getPage is slow, it's most likely the underlying snippet (usually getResources). Check things like what TVs are included (use &includeTVList) and make sure that if you use TVs that they aren't being processed unless really necessary.

&includeTVList is the tip of the day!! That's a (relatively) new property that I didn't know about... until now ;)

Thanks!!

marc (fourroses) marc (fourroses)
Jul 26, 2012 at 03:26 PM

&includeTVList is really a lifesaver, i did know about it but didnt seem to use it on my latest rev site.

it made my website be 2x fast. (still a bit slow but do-able)

I don't get the process thingy, do i need to use that too for the TV's ?

processTVList=`tvname1,tvname2` ?

Hey Marc,

You only need to use &processTVs or &processTVList if you need to run the TV value through the output type defined for the TV - in other words if you changed the output type to something else than "default" and you want the processed value of that output type, then you need to process it.

Processing TVs is a costly operation and I tend to recommend against using non-default output types and using &processTVs.

Up until a recent version of getResources you needed to process the TV in order to get the complete file path to an image using a media source, but that has been corrected.

Comments are closed :(

While I would prefer to keep comments open indefinitely, the amount of spam that old articles attract is becoming a strain to keep up with and I can't always answer questions about ancient blog postings. If you have valuable feedback or important questions, please feel free to get in touch.