Stand With Ukraine. Stop Putin. Stop War.

Hello! Welcome to my humble web presence. I'm Mark Hamstra, 32 years young, wearer of many hats at modmore, resident of Leeuwarden, the Netherlands. Most of my time is spent building and maintaining awesome extras and tools at modmore, but I also love gaming, cooking, and my dogs. More about me.

This site is where I share thoughts, cool projects and other oddities related to MODX, xPDO and ExtJS. I also write about MODX regularly over at MODX.today. Sometimes I post three blogs in a week, sometimes there's nothing new here in a year. Read a random article.


VersionX 1.0 was first developed in December 2010 - it was one of the first open source project I started actually. It also was the first tim that I actively developed a custom manager page, and, well, the code is terrible.

Luckily VersionX 2.0 is coming to town!

VersionX 2.0 is a complete rewrite - all that is reused is the name and the initial thoughts behind the plugin: providing a simple versioning component that allows viewing, comparing and restoring of past versions.

The rewrite is in progress, and as with any open source project - not quite as fast as I had wanted - but it's starting to get shape. Here's a quick list of what is currently in place:

  • Saving a resource causes VersionX 2 to check if any changes were made, and if so it will save a new copy to the database. If on MODX 2.1 or higher this includes Template Variable values.
  • Saving any element (templates, template variables, snippets, chunks or plugins) creates a new copy in the database.
  • Resource Versions can be seen and compared to any other version of the same resource through the VersionX Component.

Yes - it versions Elements and Resource TV values - that's new to VersionX! However, having built the versioning functionality for elements, there's not yet a grid to interact with it. The element data will be in the database if you need it, but there's no way to view or compare it from the manager.

As of writing, VersionX 2.0 is at release dev7 and I am expecting to be able of releasing it publicly at around dev15 or so. Here's the things I'm looking to implement by then:

  • Proper upgrade from VersionX 1.0. The database is set up way differently (to allow a more scalable and future-proof strategy), and there's not yet a simple upgrade procedure to re-use the data already stored. This needs to be different before a first public release.
  • Restoration of Resource versions - after all, that's what this whole thing is about.

When those two have been finished I will be releasing VersionX 2.0 as a free package available from the MODX Repository.

After that, hopefully some donations will pop up and I will be able to develop VersionX further. Adding the interface for elements, bringing new tabs to resource update screens to view the versions without leaving that page and more.. all of that and much more is on the roadmap for VersionX 2.0.

I'll admit: deep down inside I'm hurt. I've spent hours and hours trying to get this blog just right with all the right features. You may have noticed that I set up some automated archiving, tags use friendly URLS and you can even leave comments (with notifications). And guess what? You can have all that and more in literally five minutes with the all new "Articles" component for MODX 2.2. Thanks Shaun!

No - but in all seriousness, the release of this all new "Articles" plugin (formerly called "modBlog") is great news. The timing could have been better (like, not right after I deemed the blog section of my new site completed), but this is a great step leap forward for MODX, and I would really say it welcomes MODX 2.2 to the stage with a bang. 

Now, for those that have been hiding under some rocks lately - Articles is a Custom Resource Type developed by Shaun "splittingred" McCormick, MODX Senior Developer. "Custom Resource Type" is real nerdy talk for saying it's something that shows up in the side hierarchical resource tree, but doesn't neccesarily act similar to your regular Documents. If you want the nerdy description - it means you can extend the modResource class, and change its behavior (methods), customizing the user experience (controllers) and more. 

So what does it do?

Well, here's a list of some of its features:

  • Provide an overview UI when clicking the Articles resource in the resource tree, where a grid of posts are shown, as well as a huge amount of settings to configure its behavior.
  • Lists the individual articles on the front-end (powered by getPage and getResourcesTag, meaning you got it paginated and filterable on tags)
  • Creates automated archives (using friendly URLs). 
  • Generates RSS feeds.
  • Allows imports from Wordpress and resources. 
  • Has built in commenting (configurable through the Settings tab) powered by Quip. Real nifty thing here is that the Quip comments view in the manager is integrated with the individual article update screens.

Well, that pretty much sums up blogging right? I probably forgot a thing or two there, and luckily there's still some things planned for future releases for the total feature-nuts out there. 

All in all Articles really is a game changer, and the fact that it is for MODX 2.2 and up (I really should write a blog post on that, too!), I have the feeling we're about to see a big spike (a new download record, perhaps?) when 2.2-pl hits the shelves.

(And yes - this blog is still powered by my own craftmansship and hours of hard labor. And 2.1 for that matter. I'll upgrade it all someday - let's launch this baby first!)

Update 29/4: Thank you Bert Oost of OostDesign for the tip on supporting different getPage configurations!

If you've ever created custom components, chances are that you have had to create accompanying snippets to actually show the data on the front-end. That can be a lot of work, and once data starts piling up you may want to add some pagination to it. This brief article will explain the requirements that getPage puts on your coding to properly paginate custom objects.

Getting the number of Results

First of all, we will need to tell getPage what the total number of results matching any filters set in place by our snippet is. In our snippet, we can easily set up the xPDOQuery object the way we normally would and take that object to get the total amount of results. No - we don't use getCollection and count() the array - that would be an insane waste of processing time. Instead we use the little jewel called modX::getCount, which is actually extended from xPDO::getCount. This will, without retrieving all the data from the database, get the total number of results for your query. Let's show an example of how we could use this - in this example we look up the amount of resources that have a class of modWeblink or modSymLink and are published:

So that will give us a number in the $total var - but that doesn't tell getPage anything - yet! Having looked at the getPage documentation trying to figure out this part, I've overlooked the answer many times and it is surprisingly simple. The documentation tells us there's a totalVar property:

And that's our answer. We need to set a placeholder, of which the key matches the value of the totalVar property: by default that's "total", though users have the ability to change this in their getPage call, so we'll need to get the totalVar property and set a placeholder with that name. Let's update our example accordingly to set the placeholder and that's all there is to it.

Starting at the right offset

Secondly we will need to make sure our snippet understand which results to display - and that it passes this on to our xPDOQuery object. In this case all we need to do is lookout for two properties: limit and offset. Limit will be set by getPage to the &limit property, which indicates how many items to show at once. The offset will be calculated based on the "page" request and the limit per page.

If we would update our example, we could end up having this - which works perfect with getPage for pagination.

You may notice a few important things:

  • We get the total amount of results, and set the total placeholder *before* we limit the query. While the docs say it may ignore the limit property, I like to be sure.
  • We use modX::getOption() (extended from xPDO::getOption) to get the "limit" and "offset" properties from our scriptProperties array available in snippets. If it does not exist we assign default values of 10 and 0 respectively.
  • We use xPDOQuery::limit() to set the limit and offset to the query. We then pass that on to our getCollection call and be on our way to process it further and give the site visitor what they want.

Hopefully this brief article will help more developers making their snippets getPage-proof, which will make it easier for front-end developers to use one tool (getPage) for all their pagination needs. Please feel free to share your thoughts and suggestions below!

On my article on Caching Guidelines for MODX Revolution, Marc Hinse asked a very good question that I don't want to leave unanswered.. however the answer is slightly longer than a simple comment, so this follow-up tries to go into what he's asking.

This was his comment:

A very good question, and now that the basic caching has been cleared up, I suppose we can move on and dive a bit deeper. The source-order recursive parsing makes it fairly complicated to explain (and I'm not sure if I know all the ins and outs myself either), but I'll give it a shot.

Parser Basics & Tag Order

From what I know, the parser collects all tags and starts executing them from start to finish, and inside out - so first the tag nested all the way down, and the outer most tags last. Everytime it has processed something, it would check that output for any tags and execute those - again "inside out". This is for all cached tags. It waits with processing uncached tags until all cached tags have been executed. So you could say this is the parsing order:

  1. Nested cached tag
  2. Cached tag
  3. Nested uncached tag
  4. Uncached tag
To make it all make more sense, I like to envision it that tags are put in some sort of queue, with the priority being set to any of the 1-4 above where 1 is top priority, and 4 is lowest priority. It's also sorted on source order, so a tag that appears in the header is executed before a tag in the footer - also based on the priority.

Furthermore it keeps this queue and parsing order in different levels. When a tag is found and executed its return value is immediately checked for tags, which are then added to a queue and executed. Any return value of those tags will be checked for tags which are added to a queue and executed. Any return valueof those tags.. well, you get the idea right? It's completely recursive and the parsing order is in effect per level (or scope, or context, whatever you want to call that).

Parsing Tags

Everytime a tag is parsed, the output will be checked to see if there are any tags in there, and if it finds any, they are added to the queue according to their priority and source order. Then it goes back to that queue and executes whatever is left based on its priority and source order.

When all cached tags have been processed, the result of all that (so with all uncached tags in place) is written to the cache for future requests. In other words - any uncached tags will appear as-is in the cache file, and will be picked up and executed on every request.

In Marc's example there's another layer of complexity added with the output modifiers. I think it first looks out for "inner tags", so in your example both [[$chunk1]] and [[$chunk2]] would be parsed before [[*tv]]. In that scenario it could be the following after one round of parsing:

Then in the next "round" of finding tags and adding it to the queue, it would parse the chunks to return your uncached snippet call and to replace the pagetitle tag with the pagetitle. So by now, we have the following:

Now we see that we have a cached TV tag, with uncached snippets in there. We know that cached take priority in the queue, so at this stage we would see the output modifier being executed (as well as other cached calls of course), resulting in an output of (assuming tv = 1):

At that point the cache would be written to file and parsed on every subsequent request. So even when nesting, as long as the outer elements are cached it should only save uncached elements for parsing on every request. This also is true when using an uncached placeholder in a snippet tpl chunk - they would be left in the rest of the output when writing to cache, for processing on every request.

Parsing Order & Conditional Logic

Basically, if you have conditional logic using output filters (or the If snippet for that matter), the "inner" tags will be executed before the outer (conditional) tags. What that means is that when you have the following example:

While this seems fine, it will parse inside-out, meaning that the FirstChildRedirect snippet will always be executed, and you end up being redirected to the first child, and the conditional logic is moot.

Now, knowing that we can nest tags anywhere, we can trick the parser somewhat, and put another pair of double brackets around the conditional logic, and remove them from within the conditional logic:

This will execute the conditional logic first, and will return either "!FirstChildRedirect" or "$video-gallery", which, thanks to the outer pair of two brackets, will form a valid tag which will then get executed after the conditional logic takes place.

This doesn't just stop at the name of the tag to load though, here's an example (from Jason "opengeek" Coward) illustrating just what you could do know that you learned this trick.. First we have a chunk (let's call it "sidebox"), which has the following contents:

And how it's being used (again, assuming the name "sidebox"):

Here's an article to read if you're new to chunk parameters and placeholders, but if you know that - and now know this trick - you may get the feeling there's some powerful applications for stufff like that..

Notes

This is the caching as I understand it at this point of time... There's probably some subtleties missing, but beyond that I'm quite confident it is correct and all. ;) I'm glad that Jason Corward was willing to give this article a read before hitting the final "Publish" button though, and pointing out a few important things. Thanks Jason!

I develop a number of MODX Extras, both open source and bespoke code, which I typically build on the latest stable release. Using a build script the code is transformed into a transport package, which by default ends up in "core/packages" of the MODX installation. This short how-to will explain you how to adjust your build script and project setup and stay in control of your packages.

My Localhost

Now, before I go into detail on how to make this ridiculously easy change, I want to briefly discuss my project set up as that is of course a vital part of staying in control of your data. I've found this works for me, but of course different developers will have different strategies.

All my files are accessible via http://localhost/, ie they are in the "www" folder of WAMP. In there I have a number of MODX installations of various versions. These are for development & testing of stuff I develop and act as a source of documentation as well. On the same level (so immediately under "root") I have a folder called "projects". So to recap, we've got this:

  • www
    • modx-1.0.5
    • modx-2.0.8
    • <...>
    • modx-2.1.3
    • modx-git
    • projects

This allows me to keep my MODX installs totally separate from my project files. Using system settings I then "link" my project into the MODX install I'm developing that project in, and we're set (possible a more extensive blog post on that later!). If I break a MODX install or otherwise screw up, I can simply delete all the files of that install and start over.

In my projects directory I have one directory per project, which in 99% of the cases is also a Git Repository (Like VersionX2, or HandyMan). In there I have a _build, core and assets directory with the right contents - I assume you're familiar with that if you're reading this article.

The Problem..

When I develop a lot of addons, or make a lot of builds, all these are added to the core/packages directory of the MODX install I linked my package to (through a config.core.php file). May be fine if you then want to install the package there, but I don't to keep my links pointing to my own files. And after some time it gets tough to find that one package in between one of the other 100's..

What I want is store packages in a _packages dir in my project Git repository so I instantly know where a package should be (in the projects dir) and as an added bonus the package is also uploaded to Git - giving an easy way to distribute it for people that do not know how / don't want to run the build but do want to check out development.

Aha! The Solution!

In a random moment where I was again browsing into the MODX install dir to get a package, I decided to check out the modPackageBuilder class API docs.

As often is the case, this problem had already been foreseen and there is a very easy way to change the directory the files and transport zip are outputted to. It's so easy, it would have fit in a tweet - but how much fun would that be? So that's why you''ve been reading this blog.

Oh you want to know how now you got this far?

Okay.

Let's say this is part of our regular build script:

That's where it instantiates the modPackageBuilder class. So that's where we'll want to tell it where to output to. We'll be using a public property of the modPackageBuilder for that: $directory. Hah, bet you didn't see that coming!

So let's just make that single small adjustment to our build script that you've been reading this for..

We're simple traversing up two directories (first we're in _build, then we go up to the root, and lastly add _packages to it, putting it in our all new special directory :)

Now hold on.. what about all those files?

Yes - the package builder also creates a lot of files (that's what gets zipped up, actually) and we probably don't feel the need to upload all that to our git repository. To fix that, we'll create a real simple .gitignore file and put that in our _packages directory.

It will ignore all folders, but not files in the root (our transport package).

And that's it folks. :) Thanks for the masterminds Shaun & Jason for thinking of stuff like this ahead of time!

This article has been translated to Russian as well. Know of any other translations? Let me know!

MODX has very granular caching possibilities. Where Evo limits caching to just snippets and has other limits in terms of embedding tags in tags, MODX Revolution does not have these limits.

In MODX Revolution you can cache / uncache any tag you may use. That includes snippets, but also setting tags, placeholders, chunks and even lexicon strings. This article will talk you through some reasons when to call what cached in MODX Revolution (but it will save you most of the inner workings).

It is important to realize that, unless you're doing real advanced stuff and have overriden the core caching, the cache will be cleared when you: save a resource, element and to some extent settings. So basically, whenever your content or structure is changed - your cache will be updated to reflect that.

How do I tell MODX to (not) cache a certain tag?

Simply add in the exclamation mark (!) in front of the token. The token is a dollar sign ($) for Chunks, a plus sign (+) for placeholders, a percentage sign (%) for lexicons and an asterix (*) for resource fields and template variables. Snippets do not have a token.

Examples cached:

[[Wayfinder]]
[[$chunkname]]

Example uncached:

[[!Wayfinder]]
[[!$chunkname]]

Four Guidelines for Caching Strategies

  1. Cache everything you can.
    In general you will want to cache as much as you can - why should MODX retrieve the content from the database on every request, when it hasn't changed? While this may seem logical (and there is rarely any question on wether to use resource fields cached or not), the same applies to other tags. If the output of a snippet doesn't change on every request - why should MODX parse the snippet on every request? It shouldn't.
  2. User-specific output needs to be uncached.
    Anything that outputs user specific information will need to be uncached. Simple - any request could be made by different users, so it wouldn't do any good to load user-specific information from cache. This applies to snippets that interact with the user system (the Login package, a Wishlist component but also Quip for commenting), but also the somewhat less known [[!+modx.user.id]] placeholder.
  3. Content related to URL parameters or POST data needs be uncached.
    If you're using a form which displays information about what was posted, you don't want that to be fixed and cached. So call that uncached as well. If your output is different depending on (non-id/q) URL parameters (for example a search results page), you will want to make sure the related snippet is not cached.
  4. Use custom caching when needed.
    If data (in a snippet) changes before the cache has been cleared, for example if data is retrieved from an external XML source that changes independantly to your MODX Content, you will want to use a custom caching method such as the getCache snippet or write your own caching within the snippet code.

Exceptions

Of course there are exceptions...actually, I could only come up with one so far: (Updated October 26th, came up with another one!)

  • site_url, http_host and related system settings may be called uncached. If you have a certain setup in which one resource+template combination may server multiple urls, you will want to make sure the related system settings are called uncached. One common application for this is your <base href="[[!++site_url]]"> tag for friendly URLs: you will want to be sure your assets are served from the same domain as the user visited. Wether you should force www/no www through htaccess is a whole different discussion, but that is very much related to this and a common cause for issues.
  • Snippets that (may) redirect or forward elsewhere will usually need to be uncached as the result may be empty, but the processing of the snippet does the work to redirect the user.

Common Errors

  1. Calling the "if" snippet uncached, when the subject is a resource field. Most likely caused by the documentation having uncached examples, a lot of people mis-use the If snippet by not caching it when the subject is static for the cache lifetime.
  2. Calling the "Wayfinder" snippet uncached. Please don't! Wayfinder especially can take a good performance hit on your website with larger menus. The only valid reason I can come up with to call Wayfinder uncached is when the output is user specific (member-only area, for example).
  3. Uncaching placeholders in chunks that are used as templates in snippets. Uncached placeholders are processed at the last possible time, and calling them uncached in a template will leave them in tact and they wont get replaced by the snippet, but instead by the core parser long after the snippet is done. (This behavior has seen some changes/bugs in the 2.1.x releases, so it could be that this is different in the latest release - I haven't actually checked in much detail recently.)

What are the errors you have made that you spotted in the above lists? Or do you have other caching tactics or questions? Share below!

Based on a comment from Marc Hinse, a follow-up to this post has been written discussing how nested caching works in MODX Revolution.