Variety.org.au_

The agency Follow, acquired my services to build them a solution that would sync content between sites for the new variety.org.au website. They had one master site, called national and many other slave sites, called state. Each state site represented a state in Australia.

They were looking for a solution that could easily migrate data between a national site and state sites when updating content on the main, national site. In other words, when content updates were applied to the main national site it would need to be synced across all the other state sites. They explored the idea of migrating the data via an XML document (transportable data file) via a timed event called a cron job.

They already had an existing system in place where they used custom post types with custom fields(ACF). Each custom field had the option to be enabled or disabled on the slave sites via the master site.

Users on the state (slave) sites are only allowed to edit the custom fields assigned to them locally and not fields added by the users on the national (master) site.

I proposed and implement them an alternative, better solution as transporting information via an XML file is not always successful for the reason that some data might not get imported correctly or not imported at all if the cron job fails. My solution was to use the Wordpress API to transport data between the sites. The Wordpress API is now included in Wordpress core since version 4.7 and no longer required to be installed via a plugin.

I implemented 2x custom 3rd party plugins. One for the master site and one for the slave site. The master site also had an additional button, just above the update button on a post edit page, to push updated content to the slave sites. This will update only the national (master) content and not the local content added by the users of the state sites. The reason for this is when the users of the local state sites add their own content that it would not get overwritten when the master site is pushing content down to the state sites when the posts are synced.

When a user on the Master site press the “Sync Post” button, it would make an API call to the state sites. One for one, the state sites would receive the API request, together with the post id of the post requiring the update. The slave site in return would then make an API call to the master, national site and request the content for that post. The request function on the state, slave site had two functions. Run a condition to confirm if the post is new or already exist. If new create a new post and update the data coming from the national site, if an update, only updates the required national custom fields, but leave the local custom fields for that post intact.

There was just one little caveat. When you make an API call to the Wordpress REST API’s endpoint to retrieve the POSTS or a POST /wp-json/wp/v2/posts it will not retrieve the post meta data or attachments (images). The post metadata and images would be needed to be retrieved via separate API calls.

Post metadata endpoint (example).: wp-json/wp/v2/jedi/42/meta

Post image attachments endpoint (example).: /wp-json/wp/v2/media/

So I customized the post endpoint of the Wordpress API to also send custom metadata together with the post data if a post or posts is requested. This limited the API calls to the Wordpress site for when posts needed to be synced.

The customer used Advanced Custom Fields to set up the custom fields for each custom post types. On each Wordpress site, the Advance custom fields are not the same as each custom field on a Wordpress site gets assigned a unique key-value called a reference field (reference the actual custom field created and tied to the setup of ACF). I needed to counter for these reference fields also and carried them over to the state site’s ACF setup without the ACF plugin knowing any better or detecting any changes that would corrupt the custom fields set up for each post type.

The other issue was the images. Images in Wordpress are not stored how you would think under a folder. Wordpress does not scan the uploads folder to know about what images is located under the media library. Every time an image is added to Wordpress, Wordpress need to create a new post type called ‘attachment’ in the database. Only by that post type ‘attachment’ does Wordpress know the location of the image file. Also, Wordpress has a predefined path recorded in the database to the location of the uploads folder. When posts are added or updated on the state sites I needed to run an additional method to fetch the images via another API call. Wordpress has separate end-point for images as stated above. With the API call, I could get the current file location and parameters. On the state site, I would first see if the current image does not exist, by scanning for the image name in the database. If it does not exist I will add the image as a post attachment type, pull in the image and save it to the uploads folder, save the new URL location to the attachment post and then attach that post (attachment) to the custom post type. The custom post type would be the post where the image needs belong too.

That is how I created a data merge setup for variety.org.au so that they can push new or updated content to their state sites on the fly and not wait for a cron job that might fail down the line.