Structuring Content leads to Powerful Content Query API

Written by Sébastien Renault in Concept on March 26,2014

One of the things that got me interested when joining the team, is the possibilities behind the concept of clean separation between content management from one side and the query api from the other side. This is what we call Writing Room and Content Query API.

Not only does it allow content writers to author, manage and publish content with a dedicated tool, but it also opens a lot of possibilities by making all content queryable in all kinds of ways through the API.

This is why I thought of writing a blog post on how structuring content in the Writing Room enables a powerful query API for finding content and integrating it in projects.

Content structure: breaking content into pieces

Structuring content simply means going away from authoring opaque, mixed with style, content blocks that can hardly be analyzed. Instead, we cut or structure content documents into more meaningful parts (we call them fragments).

Let's take an example of a Cup Cake document: instead of creating a huge html document with all the description of a cupcake all mixed together, we rather structure that document to have different fields for title, flavor, price, main image, color, etc.

Not only we structure our documents in different fields, but we also specify the kinds of these fields: title is text, price is a number, etc. This allows us to better organize our content, and more importantly to further exploit the power of the Content Query Api.

Querying content: retrieve documents

Now that we structured our cupcake into different fragments. Let's search for pastries with the flavor "Chocolate". In, we use predicates to express a search. Here is the predicate for finding documents with a flavor:

:d = at(my.product.flavour, "Chocolate")

Basically here we are saying that we're looking for documents containing "Chocolate" in their flavor fragment. In every repository there is an API browser allowing you to experiment with queries on your content. You can try the flavor predicate yourself in the "Les Bonnes Choses" API browser.

In the structure we described above, we saw that we have a price fragment which is a number. This simply means that we can, for instance, search all products with a specific price:

:d = at(my.product.price, 2.5)

To make more useful let's rather search a range of price:

:d = number.inRange(my.product.price, 2, 4)

Of course you can use other number predicates, and in the same way other kinds of fragments like date, color, link, etc.

Full-text search

One of the important feature of a website or app is the famous search box. It allows user to quickly find what they're looking for. A search box uses what is called full-text search: the capacity to search some words inside a lot of content base, and find all relevant documents.

If you have ever tried to implement a search box, and so provide full-text search capability to your project, you know how tedious and inefficient it is to do with classic CMS approaches. Structuring content naturally solves that.

For example, our cupcakes have a textual fragment "Description". This means that we can search over descriptions of pastries for, say, "chocolate cream":

:d = fulltext(my.product.description, "chocolate cream")

Since a document can contain several text fragments, we can choose to do a full-text search in all fragments of a document:

:d = fulltext(document, "chocolate cream")

In, all text and more particularly Rich Text, fragments are analysed and indexed for a full-text search. Moreover, the structure of the text is taken into account for boosting important terms, and to calculate how relevant a search result is.

Combine different predicates

What if we want to look for pastries with the "Chocolate" flavor in a certain price range? We simply combine the different predicates in the same search:

:d = at(my.product.flavour, "Chocolate")
:d = number.inRange(my.product.price, 2, 4)

What about searching all products with "Chocolate Cream" in a certain range of price? We can combine these too:

:d = fulltext(document, "chocolate cream")
:d = number.inRange(my.product.price, 2, 5)

More to explore

Structuring content and combining different predicates provides a powerful way to lookup a repository content and integrate it into a project. We've seen a few examples of possible predicates, but there are many other interesting ones.

There is so much search power one can provide by simply structuring correctly their content, think of geo-localisation, color range, related documents, etc. Predicates certainly provide a flexible mechanism to increasingly integrate more search power.

Sébastien Renault

Core developer and architect.