Better Software with Entity Systems

In today’s post, I want to introduce you a concept called Entity-component System and to show how to use it for web development. The idea is not new. In fact, it is widely used in game development. But I have never heard about it when talking about backend development and data modeling.

The Problem

I suggest we start with a very typical feature: featured content. Let’s say we have a list of books:

{
  "title" : "The Hobbit",
  "author" : "Tolkien"
},
{
  "title" : "Lord of the Rings",
  "author" : "Tolkien"
}

We want to mark some of them to be rendered in the “Featured Books” section. Easy:

{
  "title" : "The Hobbit",
  "author" : "Tolkien",
  "featured" : true
},
{
  "title" : "Lord of the Rings",
  "author" : "Tolkien",
  "featured" : false
}

Then we realize that we want to specify the order in which they appear:

{
  "title" : "The Hobbit",
  "author" : "Tolkien",
  "featured" : true,
  "featuredOrder" : 1
},
{
  "title" : "Lord of the Rings",
  "author" : "Tolkien",
  "featured" : false,
  "featuredOrder" : 0
}

Looks good, but… We also want to have a particular order for our mobile client, but the order is different from the one we have on the website. We can add another property:

{
  "title" : "The Hobbit",
  "author" : "Tolkien",
  "featured" : true,
  "featuredOrder" : 1,
  "featuredOrderMobile" : 2
},
{
  "title" : "Lord of the Rings",
  "author" : "Tolkien",
  "featured" : false,
  "featuredOrder" : 0,
  "featuredOrderMobile" : 0
}

You can imagine that such a list of features can grow further until the sun goes out. It has at least two disadvantages:

  • there is no explicit schema (some objects do not need all those featured* properties)
  • given that we store last modification date (e.g. updatedAt): each time we change the order of a book, it is marked as changed, even though the essence of the record (title, author, description, etc.) is still the same. Technically the book has changed, but logically it is not.

Let’s see if we can fix this.

Entity-Component Systems

There are many great explanations on the Internet, so I will not dive too deep. But here is a concise example: let’s say you build a game and you need to model things like swords and axes and bows and whatnot. For the sake of simplicity let’s consider the weapon to have a few properties:

interface Weapon {
  string name;
  double damage;
  double weight;
}

As in any game you could sell and buy weapons, so we need to add a price:

interface Weapon {
  string name;
  double damage;
  double weight;

  double price;
}

However, there might be weapons that can never be sold because they are part of the storyline. It is easy to fix:

interface Weapon {
  string name;
  double damage;
  double weight;

  double price;
  bool canBeSold();
}

There can be many more properties such as magical effects (one, or none, or many), or condition (broken, charged, enchanted), or distance (bows, magic weapon), or anything else you can imagine. It is evident that the classical OOP approach would not work very well here: classes will become bloated very fast, they will contain lots of useless properties.

One of the possible solutions is an Entity-Component Systems approach: an entity is an entity, simple. The component is an aspect of an object (can be sold? has magical effects?). Systems: some aspect of the world, that handles specific components of an entity. Using entities and components, we can create some weapon:

Entity MagicalSword() {
  Entity entity;
  entity.addComponent(PriceComponent(20));
  entity.addComponent(MagicalEffects.fireballComponent(10));
  entity.addComponent(Weapons.swordComponent("Fireblade", 15, 10));
  return entity;
}

Entity QuestAxe() {
  Entity entity;
  entity.addComponent(MagicalEffects.lightning(10, 4));
  entity.addComponent(Weapons.axeComponent("Thunderstorm"));
  return entity;
}

Then, those elements can be used by a system (e.g. SellerDialogueRenderer):

class SellerDialogueRenderer {
  void renderItems(Inventory inventory) {
    for (entity in inventory.items()) {
      if (entity.hasComponent(PriceComponent)) {
        renderEntity(entity);
      }
    }
  }

  void renderEntity(Entity entity) {
    some_basic_entity_rendering(entity);
    if (entity.hasComponent(MagicComponent)) {
      render_magic_effects(entity);
    }
  }
}

The example might not be the most correct one, but I hope it illustrates the idea of splitting the entity and its aspects. Let’s see how it is applicable for web development.

Data Modelling using ECS

Fortunately, ECS, or its concepts, applicable not only to games. Let’s look how we can implement initial task by separating data and its aspects:

// Books Collection:
{
  "_id" : "the_hobbit",
  "title" : "The Hobbit",
  "author" : "Tolkien"
},
{
  "_id" : "lord_of_the_rings",
  "title" : "Lord of the Rings",
  "author" : "Tolkien"
},
{
  _id: "harry_potter",
  title: "Harry Potter and the Philosopher's Stone",
  author: "Rowling",
}

Instead of extending existing records we can create a separate collection:

// FeaturedBooks:
{
  "_id" : "whateverhere",
  "book_id" : "the_hobbit",
  "order" : 2
},
{
  "_id" : "another_id",
  "book_id" : "lord_of_the_rings",
  "order" : 1
}

The same for the mobile clients:

// FeaturedMobileBooks:
{
  "_id" : "whateverhere",
  "book_id" : "the_hobbit",
  "order" : 2
}

Managing complexity this way is much easier. We can do whatever changes we want without affecting the real data: add more properties (via new collections), remove unneeded properties (by dropping a collection). If we change the order now, then it doesn’t affect the book itself.

However, it comes with a little disadvantage: fetching data is a bit less trivial now. Fortunately, we can use MongoDB Aggregation Framework for this task.

Fetching data

To prepare the data for a client we need to retrieve all FeaturedBooks, then for each featured book fetch the corresponding book (via book_id), then order results based on the order field, and then drop all unnecessary data. Here is the required pipeline:

db.FeaturedBooks.aggregate([
  { $lookup : {
    from : 'books',
    localField : 'book_id',
    foreignField : '_id',
    as : 'book'
  } },
  { $unwind : '$book' },
  { $sort : { order : 1 } },
  { $replaceRoot : { newRoot : '$book' } }
])

The command will give us the following result:

{
  "_id" : "lord_of_the_rings",
  "title" : "Lord of the Rings",
  "author" : "Tolkien"
},
{
  "_id" : "the_hobbit",
  "title" : "The Hobbit",
  "author" : "Tolkien"
}

Let’s look at each stage.

$lookup will find all the books which _id equals to the book_id from the FeaturedBooks. It will put results into a field names book, as we requested. The book field holds an array since there could be more than one match.

In our case, the book array has only one object inside. We can safely $unwind it to inline the book.

$sort is straightforward. We sort objects based on a value of order field. 1 means sort in ascending order. To flip the order, you can use -1.

At this moment our records have the following form:

{
  "_id" : "whateverhere",
  "book_id" : "the_hobbit",
  "order" : 2,
  "book" : {
    "_id" : "the_hobbit",
    "title" : "The Hobbit",
    "author" : "Tolkien"
  }
}

We need to replace the whole thing with the nested book object. $replaceRoot does exactly that. Note that the type of a newRoot must be an object.

To conclude

That was it. The Entity-Component System is much more powerful, but also more sophisticated. However, even by applying simple parts of it we can model our system much better.

I highly recommend reading the following series of articles on the ECS: Entity Systems are the future of MMOG development. The author is talking about game development, but the concept is applicable in other areas as well.

Subscribe to our mailing list