Link Deduplication

All posts, RSS Feed

Campaigns in RPGpad have a lot links between pages: the text of a wiki page can link to another page, you can link to forum threads, and from forum posts to wiki pages, every Person on the wiki can link to where they live, all tags on a wiki page are links to those tags, and so on.

We are happy with all these links, because a campaign with a lots of interlinking between pages is a campaign that can be navigated through quickly. And those links often describe how topics are related to each other, which informs players of the campaign and helps them get the information they are looking for.

That’s why, about two years ago, we built the Wanted Pages feature (as described in Orphans and Experiments). Last week, we upgraded the way we handle links internally. In this post we want to give you an idea of what we changed, and why.

Before our change, a page with six links to other pages would work as follows:

Current situation

The page on the left is the page you just saved, the other pages are pages that are already there. As seen in the above image, each Link is made into a separate little piece of data, and the page references those little pieces of data. Each of these pieces of Link data then in turn references the actual page it links to.

The reason we keep the Link data separate is to allow all kinds of interesting things. For example, links to wanted pages (also known as redlinks) are represented by a Link that does not reference to an actual page, because there is no page that matches the link. The wanted pages overview then grabs all links that do not refer to a page, and shows them in an overview.

As you can see, the six links on the page actual link to only three other pages. Here, we’ve coloured them to clarify:

Analysis of links

Having multiple links to the same page adds additional data to RPGpad that does not really provide new information to the system. So, we have changed RPGpad to analyse the page each you save it. This analysis bundles together links going to the same place — this is done by remove duplicates, or “deduplicating” the links:

Links deduplicated

As seen in the image above, of the six links only three remain. Note that the page you just saved still references the link multiple times: we want every link the player writes on the page to remain, the goal here is to remove duplicate data, not to remove duplicate links in the text.

This deduplication has several benefits. The first is reducing the amount of data RPGpad has to work with. Some campaigns have pages with up to 700 links! That is a lot of links. To be fair though, those pages are mostly linking to the same gallery exhibit image at many places on the page — see the bottom of this post for information on gallery image inclusion.

However, many processes and features in RPGpad check all links, or perform some operation on all links in a campaign. Having a few pages with 700+ links quickly adds up to having thousands of duplicate links that need to be checked. By reducing the number of links, we have reduced the amount of work that needs to be done for those campaigns, making everything just a little bit faster!


As always, we have this week’s changelog ready for you. If you have any ideas, bugs, feedback, or just want to talk with us, please open a thread on the community forum.


Did you know? Gallery image inclusion It is possible to include an image from a gallery directly in another page (or forum post, or chat message). To do so, use the following marking:

{{gallery>Page with Gallery#Exhibit Title}}

In a forum post or chat message you might have to use setting/Page with Gallery to link to the correct wiki.

For more information see the community campaign help, or your campaign’s help pages.