Notifications System Design Document

Purpose of Notifications System

In the form that is easiest for end-users to understand, the notifications system is simply a way for events in the site to trigger the site to contact them (i.e. send them an email, text-message, etc.). In general, however, the notifications system provides a level of indirection between certain occurrences on the site and certain actions that should be taken as a result. This is quite a bit more general and extends the usefulness of the system considerably.

The simplest standard example is, for instance, when a user receives a message or is tagged in content, he or she can receive notification through email of that event. Some possible extensions of this include receiving notification of events that happen to objects connected to a user (such as comments on content, actions on a group, etc.), although we should be careful with the default behavior here.

Where the generality of the system helps us is that the system can be used as a dispatch system between different parts of the system. For instance, the completion of a video conversion process might trigger an event in the notification system. The listener for this type of event could then set up the changes to add the converted video to the users profile, etc. Another example is that the notification system could be used to queue up items for re-ranking by the content appraiser. In fact the content appraiser could be implemented as just a client of the notification system.

Design Philosophy

The notification system is loosely based on the Observer Pattern, but with many changes to deal with a number of key issues related to the details of our system. Some issues that make the Observer Pattern in-applicable to our situation:

  • We need to answer web requests in as timely a manner as possible. Anything that could take time and isn’t necessary to respond to the user’s request must run in a background thread.
  • We need notifications to be persistent across instances of the application and across servers. To satisfy this, the notification registrations must be persisted in the database.
  • Some types of notification responses may require special functionality to be handled. This includes sending emails (as this probably has to be serialized to the SMTP server) and possibly text-messages. Thus the system should be general enough to serialize some actions when needed without forcing the user to wait when those things are unrelated to actually answering the web request.

In order to address these issues, the flow of a notification through the system essentially works out like this:

  1. The source of a notification creates an instance of a class that implements INotification. In a model similar to the exception framework, this can be sub-classed to divide into different types of notifications. Then the source calls Notify on the backend object to receive the notification. If the need arises, we might also create some sort of system notification, not tied to any backend object.
  2. The notification is serialized[1] and put onto a queue (implemented in a database table[2]).
  3. In a background thread, a notification handler pulls a notification off the queue and de-serializes it back into a notification object.
  4. A visitor on objects is run on the backend object to create another visitor for handling notifications. It is at this stage a number of interesting things happen. If the object is a user object, the code builds a visitor (starting from some type of default) based on the user’s preferences about notifications. In the case of groups, the group builds a visitor by building one for each of the sub-entities of the group, but handing them notifications decorated with a special instance of INotification (perhaps called IGroupNotification). In the case of content items, it can notify people with various relations to the content item (i.e. authors, those who have commented, etc.), again decorating these with appropriate special types of INotification.
  5. This visitor on notification objects is then called to handle the notification. By using different preset defaults, different notifications can result in different outcomes. Delegates to go into these visitors can be written to handle notifications as SMTP or Text-Messaging, or save them into a database structure to serve and handle RSS feed requests.

Examples
I think an example will provide a good basis for understanding how notifications will be processed.

Assume user Aaron sends a message to User Brad. The message handling code calls

UserBrad.Notify(new NewMailNotification(…details about mail message…));

Of course this assumes that UserBrad is an instance of the user object for the user. This is handled by the backend to actually call a method to save the Guid of UserBrad and the NewMailNotification object (which is serialized) into the database.

Then at some point in the future, when the notification handler reaches this item in its queue, it pulls back out UserBrad and the NewMailNotification object. It then uses a notification handler visitor to build the notification handler for this type of object. This visitor knows how to load preferences for users and it knows how to handle notifications on other types of objects appropriately.

So the code calls:

notificationHandler = UserBrad.Execute(new NotificationHandlerFactory());

Then it uses this notification handler visitor to decide how to handle the given notification:

notificationObject.Execute(notificationHandler);

In this case the notificationHandler that gets created might have a delegate to support NewMailNotification that ends up sending SMTP mail to the user. However, the system provides flexibility in that when the notificationHandler is constructed, the user’s preferences can be consulted to decide which handler to install in the (extended) visitor in each case. So if User Brad had decided not to receive an email when this event happened, the NotificationHandlerFactory would have put a no-op (or something else) as the delegate to handle this type of notification.

Some Notes the Visitor Design

The double visitor design is somewhat obtuse, and I want to clear up why I chose to do it the way I did (also noting that some of these are “engineering” decisions that might be revisited).

First of all, using a visitor to construct a visitor is essentially saying that I want a visitor that operates on both the backend object and the notification object (i.e. instead of being parameterized on either its parameterized on the Cartesian product of them).

The reason I chose to have the backend object create a visitor to handle notifications instead of the other order (i.e. a visitor running on notifications to create a visitor on backend objects) is two-fold. Firstly, there are likely to be a lot more notification types than backend types (which I believe should be rather fixed now). Secondly, I don’t see a clean way to implement the decoration of notifications before passing them along to relevant parties using the other system. What I mean by this is, if I comment on a piece of content, that should notify the piece of content. The piece of content the creates a notification handler that notifies each interested party (i.e. authors, etc.), but calls it with a decorated form of the current notification. So the notification handler visitor returned when the backend object is a content might simply have a default delegate that called notify on each of the authors, like this:

foreach(IUser a in Authors)

a.Notify(new AuthorNotification(notification,thisBackendObject));

This re-notification allows the users to be able to fine-tune the notification they get on things they are related to, but may not be interested in.

Formatting of Messages

Common to many of the approaches is the fact that the eventual user notification will require conversion of the notification object into some user readable form. I believe this should occur in a special visitor that can be delegated to. This visitor can take the relevant information provided by the different types of notifications and convert it into string format (or perhaps different string formats for different media – a longer message for email, a shorter one for text messages and RSS items). Separating this from the concrete INotification classes enables more flexibility in this framework to add new types of formatting, and may make certain other things (like internationalization) easier.

This visitor can be called by the visitors that handle turning notifications into email, etc.

Forwarding Notifications to Different Receivers

By carefully writing the visitor on backend objects that constructs the notification handler visitor as well as coordination with the user preference system, we will easily be able to add new forms of notification in a simple way and allow the user to choose them from within his or her preferences. In particular, I envision having something like an IUserNotificationReciever interface that could be extended to have different forms of notification receivers, like email or text messages, and would take an input like a string that was formatted and handle the details of delivering it to the appropriate place based on the user’s account (i.e. sending it to the right email or phone number). The correct type would be initialized for each handler based on the user preferences.

Dealing with Complexity

In the above system, I can see approximately 4 levels of indirection (not counting the ability to re-notify, or have notifications cause other notifications). It isn’t by accident that the system is so complicated, as there are a tremendous number of variables that affect how a notification should be handled (what type of notification it is, what object it’s on, the user’s preferences, and the needs of the notification receiver). However, because the system is complex, it may be harder to understand and debug (that’s one of the reasons for this document though). In order to simplify things, I have endeavored to clearly separate what each part is responsible for doing (and not give them enough information to do something else). This is purposeful so that the pieces stay independent and so that it’s simpler when looking at the system as a whole to know where to go look if you are trying to do something (or fix something). We should try to keep it this way.

[1] Presumably using XML Serialization, but perhaps some other way.

[2] Implementation is not so relevant to this document, but someone might look at T-SQL’s CREATE QUEUE