The Clifton Method - Part I

• Download source code - 38 KB
• Complete source code, including the demo here

Series of Articles

• Part I - Module Manager - Dynamically Loading Assemblies at Runtime (this article)
• Part II - Service Manager - Instantiation by Interface Specification
• Part III - Bootstrapping with the Module Manager and the Service Manager
• Part IV - The Semantic Publisher/Subscriber

Introduction

Building applications with runtime loaded assemblies is a useful capability, as it provides the ability to:

1. customize an application from "logical components."
2. replace components with new or different behavior, for example, mock objects.
3. extend an application's behavior by adding new components.

By "logical components", I mean an assembly (DLL) that encapsulates all the functionality of "something" into a group. An easy way to first understand this concept is by thinking about physical devices, for example, a camera. All the functionality that a camera might provide, such as taking a picture, streaming video, configuring its resolution, and other settings, can be organized into a logical component. Because different cameras will most likely have different APIs and options, one can create different DLLs for each physical device and customize the application for the actual camera (or cameras) that are part of a particular installation.

Creating services as logical components is another useful thing to do. For example, a web server can be a component, encapsulating handling HTTP requests. The router might be a different component. If you're implementing a server that provides public services, you probably don't need a router that performs authentication. If you're implementing a server that serves non-public pages or REST calls, you would want a router component that performs authentication. Another good example is interfacing to different databases, such as SQL Server, Oracle, or a NoSQL database.

When is a set of classes suitable for wrapping in a module as a component, and when are they not? Ideally, you should already be asking yourself this question when writing a non-modular application because even the internals of such an application should be organized in such a way that the code is structured "logically."

What is Suitable For Modularization?

Ask these questions:

Does It Behave Like a Component?

1. Is the set of classes intended to be re-usable?
2. Would it be useful to mock the functionality provided by these classes?
3. Am I coding against a specific physical (hardware) object or against a specific behavior?
4. Can I foresee the physical hardware changing or the behavior changing depending on application/customer-specific needs?
5. Are the classes implementing high level behaviors?
6. Are the classes implementing business rules that might vary depending on application-specific needs?

In the last question, what do we mean by "high level behaviors?"

Is It High Level?

High level behavior typically:

1. Interfaces with other high level components in the system.
2. Communicates to other applications (such as a database).
3. Handles asynchronous events, such as HTTP, TCP/IP, serial, USB, or other transports.
4. Interfaces with physical hardware.
5. Is implemented as a prototype (is instantiated) but acts as a singleton for the specific functionality that it provides.
6. Implements a common interface for all logical components of the same "kind" (hence #5, instantiated.)
7. #6 means that the behavior is abstracted through an interface.

Is it Low Level?

Low level behaviors typically fall into the "utility" category, classes that implement:

1. Extension methods
2. Conversion methods
3. Can be, and often is, implemented as static methods
4. Does not typically implement interfaces to abstract the behavior

You essentially have all "yes" answers for "Does it behave like a component?" and "Is it high level?", and all "no" answers for "Is it low level?"

Modular Component vs. Inheritable

The typical approach to building "components" is to use inheritance and a factory pattern.

A Typical Inheritance Architecture

Here, the code, in a monolithic manner, implements the each of the concrete server implementations and asks the factory method to create the desired concrete object.

A Typical Modular Component Architecture

In this scenario, the application gets an instance of whatever is specified in a separate configuration file as to the implementing concrete type. The concrete types are implemented as separate assemblies.

What do you notice about this?

1. The module loader is similar to a factory pattern, but you don't ask it to get you anything, instead the application, through a configuration file, determines what is loaded. The "get me something" factory pattern is replaced a "you get this", more like a strategy pattern.
2. The concrete classes that are implemented in the module typically don't inherit from a base class but rather implement an interface.
3. Inheritance is replaced with modularity.

That last point is worth repeating: Inheritance is replaced with modularity. Instead of creating a potentially complex inheritance tree of specializations, each specialization lives autonomously in its own module, and the inheritance is usually very shallow, in fact usually just implementing the interface requirements.

The Annoying Thing about Modular Architectures

In order to utilize a modular architecture, the interface classes must be shared, either by referencing the same files, typically in some common folder, or wrapping the shared files in an assembly shared both by your application and the module. The salient point is that the files, or assembly of the files, must be shared. Why? Because:

1. The application needs to know how to talk to the module, which is through the interface
2. The module needs to know what it implements, again through the interface.

A monolithic application doesn't have this issue because everything is in the same application solution:

Once you start writing your code so that you have different projects (assemblies) for your components, you must share the interface source code or abstract class code in a shared assembly:

This is also true when you start writing components as runtime loaded modules:

Here again, we see an inversion of more monolithic application development. Rather than:

• everything living in one project (highly monolithic)

or:

• many projects with each project referencing other projects that it needs to know about (compile-time modularity)

we instead have:

• an application that shares an interface specification and separate modules (assemblies) which are not referenced by the application nor which reference each other, except through the shared interface.

Certainly, the project files for the different modules may live together in the same solution (and even in multiple solutions, as these modules are often shared), but the salient point is, neither the application, nor the modules, directly reference other modules.

Implementation

After that lengthy introduction, we can look at the implementation details.

Side Note

One thing to note is that in this code, I was dabbling, somewhat inconsistently, with semantic types, as I wrote about in Strong Type Checking with Semantic Typess. So you'll see a couple types like XmlFileName and AssemblyFileName that are type wrappers for strings. I'm not sure whether there's benefit to this or not -- it's probably unnecessary complexity here, though I do like the idea of specifying what type of string is expected in the parameter, based on type rather than variable name.

I also rely on some Linq extension methods and assertion methods that are in the repo, but are not discussed here.

Specifying Modules

I specify modules in an XML file. You could just as easily put this in the application config file, a database table, a JSON document, or something else. The XML format that I use looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<Modules>
  <Module AssemblyName='[some module name].dll'/>
  <Module AssemblyName='[another module name].dll'/>
</Modules>

I load the XML file into the List<AssemblyFileName> that the module registration requires (see below) with two helper methods in your application:

/// <summary>
/// Return the list of assembly names specified in the XML file so that
/// we know what assemblies are considered modules as part of the application.
/// </summary>
static private List<AssemblyFileName> GetModuleList(XmlFileName filename)
{
  Assert.That(File.Exists(filename.Value), 
             "Module definition file " + filename.Value + " does not exist.");
  XDocument xdoc = XDocument.Load(filename.Value);

  return GetModuleList(xdoc);
}

/// <summary>
/// Returns the list of modules specified in the XML document so we know what
/// modules to instantiate.
/// </summary>
static private List<AssemblyFileName> GetModuleList(XDocument xdoc)
{
  List<AssemblyFileName> assemblies = new List<AssemblyFileName>();
  (from module in xdoc.Element("Modules").Elements("Module")
    select module.Attribute("AssemblyName").Value).ForEach
          (s => assemblies.Add(AssemblyFileName.Create(s)));

  return assemblies;
}

Module Registration

Using the above code (for an XML file), I register modules with:

IModuleManager moduleMgr = serviceManager.Get<IModuleManager>();
List<AssemblyFileName> modules = GetModuleList(XmlFileName.Create("modules.xml"));
moduleMgr.RegisterModules(modules);

Module registration is performed here:

/// <summary>
/// Register modules specified in a list of assembly filenames.
/// </summary>
public virtual void RegisterModules(
  List<AssemblyFileName> moduleFilenames, 
  OptionalPath optionalPath = null, 
  Func<string, Assembly> assemblyResolver = null)
{
  List<Assembly> modules = LoadModules(moduleFilenames, optionalPath, assemblyResolver);
  List<IModule> registrants = InstantiateRegistrants(modules);
  InitializeRegistrants(registrants);
}

OptionalFolder Parameter

The idea here is to allow the application to specify a sub-folder in which modules (DLLs) are located. This helps to create a cleaner separation of module assemblies from other, "statically linked" dependencies.

AssemblyResolver Parameter

This is an interesting optional parameter. It is a function that takes the module name and returns the Assembly. The idea here is that the assembly may be located in a strange place, such as a resource file of the application. I haven't written about this technique, but when I do, I'll come back here and provide a link to that concept. In general though, you can use this optional function to attempt to resolve an assembly located somewhere other than a sub-folder of the application.

Loading the Modules

The method LoadModules iterates through the list of modules:

/// <summary>
/// Load the assemblies and return the list of loaded assemblies. In order to register
/// services that the module implements, we have to load the assembly.
/// </summary>
protected virtual List<Assembly> LoadModules(List<AssemblyFileName> moduleFilenames, 
OptionalPath optionalPath, Func<string, Assembly> assemblyResolver)
{
  List<Assembly> modules = new List<Assembly>();

  moduleFilenames.ForEach(a =>
  {
    Assembly assembly = LoadAssembly(a, optionalPath, assemblyResolver);
    modules.Add(assembly);
  });

  return modules;
}

This returns a list of Assembly instances.

LoadAssembly attempts to actually load the assembly, optionally passing the "I need this assembly" over to the assembly resolver function that you provided in the registration call.

/// <summary>
/// Load and return an assembly given the assembly filename so we can proceed with
/// instantiating the module and so the module can register its services.
/// </summary>
protected virtual Assembly LoadAssembly(
  AssemblyFileName assyName, 
  OptionalPath optionalPath, 
  Func<string, Assembly> assemblyResolver)
{
  FullPath fullPath = GetFullPath(assyName, optionalPath);
  Assembly assembly = null;

  if (!File.Exists(fullPath.Value))
  {
    Assert.Not(assemblyResolver == null, "AssemblyResolver must be defined 
              when attempting to load modules from the application's resources.");
    assembly = assemblyResolver(assyName.Value);
  }
  else
  {
    try
    {
      assembly = Assembly.LoadFile(fullPath.Value);
    }
    catch (Exception ex)
    {
      throw new ModuleManagerException("Unable to load module " + 
                                       assyName.Value + ": " + ex.Message);
    }
  }

  return assembly;
}

The optional path is appended to the executing assembly location in the GetFullPath method:

/// <summary>
/// Return the full path of the executing application 
/// (here we assume that ModuleManager.dll is in that path) 
/// and concatenate the assembly name of the module.
/// .NET requires the full path in order to load the associated assembly.
/// </summary>
protected virtual FullPath GetFullPath
         (AssemblyFileName assemblyName, OptionalPath optionalPath)
{
  string appLocation;
  string assyLocation = Assembly.GetExecutingAssembly().Location;

  if (assyLocation == "")
  {
    Assert.Not(optionalFolder == null, "Assemblies embedded as resources require that 
              the optionalPath parameter specify the path to resolve assemblies.");
    appLocation = optionalPath.Value; // Must be specified! Here the optional path 
    //is the full path. This gives two different meanings to how optional path is used!
  }
  else
  {
    appLocation = Path.GetDirectoryName(assyLocation);
    appLocation = Path.GetDirectoryName(assyLocation);

    if (optionalPath != null)
    {
      appLocation = Path.Combine(appLocation, optionalPath.Value);
    }
  }

  string fullPath = Path.Combine(appLocation, assemblyName.Value);

  return FullPath.Create(fullPath);
}

Unfortunately, in the above code, there is a dual use of the optional path. There is a nuance of loading assemblies when the Module Manager is itself an embedded resources assembly -- the executing path in this case is an empty string, because the assembly, loaded by .NET assembly through a separately implemented assembly resolver (not discussed here), isn't associated with the executing assembly! This is very odd behavior and this whole situation should be ignored until I write the article on embedding assemblies as resources. In this case, the optional path is the full path to resolves the assembly location. Quite honestly, this whole issue requires a refactoring of how assemblies embedded as resources should be handled.

Instantiating Registrants

Once the assemblies have been loaded, the Module Manager instantiates the registrants. A "registrant" is a special class (one and only one such class) in each module that has implements the method in the interface IModule. In other words, our modules "know" that they are modules and can do some special things because they are modules. What those special things are is essentially up to you. In my library, the module is initialized with a Service Manager so that the module can register the services that it provides. That is described in the next article in this series. For now, we just need to know that the each module must provide a class that implements IModule.

/// <summary>
/// Instantiate and return the list of registrants -- assemblies with classes 
/// that implement IModule.
/// The registrants is one and only one class in the module that implements IModule, 
/// which we can then
/// use to call the Initialize method so the module can register its services.
/// </summary>
protected virtual List<IModule> InstantiateRegistrants(List<Assembly> modules)
{
  registrants = new List<IModule>();
  modules.ForEach(m =>
  {
    IModule registrant = InstantiateRegistrant(m);
    registrants.Add(registrant);
  });

  return registrants;
}

/// <summary>
/// Instantiate a registrant. A registrant must have one and only one class 
/// that implements IModule.
/// The registrant is one and only one class in the module that implements IModule, 
/// which we can then
/// use to call the Initialize method so the module can register its services.
/// </summary>
protected virtual IModule InstantiateRegistrant(Assembly module)
{
  var classesImplementingInterface = module.GetTypes().
    Where(t => t.IsClass).
    Where(c => c.GetInterfaces().Where(i => i.Name == "IModule").Count() > 0);

  Assert.That(classesImplementingInterface.Count() <= 1, 
             "Module can only have one class that implements IModule");
  Assert.That(classesImplementingInterface.Count() != 0, 
             "Module does not have any classes that implement IModule");

  Type implementor = classesImplementingInterface.Single();
  IModule instance = Activator.CreateInstance(implementor) as IModule;

  return instance;
}

Initializing Registrants

Each registrant in the module, once instantiated, can be initialized. The initialization method is a virtual stub in the ModuleManager class:

/// <summary>
/// Initialize each registrant. This method should be overridden by your application needs.
/// </summary>
protected virtual void InitializeRegistrants(List<IModule> registrants)
{
}

If your modules need initialization, you would derive from ModuleManager and implement the specific initialization you require.

Example Program

Our demo solution consists of these four projects:

1. CommonInterface - This holds the IModule definition
2. ModuleConsoleSpeak - Emits a message to the console
3. ModuleManager - A console demo of the module manager
4. ModuleVoiceSpeak - Speaks the message using .NET's voice synthesizer

The example program implements two "speakers", one that emits a console message, the other that speaks the message with .NET's speech synthesizer. The application is quite simple:

static void Main(string[] args)
{
  IModuleManager mgr = new ModuleManager();
  List<AssemblyFileName> moduleNames = GetModuleList(XmlFileName.Create("modules.xml"));
  mgr.RegisterModules(moduleNames, OptionalPath.Create("dll"));

  // The one and only module that is being loaded.
  IModule module = mgr.Modules[0];
  module.Say("Hello World.");
}

Note how the application is loading the modules from the "dll" sub-folder:

The "dll" folder contains the assemblies for the two modules:

Also note that a post-build step copies the module into the "dll" folder:

copy ModuleConsoleSpeak.dll ..\..\..\ModuleManager\bin\Debug\dll

copy ModuleVoiceSpeak.dll ..\..\..\ModuleManager\bin\Debug\dll

Now, by changing the modules.xml file, the application responds by emitting the text to the console window or by speaking it. To speak the text, use this:

<?xml version="1.0" encoding="utf-8" ?>
<Modules>
  <Module AssemblyName='ModuleVoiceSpeak.dll'/>
</Modules>

To emit the text on the console window, use this:

<?xml version="1.0" encoding="utf-8" ?>
<Modules>
  <Module AssemblyName='ModuleConsoleSpeak.dll'/>
</Modules>

The Application does not Reference the Modules

Note that in the demo application (called "ModuleManager", a bad name!), there is no reference to the modules we're loading:

IModule

This implementation is for demonstration purposes only!

namespace Clifton.Core.ModuleManagement
{
  public interface IModule
  {
    void Say(string text);
  }
}

Here, we see the problem I talked about earlier with runtime modular applications -- the common interface must exist in a separate assembly! If you're not careful, this can and will cause and explosion of assemblies that simply define interfaces and other shared types. We'll talk about this further in other articles.

The actual implementation of IModule is tied in with the Service Manager (the next article):

namespace Clifton.Core.ModuleManagement
{
  public interface IModule
  {
    void InitializeServices(IServiceManager serviceManager);
  }
}

ModuleConsoleSpeak

using System;

using Clifton.Core.ModuleManagement;

namespace ModuleConsoleSpeak
{
  public class Speak : IModule
  {
    public void Say(string text)
    {
      Console.WriteLine(text);
    }
  }
}

ModuleVoiceSpeak

using System.Speech.Synthesis;

using Clifton.Core.ModuleManagement;

namespace ModuleVoiceSpeak
{
  public class Speak : IModule
  {
    public void Say(string text)
    {
      SpeechSynthesizer synth = new SpeechSynthesizer();
      synth.SetOutputToDefaultAudioDevice();
      synth.Speak(text);
    }
  }
}

A Word About the IModule Interface

This a demonstration only! Placing module-specific requirements into IModule is not recommended. Modules are intended to implement a wide variety of things, and you can't and shouldn't describe those implementations in the IModule interface. I am doing so here only because it's simple to demonstrate just the Module Manager, knowing that the two modules I've implemented are very limited and have common behavior. In any real application, I would never do this, which is why the next article is about the Service Manager.

Conclusion

The demo illustrates the purpose of the Module Manager -- to be able to change the application's behavior:

1. Without recompiling the application
2. Through a configuration file

That's all I intended to do with this article, as the Module Manager is the core component of The Clifton Method.

A Word about Modules vs. Dependency Injection

Another way to go about this while issue is with dependency injection -- using reflection and type information, instantiate an object that implements a particular interface and assign it a property of that interface type. Personally, I find DI to be overly complicated (you might say the same of what I'm doing here!) and the DI frameworks that I've seen are bloated, slow, and it becomes difficult to debug the application.

Besides, I use the Module Manager in a specific way -- to load assemblies that implement services. The registration process is simple to understand, simple to walk through with a debugger, and the entire implementation is very few lines of code, which makes it easier to maintain.

Other Features

Deferred Loading

At the time of this writing, the Module Manager loads all modules immediately -- there is no concept of deferred loading. This might be useful to implement in the future -- an "on demand" loading of modules, but there are certain complexities, in that the Service Manager (which is the next article) would need to know what module to load for a particular implementation. I've not really encountered a requirement for this kind of behavior, so there's really no reason to implement it, yet.

Unloading Modules

Similarly, the idea of being able to unload a module at runtime is attractive in that you could potentially replace a module without bringing down the application. However, as I've written about regarding Application Domains, the performance and potential problems with event wire-ups makes this feature somewhat undesirable.

History

• 25^th August, 2016: Initial version