Upgrading NServiceBus to V5 from V4 - Part2

This is a continuation of my previous NServiceBus upgrade post.

Logging

The logging functionality that used to be in NServiceBus.Core is moved to a separate set of nuget packages such as NServiceBus.CommonLogging, NServicebus.Log4net and NServiceBus.NLog

SetLoggingLibrary from V4 is removed from V5. LogManager.Use<Log4NetFactory>() from the NServicebus.Log4Net will get the job done for Log4Net implementations. The obsolete error message clearly states that.

EndpointName

In order to stop the machine name from being appended, I thought the line below was sufficient.

configuration.ScaleOut().UseSingleBrokerQueue();

It didn't work. Then, I stumbled on this Stackoverflow post and this Github issue. It looks like RabbitMQ, SQLServer and ActiveMQ transports override that setting and try to create a queue with a machine name at the end of it even though one without the machine name exists.

To disable this behavior, you can do something like below (I am using SqlServerTransport for the sake of an example, the same should work for RabbitMQ and ActiveMQ).

configuration.UseTransport<SqlServerTransport>().DisableCallbackReceiver();

StructureMap

It is a good idea to upgrade the dependent packages if they are being used. So, for the StructureMap, after upgrading the NServiceBus.StructureMap package, the configuration that looked like below in V4

Configure.StructureMapBuilder(ObjectFactory.Container)

is like this in V5

configuration.UseContainer<StructureMapBuilder>(b=>b.ExistingContainer([container]));

It is clearly stated in the the error message from StructureMapBuilder extension method. This is an example of a good message.

Assembly Scanning

NServiceBus scans the directory where the assembly with the class that implements IConfigureThisEndPoint resides. This has to be done with care because it is very easy to fall into a dependency hell hole, and the endpoint will not come up because it may start scanning too many assemblies and their dependencies. It can be configured with a finite set of assemblies.

In V4, Configure.With([ListOfAssemblies]) used to be a way to pass the list of assemblies. In V5, configuration.AssembliesToScan(listOfAssemblies) will get the job done where configuration is an instance of BusConfiguration. The name makes more sense in V5. It can take IEnumberable<Assembly>,IIncludesBuilder or IExcludesBuilder.

I think the IIncludesBuilder approach is handy because the list of assemblies will be finite and the rest of the assemblies are excluded at the time of bringing the endpoint up. You can also do some patterns matching.

var listOfAssemblies = AllAssemblies.Matching("YourNameSpace.").And("SomethingElse");
configuration.AssembliesToScan(listOfAssemblies);

More on assembly scanning can be found at this Particular documentation link. This page makes me optimistic that the documentation will eventually catch up.

If too many assemblies are included, the dependency check can spiral into a hole. If too few are included, then you may see errors like No handlers found for the message type or Could not determine type for node like this google groups discussion

MSMQ utilties

The MsmqUtilities class is not public anymore in V5. I don't think it was ever meant to be. It is however allowed to be copied if needed. The V4 source is here and the latest is here

Persistence and features

Particular has provided three types of persistence implementations. InMemory persistence comes with core nuget. NHibernate has its own NServicebus.NHibernate nuget so does the NServiceBus.RavenDB

While using persistence, the order is important. The last option wins. It is highly recommended to take a look at this documentation link

If you want to roll out your own, you can't do it as a BusConfiguration extension because it won't work. It could be implemented as features. The endpoint below is using MyFancyPeristence

 public class MyEndpointConfig
    :IConfigureThisEndpoint
    ,AsA_Server
    ,IWantToRunWhenConfigurationIsComplete
{
    public void Customize(BusConfiguration configuration)
    {
        configuration.UsePersistence<MyFancyPeristence>().For(Storage.Subscriptions, Storage.Timeouts);
    }
    public void Run(Configure config)
    {
        //read settings here.
        //var settings = config.Settings;
    }
}

We are demanding the MyFancyPersistence to provide implementations at least for subscriptions and timeouts. The Storage enum looks like below in NSB codebase as of today.

public enum Storage
{
    Timeouts = 1,
    Subscriptions = 2,
    Sagas = 3,
    GatewayDeduplication = 4,
    Outbox = 5,
}

(source - NSB Storage enum (subject to change))

MyFancyPersistence (besides inheriting from PersistenceDefinition) declares default features with Defaults method and what it can support with Supports method.

public class MyFancyPeristence :PersistenceDefinition
{
    public MyFancyPeristence()
    {
        Defaults(s => s.EnableFeatureByDefault<MyDefaultFeature>());
        Supports(Storage.Timeouts, s => s.EnableFeatureByDefault<MyTimeoutsFeature>());
        Supports(Storage.Subscriptions, s => s.EnableFeatureByDefault<MySubscriptionFeature>());
    }
}

The individual features look like below. Again, this is a simplistic implementation.

public class MyDefaultFeature :Feature
{
    protected override void Setup(FeatureConfigurationContext context)
    {
        var settings = context.Settings; //instance of ReadOnlySettings to get endpointname,etc
        var pipeline = context.Pipeline; //instance of PipelineSettings to register steps in NSB pipeline
        var items = context.Container; //intance of IConfigureComponents to ConfigureComponents
    }
}

public class MyTimeoutsFeature :Feature
{
    protected override void Setup(FeatureConfigurationContext context)
    {
        //Configure components that implement timeouts implementation into fav storage
    }
}
public class MySubscriptionFeature :Feature
{
    protected override void Setup(FeatureConfigurationContext context)
    {
        //Configure components that implement subscription implementation into fav storage
    }
}

For more detailed implementations, please take a look at NHibernate, InMemory or RavenDb.

Conclusion

In my opinion, most of these changes are good changes and make sense. They provide more flexibility like this stackoverflow post. If you are an early adopter, you will have to deal with the documentation that is catching up and a lot of changes in the public API. The open source nature of the project overcomes all of these even if it can be a little time consuming to dig for little changes. I hope these posts help and save some time and grief.

This is a continuation of my previous NServiceBus upgrade post. Logging The logging functionality that used to be in NServiceBus.Core is moved to a separate set of nuget packages such as NServiceBus.CommonLogging, NServicebus.Log4net and NServiceBus.NLog SetLoggingLibrary from V4 is removed from V5. LogManager.Use<Log4NetFactory&…

Read More

Upgrading NServiceBus to V5 from V4 - Part1

Sometimes upgrading a framework could be a pain. Upgrading NServicebus to 5.X.X from 4.X.X is not any different. There are a lot of changes in the public API and not everything is documented or obvious. Sometimes the error messages could be misleading, too. This was a surprise for me from a mature open source project. So, I decided to document the missing links and save some time for anyone who is facing similar issues.

Documentation

On Particular's site, under the migration tag, you will find only two posts which are fairly long.

The API-diff documents only shows what was removed like below:
The API difference doc It only shows what was removed. So, if you were using one of the removed types or methods, you are lucky to find a mapping in V5 straight away. In my opinion, users would much appreciate the document that shows how a method looked in V4 and how it is in V5 now. The complete API differences can be found here. It looks like a work in progress and there are signs of improvement.

The upgrade guide from V4 to V5 is a little better but does not cover everything that is changed in V5.

Upgrade Guide

The best form of documentation is the code base. For V4.6.7, you can go here and for the latest here. This is the beauty of the open-source software. The good tests are always more helpful to show the usage than the exhaustive documentation. If so many changes get done in a big bang approach like this release, it can be a time consuming affair for the user.

To be fair, very few open source projects have awesome documentation like Knockoutjs

After upgrading the version, depending on your usage, you may see a ton of compile time errors. What is the reason behind this?

The biggest change in V5

The Configure class has lost most of its useful static methods such as Configure.EndpointName. The BusConfiguration is introduced. An instance of BusConfiguration is being passed in. So, to further illustrate this, if you have your endpoint like below in V4:

public class MyEndpointConfig : IConfigureThisEndpoint, 
                              AsA_Server, 
                              IWantCustomInitialization
{
    public void Init() 
    {
    }
 }

The Init() is a part of IWantCustomInitialization.

In V5,it looks like below

public class MyEndpointConfig : IConfigureThisEndpoint,
                              AsA_Server
{
    public void Customize(BusConfiguration configuration)
    {
    }
}

IWantCustomInitialization is removed IConfigureThisEndpoint is sufficient.

This means you will have to change your endpoints and build them if you are implementing NServicebus Endpoint interfaces.

Configuration Changes

If you had extensions added on to Configure class, they are pretty much useless. You will have to re-implement those on BusConfiguration instead.

Some of the methods that are available on BusConfiguration are :

ImageForNSBConfig

Backwards compatibility and obsolete errors

The way I understand backwards compatibility is, you keep old and new implementations marking the old one with obsolete attribute. It gives users the opportunity to make gradual changes.

The V5 of NSB has methods like below:(Source- NSB github repo)

[Obsolete("Please use `ReadOnlySettings.GetConfigSection<T>` instead. Will be removed in version 6.0.0.", true)]
[EditorBrowsable(EditorBrowsableState.Advanced)]
public static T GetConfigSection<T>()
{
  throw new NotImplementedException();
}

I can imagine the reasons behind not keeping two implementations (probably too much of a hassle) but that error message is not helpful at all. Will be removed in Version 6.0.0? It is removed in this version too,right? Moreover ReadOnlySettings is an interface and does not have GetConfigSection<T> method. It is an extension method of ReadOnlySettings. The extension code file can be found here. The SettingsHolder class implements ReadOnlySettings interface, so you can access it from a property Settings (an instance of SettingsHolder) like below:

public class MyEndpointConfig : IWantToRunWhenConfigurationIsComplete
{
    public void Run(Configure config)
    {
       var configSection = config.Settings.GetConfigSection<TransportConfig>();
    }
}

If you need to access EndpointName then:

public class MyEndpointConfig : IWantToRunWhenConfigurationIsComplete
{
    public void Run(Configure config)
    {
       var endpointName = config.Settings.EndpointName(); // readonly
    }
}

However, it is a readonly property.

Now, why do we need to implement IWantToRunWhenConfigurationIsComplete to access this type of information? IConfigureThisEndpoint has Customize method with instance of BusConfiguration parameter. I tried to do

configuration.GetSettings().GetConfigSection<TransportConfig>();

for config section and

configuration.GetSettings().EndpointName(); 

to get the name.

The prior throws the KeyNotFoundException with a message "The given key (TypesToScan) was not present in the dictionary." The endpoint one also throws the KeyNotFoundException with a message "The given key (EndpointName) was not present in the dictionary." but you can definitely set it there like

configuration.GetSettings().EndpointName("MyEndpointName")`.

Perhaps, this could be a write-only property.

ConfigurationComplete event that used to be in V4 has disappeared. I took it to Stackoverflow. The community and open source nature of the project helped. Again, IWantToRunWhenConfigurationIsComplete comes to the rescue but it was not obvious. So, the endpoint class may start looking like below in order to make it work:

 public class MyEndpointConfig : IConfigureThisEndpoint
    , AsA_Server
    ,IWantToRunWhenConfigurationIsComplete
{
    public void Customize(BusConfiguration configuration)
    {  
        configuration.Transactions().Disable();// 
    }

    public void Run(Configure config)
    {
        var configSection = config.Settings.GetConfigSection<TransportConfig>();
        var endpointName = configuration.GetSettings().EndpointName();
    }
}

The follow up post for more changes such as persistence is coming.

Update

The part2 of this series is now online.

Sometimes upgrading a framework could be a pain. Upgrading NServicebus to 5.X.X from 4.X.X is not any different. There are a lot of changes in the public API and not everything is documented or obvious. Sometimes the error messages could be misleading, too. This was a…

Read More

Be a Git Ninja deck

It was a lot fun presenting at the Boston code camp 21.  The slide deck with all the links is below:

It was a lot fun presenting at the Boston code camp 21.  The slide deck with all the links is below:…

Read More

Micro-ORM Code Camp 6 Presentation Links

Presentation from Code Camp 6 (run index.html from the presentation folder)

MicroStackOverflow demo project from the presentation

StackExchange public data dump

StackExchange data explorer

Forked Data importer for StackExchange 

Why Micro-ORM?

My prologue post about the talk

That annoying insert problem by Sam Saffron

Select N+1 problem (stack overflow)

Dapper Links:

Dapper Site

Dapper Extensions

Dapper Simple Crud

PetaPoco Links:

PetaPoco link

PetaPoco  on github

Simple.Data Links:

Simple.Data on github

Introducing simple data post

Simple Data documentation

Simple.Data.sample

Simple.Data.Pad (similar to LINQ pad)

Mark Rendle on Herding code pod cast

Miscellaneous  links:

Hanselminutes from with Rob Conery and Sam Saffron talking about Micro-ORM

Similar presentation

Jon Skeet on Dynamic Gotchas

Presentation from Code Camp 6 (run index.html from the presentation folder) MicroStackOverflow demo project from the presentation StackExchange public data dump StackExchange data explorer Forked Data importer for StackExchange  Why Micro-ORM? My prologue post about the talk That annoying insert problem by Sam Saffron Select N+1 problem…

Read More

Micro-ORM, Why should you care?

Before we look at anything, we must try to understand what an ORM is. It stands for object relational mapper. It tries to solve a very common problem of impedance mismatch between the set based SQL world and the loops based programming world. (This impedance mismatch has given birth to NoSQL databases (MongoDb, Couchbase, RavenDB,etc). ORM takes sets of results returned by running the SQL and maps it to POCOs (plain old CLR objects). It also tends to have some SQL like capacities that can used to send the modified POCOs to the database to manipulate data. The typical ORMs used in the .Net world include NHibernate, EF (EntityFramework), and a whole bunch of micro ORMs such as Dapper, Massive, PetaPoco ,Simple.Data, OrmLite from ServiceStack ,etc.

These tools gained ground very quickly.  Unfortunately, that spread the myth of “You don’t have to know SQL if you use ORM”. Can one say, “I am going to write something in my favorite web framework and I’m not going to care about HTTP and HTML.”?   Abstraction makes things easier to deal with but does not replace the need for understanding the underlying layer. While accessing the data, if we know our SQL well enough, we can work with these ORMs in a better way or at least not abuse them.  Otherwise, we are at a high risk of spiraling down the hell hole of Select N +1 problems along with others.

Why use Micro-ORMs again?  What’s wrong with EF?

Sam Saffron says:

“It is that the one-size-fits-all approach does not offer a snug fit. When ORMs compete, they often use a big table to describe every feature they have that the competition lacks. Unfortunately, this is a big driver for bloat and complexity, it leaves us developers confused.”

(You can read  the entire post here )

This is coming from a guy I have great respect for. He is one of the creators  of MiniProfiler and Dapper (and some other stuff), one of my favorite Micro-ORMs. He also has done Speed Asp.net MVC edition series on Tekpub.com. I could not agree more with Sam here. It does seem like EF is trying to please everyone by jamming in a ton of unnecessary features .

The maintenance of EDMX(glorified XML) is usually a pain. Code first in EF is there, but there is no stored procedure support. This big “enterprisy” framework tries to do too many things.

It usually does a very good job working through the expression trees to generate SQL to fetch the correct data, but the generated SQL can be hard to understand, let alone controlling it. You will have to use profilers like EFProfiler, MiniProfiler or SQL profiler to keep an eye on the queries being generated. The query initialization in EF tends to be slower the first time. It is also hard to debug.

Why not let the database do some of the work of query optimizations? SQL server’s (if you are using SQL server) query optimizer is exceptional. Microsoft has put a lot of research into it.  Stored procedures can give that extra performance boost by precompiling the SQL.

Performance is a critical factor when it comes to data access. We don’t want to have nasty queries consuming too many resources and taking too long to execute. Especially if you are running it on services like Windows Azure which has a “pay as you go” model. (You pay only for the resources you used).  Good performance can directly result in saving money.

If you have a Select N+1 problem, the app could be hitting the database a thousand times to get one result, which can be converted to one simple query returning all the results. It is very easy to create this type of problem with EF.  Sometimes, the developers are not even aware of it.

In addition to all this, EF can create leaky abstractions.  In your typical Asp.net MVC application, you can make it hit the database from Razor View, Controller and business logic or service layer. Everything can get tightly coupled and may end up with duplicated logic.  It is hard to stop this kind of stuff. 

Micro-ORMs, on the other hand , work on the principle of less is more. They generally have a small set of features that work very well. So where is it all going? Are we going to be stuck in SQL land all the time? On Hanselminutes, Rob Conery, nailed the answer to perfection by saying "the complexity (of big ORMs) is resolving to something simple (micro ORMS)."

My take on it is that it is good to have options, have a good  understanding of them and use the right tool for the job.

This is a prologue to my talk on Micro ORMs at Connecticut dot net user group’s Code Camp. I have put together a project on github to demonstrate Dapper, Massive, Simple.Data and PetaPoco.

Before we look at anything, we must try to understand what an ORM is. It stands for object relational mapper. It tries to solve a very common problem of impedance mismatch between the set based SQL world and the loops based programming world. (This impedance mismatch has given birth to…

Read More