Moved from my previous blog, posted on Nov 13 2011, 12:00 AM.

The contents of this article is no longer actual, but will be refered to in a new and more actual view on this matter.

 

Managing Development (Part I)

With my previous post I unintentionally caused quite a stir between the partners who develop tools that allow us to manage our development, not just because my big mistake in thinking Mergetool was not yet updated to pages, which it was, but also in my firm opinion that metadata and tools should not be in a customer or development database.

So I believe that this deserves more attention, explanation and clarification.

Disclaimer

This post is solely my option on this subject. I have been involved in Dynamics NAV development since 1999 and been busy with managing this ever since. I’ve tried many ways and my current opinion is based on this experience.

As for the iFacto ReVision tool, you should know that Eric Wauters (Waldo) used me as a sounding board for their ideas, so it is not a surprise that this tool is based on my current opinion of how object management should be done. However I do not own any shares in this tool or the company, and I am not using the tool in my own projects.

Introduction

Both at Directions USA 2010 and NavTechDays 2011 I presented a session called “Tools and Tricks that make development easier and more organized”. In this session I spent a lot of time on how to manage objects. Reasons for this are both that many partners are struggling with this and doing it the wrong way. But when it comes to managing development there is much more than managing objects. Methodology and standards are a big part of this. No matter how good your objects are managed your development will be a mess if all your developers use their own standards.

This first post will be about managing your objects and is hopefully the start of a series, depending of the comments and reactions.

Other parts may be about methodologies like Waterfall & Scrum, Architecture, Standards and maybe even the CfMD label in this context. I would also like to write about how the object designer can be changed to accommodate our needs.

What does Object Management mean

Whenever you change an object in Microsoft Dynamics NAV it’s nice to document why you have made that change and if you develop a feature that requires you to change more than one object you want to know which objects are changed when you move this feature into another database. That is what Object Management means in a nutshell.

Object Management in Microsoft Dynamics NAV

The object designer does not allow you to do much with this information. There are four features you can use.

Modified Flag

Whenever you change or create an object in the Object Designer the modified field is set to Yes. It is considered best practice to disable this flag whenever you move objects out of the database into another system. Doing this the modified flag allows you to manage all changes.

Example

Mike does development for a company called Contoso. The database does not contain any unreleased changes so none of the objects have the modified flag set to “Yes” and there are no other developers in the database. Mike changes three objects for his development project. By filtering on the Modified flag he can easily see what he has done and which objects to export when he is finished. When Mike is finished he clears the Modified flag to “No”.

Limitations

As you can read in the example this method have many limitations. It only allows you to work on one issue at a time and requires you to export them before starting on a new change. Other than the export file there is no history of the change that is easy to see for others.

Version List

The Object table has a field called Version List. This is a text field that allows you to store 80 characters. The field is used by Microsoft and indicated the last version the object was changed and if it was changes for localization, also with a version number.

If used fully by Microsoft, the field allows us to add maybe 60 characters.

Example (I)

John has also been asked to make a change in the Contoso system. He works on the same database as Mike. This means they cannot use the Modified flag anymore. They agree to update the version list field with their initials so they can see who has modified which object. When they export the objects they remove their initials so the version list is clean.

Example (II)

Mike wants to keep track of changed objects per change request. He decides to assign numbers to each request and add this number to the version list.

Limitations

The first example requires both developers to manually populate the version list field. The second example will run into issues when the limit of 80 characters is reached.

Documentation Trigger

Each object in Microsoft Dynamics NAV allows you to write a book in the Documentation Trigger. This trigger is not used by Microsoft and ships blank with the product.

Each field in the Tables also has a Documentation property.

Example

John needs to fix an issue in one of the changes that Mike has done in the Contoso system. Because Mike always explains his change in the documentation trigger making it easier for John to find the place that Mike has modified.

Limitations

As with the Version List, using the Documentation trigger is not mandatory for developers making it easy to forget. Also finding a balance in the details of the change can make it difficult to read them. Solutions with a long history might have so much documentation that it’s no longer easy to find the information required.

Object Locking

Version 2009 R2 of Microsoft Dynamics NAV got shipped with a new feature Object Locking. This process can be done both manually and automatically and will update the Locked and Locked By field in the Object List.

Object Locking is mainly intended to prevent developers from working on the same objects and not to implement versioning. In fact, when used automatically the locked flag is already activated by opening the object and left that way even if the object is not changed and only looked at. This makes it difficult to use.

The Locked and Locked By fields are NOT exported when the object is moved to another database.

Object Versioning vs. Objects Release

Before we can proceed to solutions to more advanced Object Management it is important to talk about the difference between an object version and releasing objects.

Object Versions

Each time something changes to an object, it is a new version of the object that can be compared to the previous version, but from a change request perspective a specific version of an object is not interesting. A change request is a group of objects. This group will move from one database to another.

Release Versions

When we release in Microsoft Dynamics NAV we don’t release objects, we release a feature or a couple of features. This release only makes sense if all objects in the release work correctly with each other and the new feature works.

From a business perspective it is more interesting to version these features as packages than versioning the individual objects.

Basically this means we are versioning to entities. We’ll have different versions of an object and when a change request ships as a package this will have a version to.

Versioning Schema

If we put this theory in a schema the change request is the central object. The Change Request leads us to change one or more objects. When finished, one or more change requests will be released as a package, being a combination of changed objects allowing you to use the new feature in another system.

Overlap & Dependencies

When two or more change requests require a modification in the same object we cannot release them separately unless we develop them one-by-one. Therefore a release can and sometimes should contain multiple change requests that are dependent of each other.

Where to implement Object Management

Even if you can do everything in Microsoft Dynamics NAV, does not mean you should do everything in it. Why is that, you might ask. Isn’t the development database the most ideal place? The answer is “No” and I’ll explain why.

Production & Development databases

The Microsoft Dynamics NAV database contains a minimum amount of metadata. Each object has only one version and does not know about its past. Reason for this is simplicity which is a very important feature of the product.

In software development it is also best practice to remove additional metadata and go into production with a clean system. Microsoft Excel 2010 does not contain information to be compiled to go back to Excel 2000 or any version in between.

Development databases need to be refreshed from time to time. The data gets old, or even corrupt. If your metadata would be in the Development database and not in the Production database it becomes increasingly difficult to easily replace Development with a fresh copy of Production.

Development in a Production database

Also from time to time you’d want to develop in another database than your development database, for example making a quick page or report change in a production system. In a test system, after running conversions, you might want to make some minor changes without going back to development and release back to test. And large organizations may have multiple teams working on different parts in their own separate development databases.

Al these scenarios are not supported if your Database Management tool is bound to one database.

Managing you Object Management

If your Database Management Tool is inside your Development databases you’ll create a new task of managing the tool. When you start a new project you’ll start with the last version of the DevTool. But an older project might run an older copy of your tool.

Database Management Outside of Microsoft Dynamics NAV

A solution to this might be to manage your development with a tool that resides outside of Microsoft Dynamics NAV and has only one copy. This way, you won’t have to import management objects into each database, you won’t have to move metadata from one database to another and have one running version of the tool.

Does this mean the tool cannot be developed in Microsoft Dynamics NAV? Well unfortunately yes, since it requires a different kind of application that is able to reside inside your windows environment and “spy” for changes that happen in these systems.

Really? Isn’t there any other way? No, not unless Microsoft offers another way of managing objects.

Where do I store my changes

There can be several places to store your changes. It can be one of the well-known repository systems like Team Foundation Server, Visual SourceSafe or SubVersion. Problem with these repositories is that they don’t know Microsoft Dynamics NAV syntax.

And what about NAV as a repository

My wet dream is a combination of having an external tool that catches changes, saves the data in a repository, but the repository can also be a Microsoft Dynamics NAV database, as long as it is a database that stands alone from the project databases such as production, development etc.

This way you keep your Microsoft Dynamics NAV database nice and clean from external tools and take advantage of the possibilities of all the tools that are out these.

What are the next steps

Currently there are three big vendors on the market that deliver Development Tools for Microsoft Dynamics NAV.

Vendor	Tool Name	Free Version	Commercial
Idyn	Object Manager (Advanced)	√	√
iFacto	ReVision Source Control		√
MergeTool	MergeTool	√	√

Currently Idyn is by far market leader with their product, and it’s a great product that goes way beyond Object Management as described. It is completely developed inside Microsoft Dynamics NAV, stores data in the Development and Production database and last but not least, the code is scrabbled. Object Manager has a free version but the cool stuff is in the paid version.

ReVision is a tool that resides outside Microsoft Dynamics NAV and just scans changes from any database that is open and connects that to a repository. In my opinion that is exactly how it should be. However, ReVision does not have change request management and/or all the nice extra features that the Object Manager Advanced or MergeTool comes shipped with. And ReVision does not have a free version. However Change Requests are scheduled for the next version as are making releases of objects. Since ReVision is running inside Windows the tool can do other great things such as start the correct Client with a single mouseclick directly in the correct database and it has a great roll-back feature, although the latter is in my opinion always dangerous with Table definition changes.

MergeTool is has a different approach than Object Manager Advanced and ReVision. MergeTool does not “catch” the object changes but requires you to import them via .txt files making it more a repository than a Version Management Tool. Just like Object Manager, MergeTool also has tons of other small tools that make your life easier like a superb internal compare algorithm that can do automatic merges and renumber tools. And best of all, MergeTool is 100% free and OpenSource. Only if you are an end user doing your own development it is a commercial tool.

So what should I do

If there were to be a conclusion it would be that none of the current products match my requirements. But, there is a roadmap to success.

Path A

The first solution would be to extend Object Manager Advanced with a tool like ReVision that monitors the system for changes and stores this into the current structure of the application. This would make Object Manager Advanced the repository.

Path B

Second solution and maybe more logical from my point of view since they don´t compete is to add the MergeTool as a repository to ReVision. I’m not really sure if that is technically possible since I am not familiar with the requirements that are necessary.

What should Microsoft do

Let’s look at the subject from a different angle. If so many partners have challenges maintaining their solutions, should not Microsoft change something?

As an MVP and as part of the Partner Ready Software team I can tell that this has the attention of Microsoft. Recently the PRS team spent a day on the Microsoft Campus in Vedbæk (MDCC) where we had a chance to present our ideas based on the sessions we had in Antwerp and Orlando. Afterwards we had a discussion with the people who should be able to make decisions and with a guy called Per Rasmussen who is doing a Thesis that touches this subject. This lead to the conclusion and quote: “The core of the matter is exactly the same”.

Simplicity

One of the great things about Dynamics NAV is simplicity. Someone can go into NAV and start changing it if he or she has the proper rights to do so. Whatever Microsoft changes, this should never go away.

Layers vs. Namespaces

Sometimes I hear people talk about layers in objects. Dynamics AX supposed to have something like that where you can stack pieces of functionality on top of each other and therefore it’s a more professional product and supports multi country environments.

Personally I more like the idea of using Namespaces which is something Partner Ready Software came up with.

Namespaces can be pieces of functionality that belong together, like “W1”, “DE” or “NL” but also “ISV1”, ‘ISV2” etc. This would make the versioning list we use today more explicit since we could start doing versioning of Namespaces. It should be as easy to remove a Namespace as it should be to remove one, unless they are dependent of each other. Still I would not like to have more than one version of a Namespace in a database, all the rest should be in the repository.

Namespaces can also remove the need for object & field numbering other than maybe for user rights.

Object Trigger Changes

Even though the “backdoor” which is used by ReVision is supported in version “6” & “7” I would like this to be the “official” way of tracking object changes, but as a NAV developer I would like to have the possibility to create a tool like ReVision from within Dynamics NAV. Actually, writing this, my guess is you could if you use DotNet interop.

Repository

I would like Microsoft to give us a repository tool that we can build upon ourselves. Making it best practice to separate development metadata from customer databases.

Data Dictionary

In Microsoft Dynamics NAV we reuse a lot of terminology. Fields like Customer No. and Description are all over the place. Still it’s easy to make a typo and we need to translate the same captions over and over again. This could easily be solved by implementing a Data Dictionary that stores all the most commonly used captions and translations.

I will elaborate more about this and namespaces in the future on the Partner Ready Software wiki pages.

Other Tools

So what about the other tools? This blog post was intended to cover Object Management, my view on the subject and compare this to the current solutions that are available and Microsoft’s responsibility.

If you look at tools like MergeTool and Object Manager Advanced, you’ll see that they are much more than an Object Management tool. In my opinion this should be separated and partly improved by Microsoft.

Let’s make a short list of things that are obviously missing or should be easier in Microsoft Dynamics NAV.

Where Used

This is probably the single biggest issue that developers run into in their daily job. It has never been in the product although in some versions the menu option existed in the Object Designer even though I cannot remember the exact version.

Fortunately the MergeTool offers this for free, or you can use Notepad if you are familiar with the txt syntax of the object export like I am.

Security Setup

To setup security in Microsoft Dynamics NAV you must totally master all the Design Patterns and even memorize object numbers by heart. When developing a new function, security is part of the process and should be easier to transfer between databases, like is done in Object Manager Advanced.

It should also be possible if a user gets a permission error to have the option “request permission”. This should be as simple as the Notifications in the RTC for a system admin to say “allow” or “deny”.

Export Current Objects

Generally considered best practice and part of Object Managers deployment is the possibility of export the current status of objects before you import new objects. This should be a simple button on the “import objects” menu.

Advanced Client Monitor

Client Monitor is a great feature but the output is very hard to analyze. In all my workshops I try to tell people about the Advanced Client Monitor that was available in the Performance Troubleshooting guide for Navision 5.0. This should be part of the standard application rather than the current output.

Conclusion

Object Management is something that can be achieved in many different ways. Everyone should investigate their own strategy and analyze best practices. It also depends if you are Microsoft, and ISV, a VAR or an end user that does in-house development.

Looking at the success of the tool vendors Microsoft is clearly leaving a gap there. The good side of it is that it allows us to make our own choices to store all metadata in the customer database or to use a repository.

From my point of view there is no best-buy on the market today.

So what do I use myself? I have a very small tool I created myself with the help of Jorg Stryk that is in the development database to track changes. I only do this because that is the only thing I can do since I’m not a .Net developer. It has only four tables to store Tasks, Task Objects, Releases and Release Objects.

Everything else I do in the MergeTool which I use as a repository. Thank you Per Mogensen for making this great product.

I hope that one day my dream of a connection between ReVision and MergeTool happens in real life.

Enjoy, and hopefully for nobody any hard feelings…

Sources

The information in this post is based on research (yes, this time I did do that).

Nevertheless I might have made mistakes despite of all my efforts. If so, please let me know via the Contact option on this blog and don’t start your own advertisements. It will be moderated.

All the products have videos and most of them I watched. I encourage you to do the same and form your own opinion.

www.mergetool.com   Video

http://idyn.nl/products/objectmanageradvanced.aspx              Video

http://www.ifacto.be/en/revision         Video