Taoffi's blog

TFS… introduction to this series

Microsoft TFS (Team Foundation Server… recently renamed to ALM: Application Lifecycle Management) is a great project management platform.

TFS is built on meta-model abstractions (that were first seen, in Microsoft's products, in SharePoint).

TFS, like SharePoint, is composed of several software components which operate around several databases: the configuration database and one or more content databases.

Configuration and content databases' elements (tables, views, functions, stored procedures… etc.) represent the meta-models (schemas and business logic for each related feature) which govern and control much of the software components' behaviors.

TFS is often presented as a software project management solution. In my view, this seems a little reductive of its value. Because TFS features truly span a much larger project management scope.

Many useful literatures have been written about TFS: its features, usage, guidance, troubleshooting… etc. Few have enlightened its meta-model structures and, as of my knowledge, none has put some clear light about its (undocumented) databases' structures. That is what I will try to do in this series, more to value TFS abstractions than to nude its internal mechanics. And also to enforce, again, the meta-model approach as a rational basis for software solutions.

TFS logical architecture

For its contents, TFS maintains a database per project-collection.

Each collection may contain one or more projects. And each project can be managed according to a selected methodology (Agile / Scrum / CMMI… etc.). In fact, TFS applies and operates the selected methodology through the provided methodology project management template. Ultimately, this means you can build your own methodology and use it to manage your project with TFS. All what you need is to provide the desired methodology template.

Managing a project with TFS consists of recording and following-up specific Work items (defined by the methodology template)… those can be items like: Tasks, Bugs, Change requests, User story… etc.

As TFS was initially brewed to manage software projects, a common item in each project is the Source control repository which helps developers to maintain their source code versions and branches in a very efficient architecture composed of 'change sets'.

To discover and figure out its components' data and their required behaviors, TFS uses a configuration database. A first look at the configuration database will be the subject of this first article in the series.

The configuration database

Like SharePoint, TFS databases are delivered without documentation or relational diagrams. They simply don't contain any explicit Foreign-keys or relationships. Whatever the reason behind this choice, that seems a little surprising of a product whose main aim is projects' transparency and documentationJ.

That said, to discover, understand and correctly interpret the role of each TFS database item and their inter-relationships you should do a deep dive and be armed with some patience and empiric methods!

Hosts and processes

Let's have a look at three important tables:

• tbl_ServiceHost contains the list of services' host information. This includes the item's name, the database connection string (when relevant), the Virtual directory and Resource directory…
• tbl_ServiceHostProcess contains information about current processes. Machine name, Process name, Identity under which the process is running, start time…
• tbl_ServiceHostInstance contains the list of processes' instances per host (Process Id / Host Id / Start time).

In my case, the table tbl_ServiceHost contained 3 entries for:

• Team Foundation: root item (parentless… see diagram ParentHostId column). Connected to the configuration database.
• Two child entries each for one of my TFS Project collections:
  • Default collection.
  • Training collection.

The above relational schema allows us to query the tables and obtain results like the following about current running services:

Sample results

Ressources and services

Again, let's follow up the following tables:

• tbl_CatalogResourceType stores a list of resource types. Like: 'Machine', 'Infrastructure Root', 'Project Server', 'Team Foundation Project Collection Database'… etc. Here is some sample entries:

• tbl_CatalogResource stores resources (each referring to one of the types in the above tbl_CatalogResourceType table. Sample entries:

• tbl_ServiceDefinition stores a list of predefined services (classes) and the web access page for each. Sample entries (among other things, you may note curly braces which are place holders for some url parameters in the RelativePath column):

• tbl_CatalogServiceReference stores per-resource services' entries. (Resource Identifier / Service Identifier…).

This allows us to query the services in a way similar to the following:

Sample results:

More about TFS meta-models in following posts!

Sometime ago, I discussed a ComboBox mouse scroll issue in Silverlight. I, again, had a new problem with the ComboBox control… this time in a WPF application.

The ItemsSource update problem

In my case, I had a ComboBox whose ItemsSource was bound to a list of items which was updated dynamically, and its SelectedItem bound to one item of the list. The problem was that the ComboBox was correctly updating the SelectedItem, while the drop-down list continued to display the older list of items! ... Here is a sample illustration:

Figure 1: This is the first context:

Figure 2: The item list changed… the ComboBox correctly displays the selected item (2.476563)… which doesn’t exist at all in the items list. The drop-down list still displays the old items list!

Such odd situations make you feel that either there should be an error somewhere in your code… or that the CombBox itself is a control that you should reinvent yourself!

Check your code!

Here is my Xaml code… that really seems so ‘standard’

According to someone advice, let-s add a IsSynchronizedWithCurrentItem=”True”:

Desperately, that doesn’t solve the problem!
Another advice: Bind the ItemsSource to an ObservableCollection<T> (instead of a List<T>): nothing changes…
A third one: add an UpdateSourceTrigger=PropertyChanged to the Binding… let’s just try

The solution

Still, we didn’t try this… let’s use the IsAsync Binding attribute:

Oh… that seems to work great!
Is sofware development a Science, an Art, or Craftsmanship?!

Few days ago, many press articles exposed what was supposed to be spectacular action of the FBI against LulzSec.

The main ‘spectacular’ action this time was that the FBI succeeded in gaining the ‘cooperation’ of one young man who was, more or less, part of the young people's movement.

The ‘cooperation’ of this poor scared young person, lasted 10 months… from June 2010 to March 2011, where he sometimes worked ‘full nights’ for the FBI to deliver his colleagues and friends in the best (illegal) conditions!

According to the press news (relating FBI personnel declarations), the operation ended up by arresting 4 suspected persons (all over the worldJ).

Apparently not hoping for many more suspects through its new agent, the FBI decided to throw him away by revealing his identity to the press! (Not very much encouraging collaborating with those people… isn’t it?)

Well… by any simple arithmetic logic: 10 months of cooperation of such a great agent to simply have 4 suspects… that really seems pitiful.

Finally: The story seems in fact more about the recurrent failure of organizations like FBI to do their job (whatever this job may be!)

Communication is a vast and ancient field which ran through many evolution phases, experiments and research. Internet, as we know it today, is one of the outcomes of this human work.

In this story, no one technology proved eternal or ever-lasting. Cycles of evolution produced some usable and convenient forms of communication at a time. Internet seems to be just one of these.

But now, each time I look at a web page, I feel that just cannot last much longer (at least I hope so:)): these elastic regions, shapeless tables, unexpected fonts changes, images and colors… hazardous page reloads and other ‘partial updates’ (sometimes even more annoying)… the whole mess of plug-ins, add-ins and other artifacts…

That really doesn’t seem to be an ever-lasting model!

Searching for something on Internet is even worse. Just go search for the word ‘sequence’ and you will find yourself with a non-classified mess of subjects ranging from cinema to molecular biology!

Looking for a solution of a problem?... you may find many, often ten or fifteen years old… which rarely relates to you current question.

To keep some ‘Lasting Value’ for old archived articles, many publishers no more mention the articles’ publication date… and search engines don’t help much in finding out a time-classification of a search result. All these ‘partners’ (publishers / search engines) are happy with this. As long as the consumer (you and I) don’t complain, the business just continue to run with minimal costs!

The appropriateness between what is needed and what is offered seems to be near a break-point.

In parallel, the great rise of technologies like web services and the wide range of their implementations may just allow us to hope for something new to emerge: new stable and appealing content explorers / new relevant and coherent search engines.

With a little refactoring, this presentation can be quite good for SharePoint 2010:

“In this fully computer-animated fantasy from the creators of Antz, we follow the travails of Shrek […], a green ogre who enjoys a life of solitude. Living in a faraway swamp, he is suddenly invaded by a hoard of fairy tale characters, such as the Big Bad Wolf, the Three Little Pigs, and Three Blind Mice, all refugees of their homes who have been shunned by the evil Lord Farquaad […]. They want to save their homes from ruin, and enlist the help of Shrek, who is in the same situation…”

“While simultaneously embracing and subverting fairy tales, the irreverent Shrek also manages to tweak Disney's nose, provide a moral message to children, and offer viewers a funny, fast-paced ride.”

It is time now to set concepts aside for some more practical problems!

Last days, through the work on two different projects, I encountered DBNull twice, and faced a funny problem about locating the target TreeNode of a contextual menu!

Let us start by the first one:

DBNull in Silverlight and Windows Forms

I encountered a DBNull problem two times (in three days… that seems a bit much). The first time was in the context of a WCF service intended to feed a Silverlight application, and the second was in the context of a GridView control in a Windows Forms application.
For your information, DBNull is a .NET Type defined in namespace System (mscorlib.dll).

The first issue emerged with what appeared to be a communication problem between a Silverlight application and one related WCF service. After some research, DBNull was reported to be an unknown Type in XML serialization. The service was transmitting data read through a SQL Server database… We had a Cell class. Its Data member was an object.
A bit more research made it visible that the following code as the source of the annoyance:

In fact, when the field at cell_index was null, data_reader.GetValue() returned DBNull. Which was assigned as the cell’s Data. Serialized by WCF, DBNull was not recognized by the receiving Silverlight application.

Though the time needed to discover the source of the problem was quite long, the solution was quite simple:

// Which clearly means:

Now that the cell content is null, everything goes right (because null is a recognize Type).
It will be a great day when DBNull (which implements the ISerializable Interface) will simply be serialized as null… let us just wait.

Yet another DBNull problem!

Once the Silverlight DBNull problem solved… precisely: the next day J, I just found myself again confronted by another DBNull. This time in a Windows Forms application.
In this new case, I had a DataGridView control which was filled using a DataTable and I wanted to programmatically add a row to those displayed in the DataGridView control.
Obtaining the row’s data was simple (calling a web service or by directly querying a database).
The task was then to add the obtained data as a new row of the DataTable used as the source of the control. The problem was again related to the cells containing null values in the new added row.

In fact, if you assign the row’s cells data through a DataRow object you may not have a particular issue. Issues appear when you try to assign an object’s properties values to the row cells.

In this particular case, we had an object with some properties of nullable types: for instance int?  parent_id. The code below generated an error:

Some solutions are proposed in several forums. They mainly propose to handle some special events of the DataGridView (like DataError event) and/or play with custom interpretation of DBNull…
The solution I used seems more simple. It is based on the fact that when the row was created (using table.NewRow() method) it contained all required cells… each of which, a priori, filled with something (a sort of ‘null’ thing… we don’t really care about its type). So, we simply may assign cell values only if the data to be assigned is NOT null:

That seems to work!

Contextual menus… yes! but where is the target node?

TreeView control is a nice and useful control to show many data domains.
One nice and efficient thing of TreeView control is that it allows the developer to handle many aspects of tree nodes by their level in the tree hierarchy. You can for instance choose the icon of items according to their nest level in the tree hierarchy
Among the customizations you can do is to choose a specific contextual menu for each hierarchy level… very cool and simple to do.
When the user right clicks a node in the tree, he or she would see the assigned contextual menu and be able to execute operations in relation to the type of the node.
The thing is, when a menu is clicked, you developer should find out which node was selected when this menu was clicked… Curiously, this is not as simple as we may expect.

I naively thought, a contextual menu object will contain some relevant information about its Target object (in our case: the target Node)… unfortunately this was not true!
So, you simply find yourself with a command to execute (the menu clicked)… but don’t know on which object of the tree should you execute this command!
Few solutions are proposed to solve this problem, and none, of what I read, seemed sustainable.
Here again, we should ‘reinvent the wheel’…

Reinventing the contextual wheel!

The problem seems to be: “Locating the target tree node”. The problem’s definition starts by “Locate”… so let us think “location”.
I first started by trying to locate the mouse cursor at the moment of the click event (using MousePosition which returns the point of the current mouse position in screen coordinates)… but that seemed random, not always related to the node position (and also quite difficult to debugJ).
The next step was to think “location of controls”.
In fact, a contextual menu is a control and as such, it should have a location… hopefully relative to its parent control (which is the TreeView).
Our Click event is received from a contextual menu item (ContextMenuStrip object) which is located into a parent control (the contextual menu box), itself is a child control of the TreeView control containing the node.
The following figure illustrates this disposition 

The Owner property of the menu strip contains its parent menu control information. Through this information, we are able to obtain the location of the menu box, and thus, the location  of the first menu strip relative to the TreeView control. With some more simple arithmetic, we can finally locate the right-clicked node (the target node) using the TreeView control GetNodeAt(Point point) method :

 That works fine for contextual menus with few elements and in a ‘normal’ screen resolution.
If the contextual menu contains many items, the first item may be quite far away from the target node!...So, what?
Let us look for a more sustainable way to locate our item… we may again, rethink ‘mouse location’!

In this solution, we will keep track of the current ‘right-clicked node by responding to the MouseDown event of the TreeView.

Now, when a contextual menu is clicked, we can first see if the target node is not null… execute the command and reset the target node to null (until the next time): 

I spent a long time during the last months writing about meta-models. That was in the context of a book for Editions ENI (France) about some concepts to which I dedicated some work since quite a while (it was around 1998, as long as I remember, that I started studying the basis of the approach).

In previous posts (see, for instance, Silverlight database to DataGrid, yet another approach- Part II) I exposed a generic approach for describing database meta-data. The direct practical purpose of those posts was reading and displaying data into Silverlight DataGrid. Through this exercise, we also saw how can our structures help us in integrating business rules at several level (meta-column, meta-table… and so on).

In this same exercise, applying the meta-models approach would consist of persisting these structures into the database… to read and create them dynamically.

We would, for instance, store our meta-columns / meta-tables definitions into the database and let our software objects load these structures and transform them into ‘live’ data, integrating all defined business and technical constraints.

What are meta-models?

I use the term "meta-model" to designate "a description of a description" stored in a database and interpreted (read and presented) by one or more (specific) software objects (classes).

That is: the information stored in the database describes an object or a behavior, and the software class that reads this information is aware of the fundamental semantics of this information and is, thus, able to interpret it independently of a particular context.

This means that, according to our view, a meta-model is composed of:

§  Database objects and structures to store the model's data;

§  Software (kernel) objects that know the fundamental semantics of the database model.

The collaborative aspect between the 'description' stored in the database and the 'runtime object' (which reads and interprets that description) is essential for this approach to be applicable.

Why use meta-models?

Let us take a sample application where the software should display a page (or form) composed of a Master / Detail connected regions… i.e. a list of items in a Master grid, with a Detail form that displays the details of the item selected in the Master grid.

That 'Master/Detail’ layout may be the software application's approach for presenting the information of several entities. The application developer may choose to create a specific page for each manipulated entity… a choice which may be correct in some contexts, but which also remains questionable by its repetitive approach.

In another, more automated, approach, the software developer may prepare templates for each entity's specific Master/Detail information, and load the required template according to the context's entity.

This latest approach would be a first step for a meta-modeling approach for entities' presentation.

In fact, storing these templates as part of software objects would make them dependent of their related objects' compiled state. That is: the template cannot evolve without changing and recompiling related objects' code. Storing template's information in a database (meta-model) would circumvent this inconvenience.

In the same time, storing templates' information in a database would also require a (more or less important) change in software object's structure and behavior. Using a database meta-model approach (to describe Master/Detail templates in our case), requires the software object to act in a more abstract way. And this is what we called 'collaborative' approach (between database meta-model and runtime objects).

Finally, in the long run, our software object would constitute a 'kernel object' capable of displaying and handling any entity's Master/Detail information stored in a database meta-model.

What can we do with meta-models?

The sample exposed above can of course be extended to more elaborate models and interactions between descriptions stored in a database and their software objects' counterpart.

Let us again take another practical sample: an application that may need to normalize the appearance of asp.net data grid controls.

We know that a GridView control contains some appearance properties like:

§  Width;

§  HeaderStyle-CssClass;

§  AlternatingRowStyle-BackColor;

§  … etc.

We may choose to use the database to store values for each of these properties and let our software objects assign these values to each DataGrid (on the page's load event for instance).

Now, to handle the appearance of our GridView controls, we just need to assign new values for these properties in the database. A task which requires less effort, less time and fewer technical skills.

Meta-models and Reflection

Reflection is the process of introspection (self-examining) of the internal structures of software objects at runtime. .NET implements Reflection in some classes defined in the System.Reflection namespace.

Our previous exercise can be extended to more abstract and useful applications.

In fact, a software object (like GridView in the last sample) has properties to which we can dynamically assign values if we may simply know their name:

After locating the property inside the specified object's Type (class) we may assign it a specified value:

Our meta-model may make use of Reflection mechanisms to store property names and values in the database, to read and assign the specified values to the specified objects at runtime.

This approach may be of interest in many software applications. Its main advantages are, again:

§  More adaptable and versatile software;

§  Reducing development and maintenance efforts, time and cost!

Meta-models and method invocation

What we explained above about using System.Reflection to discover an object's property by name and assign a value to the retrieved property, is also applicable to methods:

On the other side, user interface’s command elements like Menus, buttons… are often associated with an event handler that represents the entry point for the action of the command.

To explain how a meta-model can be useful in this area, let us take an example of an update button.

An update button collects the user entries into the form fields, checks their integrity and submits them to the server for updating the specified table;

Having the meta-data and business rule constraints for the columns of the entity, this update operation can be handled through one same handler. Or be handled by a dynamically selected handler specific to the context.

So, we know that a data entry form would have an update button. And we may need to assign a specific method as the handler of the Click event for this button.

All these required information elements can be stored into a database meta-model. The meta-model software would then:

§  Read (or otherwise find) the name of the method assigned as handler of the button's action event;

§  Dynamically locate the method in the software (loading the assembly when required);

§  Invoke the method.

Again, this meta-model solution can be applied on different contexts (records submission, web service operations invocation… etc.) and/or on different objects or controls (buttons, menus… etc.).

Interoperability and Integration

The database meta-model approach may also be used in a way similar to what we already use in Interop assemblies (which create links between 'unmanaged / native' code end 'managed code').

A meta-model-aware assembly method may, for instance, prepare entries to methods that reside in non-meta-model-aware assemblies or methods.

Such methods can also be defined in different projects independent of the 'meta-model-kernel' assembly. And may, thus, allow a high level integration of existing applications (or data) into the meta-model solution.

Extensibility

As we said above, the meta-model as exposed in this paper, is a collaborative ensemble where the database descriptions are tightly connected to the meta-model kernel software.

Two more questions though:

§  What about the evolution of the meta-model's own kernel?

§  How to describe objects that we don't yet know about?

Here comes another important point which seems to represent a good basis for answering the above questions:

§  An object is represented by a set of properties;

§  A property is, essentially, a definition composed of

·         A Name

·         And an Attribute (the Attribute defines property's constraints: mainly 'Data Type');

§  A Value can be assigned to a property in the context of either a class (static property) or to an instance of this class. (Note: Value should conform to the attribute's constraints).

So we may consider the data of an object as a collection of (Property / Value)… i.e. a collection of ((Name/attribute)/Value).

This seems to be, more or less easily, presented in a meta-model, which may allow us to handle future 'unknown objects' or future unknown 'properties'.

Meta-models self-describing

On one hand, meta-models are data stored in a database… and on the other, they are extensible. Managing them can also be described by meta-models. This recursive aspect can be of interest to some applications to allow users to autonomously manage software semantic behaviors without the need of extensive technical knowledge.

Two days ago, my friend Olivier Vallée sent me an article (nytimes) talking about the emerging role of software tools in reducing costs of lawyers’ work by analyzing documents contents. This reminded me the work we did, Olivier and I, some years ago around the use of a nucleotidic approach for analyzing textual data.

So, here again to continue the subject of the earlier post!

In part I, I presented a first view of the analogies between molecular biology elements and other real world problematic. And a probable benefit of using molecular biology’s structures and be behaviors to in data modeling and software solutions architecture.

In this part, I will try to expose a simple and direct application of this approach: language modeling and analysis. There are several advantages for starting with textual analysis as a first application:

·         First, a textual object structure can easily, and somehow directly, be mapped to molecular biology elements (nucleotides, DNA strands, amino acids…);

·         Second, the textual analysis is becoming (re-becoming!) a must know (and probably lucrative as well J) technology  (see nytimes article);

Let us first try to analyze the physical (organic) structure of a textual object:

·         Characters;

·         Composing Words;

·         Composing phrases;

·         Composing paragraphs;

·         … etc;

This same physical structure can be interpreted according to other (different / parallel) logical  components:

·         Symbols (óCharacters);

·         Composing phonemes (ówords);

·         Composing expressions;

·         Composing sense (social / intellectual orientation (or mood) of the text)

Again, this parallel presentation of a same object (encountered in so many data modeling problematics) is one of the important analogies with molecular biology elements (i.e. similar, for example, to nucleotide sequences mapping to amino acids).

To illustrate another aspect of this analogy, let us take a look at the characters / words ó characters / phonemes parallel. To correctly read a phoneme, we need to locate its start-end characters. Which may overlap or be part of the ‘Words’ sequence of our text. In some way, we need to retrieve the reading frame (or phasing) of a sequence of phonemes through our nucleotidic characters structure. This phasing is independent of the characters/words sequence presentation. Which, again, brings to our mind the question of reading frames of an amino acid sequence contained in a nucleotidic sequence.

In the case of our text sequence question, characters, words, phonemes… etc. are, of course, language-dependent. However, for sake of simplicity, let us stay in the context of the English language.

Let us now take an example of how we can read a ‘mood’ sequence inside a physical text sequence (characters, words…) sequence.

The mood (social sense) of a text can be detected through the interpretation (or translation) of specific (predefined) expressions sequence(s).

For example:

“That’s great” can be interpreted differently from “hey… fantastic”

Or “Hello”, differently from “Hi”…

Or “L” differently from “J”

… etc.

Social expression of a text sequence is also, in some way, related to its internal phoneme sequence. Text phonemes actually produce a sequence of sounds that give a particular internal ‘music’ to the text. Which participates, in the end, in transmitting the specific social sense of the initial text sequence. That is probably an important aspect of poetry (?)

I will provide a practical coded example in a future post.

Egypt’s January 25 revolution is a great hope for humanity.

Against a regime of criminals with 1.8 million of “well-equipped” and “well-trained” anti-riot forces… and some more thugs, opportunists and other hypocrites… young peaceful people with human values could win.

We are living a new dawn for human values.

A new year is here… and this reminds me of Stephane Hessel, a young man born in 1917.

After having fought against the Nazis in World War II, he still keeps teaching us so many (simple) principles of humanity... still offers us a fresh and critique view of the True and False wellbeing of our ‘modern model’ of civilization… inviting us to prefer human values against established laws, and to keep our indignation in front of ‘financial market dictatorship’. (Read this interview)

Happy New Year Stephane Hessel… we will try harder!

Pour nos amis francophones :

·         Stéphane Hessel sur Wikipédia FR…

·         Lire la présentation de son livre ‘Indignez-vous’ !