Taoffi's blog

prisonniers du temps

Meta-models: towards a universal dependency injection framework

Intro

I joined an interesting presentation today about unit tests. Part of the presentation was related to dependency injection usage for unit testing. That brought to my mind again the benefits of meta-models.

The problem

Unit testing often faces the problem of having to instantiate objects that may impact the test itself or simply make the test impossible.

A sound example of this is when the method to be tested requires the instantiation of an object requiring, for instance, a database connection.

In this schema, we have 3 actors

  • The function
  • The caller
  • The object (SomeObject) which cannot be instantiated.

 

Traditional recipe

One, now traditional, recipe is to use an Interface (instead of the specific object) as a function parameter. And, according to the context, instantiate and use an object that implements the defined interface. The figure below illustrates this.

 

Good solution… a little acrobatic, but OK… good.

 

A problem, though. We now have 6 actors (to code and maintain):

  • The Function
  • The caller
  • The original 'true' object (SomeObject)
  • The Interface
  • The interface implementations (=2)!

 

On the other hand, testing a function may also need to instantiate a 'true' object context. In which case, we would have to rewrite (and recompileJ) our test code to instantiate a 'true' implementation when required.

A third point: in this solution, we defined the Interface to represent an abstraction of the original object. What if the original object itself was, for instance, a database record? This would then introduce a new actor in our playgroundJ

 

As Bjarne Stroustrup puts it:

"Any verbose and tedious solution is error-prone because programmers get bored" J

 

Meta-models may be a better way

In the current case, meta-models (see my previous posts) can be used to more simply describe all actors (object structures / methods / properties…).

An illustration:

 

 

Reminder: meta-models are closely related and enforced by Reflection (assemblies' meta-data). Through meta-models AND Reflection (namespace System.Reflection) developers can gain more flexibility to create versatile and scalable software.

 

In our present case, meta-models not only expose less actors to code and maintain, but also lower the complexity level of involved actors (for instance: maintaining database records instead of hard coding implementation).

 

The main goal for this approach is to ultimately write fewer universal methods to invoke and instantiate any object!

Will try to deliver a code sample in a future post. In the meantime, you may download meta-model sample code here!

TFS & SQL Server 2008R2 msxmlsql issues

I recently went through another new issue with TFS:

I had the TFS database and Data warehouse both installed on a SQL Server 2008 instance.

After installing a new SQL Server 2008R2 instance, TFS just stopped working complaining about database access.

It took me a while to re-discover that the source of annoyance was – again - the msxmlsql.dll and msxmlsql.rll files located at:

\Program Files\Microsoft SQL Server\100\Shared

And

\Program Files\Microsoft SQL Server\100\Shared\Resources\1033

respectively.

 

For some reason (seems useless to spend time searching J) the versions to use are with TFS are:

msxmlsql.dll (1,311,256 bytes); 07/10/2008; file version: 2007.100.1600.22

msxmlsql.rll (52 248 bytes); 07/10/2008

 

The files that caused my problem:

msxmlsql.dll (1 308 000 bytes bytes); 04/03/2010; file version: 2009.100.1600.1

msxmlsql.rll (48 992 bytes); 04/03/2010

 

 

 

 

MOD: Mur Oriented Development

MOD est un pattern connu de gestion de projets. Il s'agit de réussir son entrée dans le mur !

Sa pratique est assez simple : elle consiste justement à éviter, au dernier moment, de rentrer dans le mur. Adopter MOD est facile. Le maîtriser et le réussir est une autre affaire !

Champollion: 20 months and 20 days in ancient Egypt

 

Jean-François Champollion who made the first translation of the ancient hieroglyph (in 1822-1824, before ever going to Egypt) made his first (and last) journey to Egypt in 1828-1829. He spent 20 months and 20 days in this journey (of which the last 60 days in quarantine in Toulon when he was back homeJ).

An amazing book "Lettres écrites d'Égypte et de Nubie en 1828 et 1829 by Jean-François Champollion" (there should be an English translation… don't really know) contains a collection of letters - written to his friends, family and some officials- that recount this journey in a very smooth and pedagogic way. In an annex, the book contains a very dense description of the History of ancient Egypt and neighboring civilizations.

Champollion was evidently passionate about ancient Egypt. During this journey, he seems to be revisiting places and events that he knew quite well. And he recounts all of this in a fascinating way.

One thing that may seem a little strange for us, is the time spent in travelling. For instance: it took him several months to travel from Paris to Luxor (in the south of Egypt)… which really seems too long now (probably unacceptable!).

In the same time, he could compose (with his small team), in less than 2 months, more than 600 entire exact colorful drawing copies of all the monuments he visited (including endless hieroglyphic texts!). All in quite harsh travelling conditions between the Nile cataracts.

Near the end of his journey, he heard that Jerusalem's Archbishop decided to honor him with a distinguished prize (Croix de chevalier du saint-sépulcure). He recounts that getting hold of the prize was too expensive for him (200 Louis)!

The book is also available in an audio version (in French) on the great Free Litérature audio web site.

(Not sure you can write code while hearing such a bookJ)

SQL Replication: Windows Sync Manager and orphaned subscriptions

It is quite useful and handy to synchronize SQL replicated databases through the Windows Synchronization Manager. You simply click the Sync Manager / Click Microsoft SQL Server / Right-click the desired subscription to synchronize and click Sync. Easy and great!

 

Annoying problems appear in some particular cases. I encountered one of these cases when I had an orphaned subscription (i.e. a subscription that doesn't have any more a declared publication). This may happen when, for instance, the publisher server doesn't exist anymore… or when the publication had been deleted in some circumstances.

The annoyance is that in this case, Windows Sync Manager continues to display the orphaned subscription and you have no means to delete it as this requires contacting the publisher which may not exist anymore and/or deleting the subscription from a publication which may not exist either!.

At this phase, you indeed have little choices… the most reasonable would be to leave this orphaned subscription displayed and simply ignore it (though a little frustratingJ)… That was what I did until I found some time to dig through (little information, if any, is available on this):

 

I ended up by finding that Windows Synchronization Manager reads the list of its displayed subscriptions in the registry key:

HKLM\Software\Microsoft\Microsoft SQL Server\MSSQL10.INSTANCE\Replication\Subscriptions

(MSSQL10.INSTANCE is the SQL version and instance of the subscriber server… which, in the figure below, is MSSQL10.SQL2008).

The subscriptions node contains one key per subscription with required synchronization settings.

 

I simply deleted the key related to the orphaned subscription. To avoid Windows mysteries, I also restarted the machineJ. That now works as I would like. (Craftsmanship again!)

Internet Explorer add-ons: manage allowed sites

Browsers add-ons and plug-ins are nice features that can sometimes become annoying. Flash Player, for instance, is nice but extensively used in boring commercial ads.

Fortunately you can either disable the annoying add-on or allow it to run only on specific sites.

To do this in Internet Explorer:

  • First click Internet options / Programs / Manage add-ons.
  • Select the add-on and click More Information.
  • By default, the add-on is allowed to run on all sites. This is displayed as * (asterisk). Click Remove all sites.
  • Click OK to confirm your choice and close the Internet options dialog. You are done.
  • Now, the add-on will no more be executed on any web site.
  • In fact, each time a web site needs to run the add-on, IE will ask you if you want to allow it to run on this specific site.
  • The list of allowed web sites is then maintained by Internet explorer.

 

 

After some weeks / months / years… this list may become quite long.

Now, what to do to remove just ONE site or TWO of this allowed list?

Internet Explorer allows you to REMOVE ALL SITES… There is no button to remove just ONE site!

So, you remove all sites and start a new history again… More annoying than letting the add-on run on all sites… isn't it?

 

WHERE does IE keep this list? Mystery!

 

After a lot of search, I didn't find any information. Until….

Yes… it is in the registry (Please be extremely cautious when modifying this vital thing: The registry)

 

HKey_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Ext\Stats\[add-on class ID]\iexplore\AllowedDomains

 

Example for Flash Player add-on:

When all sites are allowed:

When only some are:

 

Now, if we want to delete just ONE web site, simply delete its key.

Again: Please be extremely cautious when modifying this vital thing: The registry

 

Another TFS pause: moving your stuff

As I already said (I should not be the only one to sayJ): TFS is a precious tool for project management. The problem with TFS is the tedious processes to install and troubleshoot specific issues.

This has greatly been changed in TFS 2010 to become much less tedious (thanks to the TFS team and to Brian Harry for all the useful information) but IT IS, still, quite tedious!

For instance: One of the usual tasks we do from time in the real world is to move our stuff from somewhere to somewhere else!

Don't expect this to be as easy as you may think in TFSJ

 

The context

Let's say we have a whole TFS installation somewhere (configuration / collections databases… etc.) and we want (or need) to move everything to another machine.

At a first glance, that should be straightforward:

  • Install TFS (and SQL server) on the new machine;
  • Attach (or restore) TFS databases to the new machine;
  • Tell the new TFS where to find the databases;
  • Done!

 

Yes, it is a good path… but somewhat tortuous!

 

The recipe

On the new machine:

SQL configuration and settings

  • Install SQL Server (2008 or 2008R2). Note the following elementary observations:
    • During installation, you should select a CI AS (case insensitive / accent sensitive) collation (either Windows or SQL);
    • Prefer a 'default instance' installation
  • Attach (or restore) all TFS databases (configuration database, collections and datawarehouse databases.
  • Install latest SQL server updates and Service Packs.

 

TFS configuration and settings

  • Install TFS
  • DON'T click Configure at the end of the install process
  • Launch TFS admin console
    • Click Application Tier node
    • Click Configure Installed Features
    • Select the 'Application-Tier only' Wizard
    • Click List Available Databases.
      • This will display detected TFS configuration databases on the SQL server. If none is detected, then you have to go back to SQL configuration and see what is missing
    • Select the configuration database
    • Select the account service
    • Check and correct errors and warnings. You may need to go back to SQL configuration steps.
    • Click Configure

 

Problems encountered?

If you encounter problems (this is often the case with TFSJ), you may need to uninstall and restart the Application-Tier configuration:

  • Open a command prompt (Run As Administrator)
  • Go to TFS tools directory (%program files%\Microsoft Team Foundation Server 2010\Tools)
  • Uninstall the Application Tier: Type the following command:

tfsconfig setup /uninstall:ApplicationTier

  • Restart the above TFS configuration steps at Launch TFS admin console

 

Security configuration issues

Original SQL server configuration, TFS and various service accounts may not match the new machine's configuration. This may generate some errors that can be corrected using SQL server management studio and, in case, TF admin console (Administer security option). It is sometimes necessary to go through TFSSecurity command line utility.

 

Please note: all this should be considered as "useful hints" to solve TFS migration from machine to machine. You should be armed with patience and try find other simple and reliable paths for your specific problems.

Anyway, Hope this may help!

Object Visibility and learning cycle… just a vision of

In everyday work, we come through new tools or new versions of tools we knew before.

The cycle of playing with and manipulating these tools’ objects to attain a reasonable level of mastering… is often daunting!

To minimize the learning cycle of tools, some helpful features have been introduced: like documentation, tooltips… etc.

Documentation is of course important, but, for me, tooltips are much more helpful. They just appear when I need them in the context of using the object or the property.

The thing is: when you manipulate an object for the first time, you really know few things about its composition and, even less, about its role in the global mechanics of the tool in question. You just know it is there, and it should be there for some reason and that you should spend some time to understand: its role, its structure, and finally how can it be useful for you (if at all it could be!)

One way to shorten this learning cycle is to let others show you how to use the tool and its object. Quite useful, but, on one hand, this occults some side of self-experience (important)… and also negatively interferes in your critical view of the tool (which is often useful for the tool’s enhancement itself)

What we do to know about (most) simple toys seems more rational: You just set something on or off (left/right or up/down…)… then put the toy on work and see... after some cycles you end up by figuring out what is the ‘best’ position for your needs (or mood!).

Another representation, which may also give an interesting slant related to this subject, is the DeepZoom technology used in maps applications where you can first see the whole world map, and then, zooming-in on the map you get more details about a given country, city, streets, buildings… etc.

A good path for reducing the time and effort needed to learn a subject or a tool would be:

  • To be able to see (explore) the global image of the subject or tool’s structure in action;
  • Be able to zoom-in on its objects and see their properties (progressively detailed according to your zoom level);
  • Be able to change the value of a give object’ property… and perceive the impact of the change on the global behavior.

This proposed path cannot of course be applied in all situations or contexts, but can be useful in many (most) cases.

We may, for instance, need to create a simulation context inside which we can ‘run’ the specific tool or object. An approach which may also be useful for product tests and benchmarking.

Some interesting works have been done on some aspects (like The Property Grid project)… More is to be done on the visualization of objects and properties by zoom level. Will try to write a sample on this in a future post.

 

A dive into the undocumented TFS meta-model – Part II

 

In a previous post, I started exploring the TFS configuration database. Let's continue here by having a look at some more configuration objects and relationships.

TFS Config security objects

 

Referenced table

Primary column

Table

Foreign column

tbl_security_identity_cache

tf_id

tbl_gss_group_membership

member_id

tbl_security_identity_cache

tf_id

tbl_security_domain_groups

group_id

tbl_security_identity_cache

tf_id

tbl_security_membership_cache

container_id

tbl_security_identity_cache

tf_id

tbl_security_membership_cache

member_id

tbl_gss_groups

tf_id

tbl_gss_group_membership

parent_group_id

tbl_security_domain

domain_number

tbl_security_domain_groups

domain_number

tbl_security_domain

domain_number

tbl_security_projects

domain_number

tbl_security_identity_type

type_id (int)

tbl_security_identity_cache

type (tinyint)

The relationship diagram above suggests that we can query user logins, group-membership for specific projects by a query like the following:


SELECT TOP (100) PERCENT
    login.display_name  AS user_name,
    proj.scope_name     AS project,
    login_grp.display_name AS user_group

FROM dbo.tbl_security_membership_cache AS membership INNER JOIN
  dbo.tbl_security_identity_cache AS
login ON membership.member_id = login.tf_id INNER JOIN
  dbo.tbl_security_projects AS proj INNER JOIN
     dbo.tbl_security_domain AS domain INNER JOIN
       dbo.tbl_security_domain_groups AS grp ON domain.domain_number = grp.domain_number
       ON proj.domain_number = domain.domain_number
     ON
membership.container_id = grp.group_id INNER JOIN
  dbo.tbl_security_identity_cache AS
login_grp ON grp.group_id = login_grp.tf_id

ORDER
BY user_name, project

 

The above query may give us results similar to the following

user_name

project

user_group

Administrator

Research education "agile

Administrators

Administrator

Research education "agile

[TEAM FOUNDATION]\Team Foundation Service Accounts

Administrator

SourceSafeProjects

[TEAM FOUNDATION]\Team Foundation Service Accounts

Administrator

SourceSafeProjects

Administrators

Administrator

DefaultCollection

[TEAM FOUNDATION]\Team Foundation Service Accounts

Administrator

DefaultCollection

Administrators

 

TFS database pause: Change set quantitative statistics sample

Where to find change-sets information?


TFS source control maintains change-sets for projects in the corresponding collection database. Each change-set is of course composed of the modified project files at the time of the Check-in.

Change-set information is stored in the tbl_ChangeSet table of the project collection database. That is: if you have a project collection 'Default Collection' stored in a database named tfs_default_collection, you will find change-sets information in the table tfs_default_collection.dbo.tbl_ChangeSet.

Files information is stored in tbl_files table, and file-versions information in tbl_Version table.

So, to obtain the list of files of a change-set number 861, for instance, you can execute a query similar to the following:


SELECT TOP (100) percent
    chg_set.CreationDate,
    chg_set.ChangeSetId,
    v.FullPath

FROM
    dbo.tbl_ChangeSet (nolock)AS chg_set INNER JOIN
    
dbo.
tbl_Version (nolock)AS v ON chg_set.ChangeSetId = v.VersionFrom LEFT OUTER JOIN
    
dbo.
tbl_File (nolock) AS f ON v.FileId = f.FileId


WHERE (chg_set.ChangeSetId = 861)

ORDER BY chg_set.CreationDate, v.FullPath

 

 

Which may produce results like the following:

CreationDate

ChangeSetId

FullPath

2012-05-05 20:22:31.800

861

$\Research education "agile\SimpleUniConverter\commands\UniConvertCommandModel.cs\

2012-05-05 20:22:31.800

861

$\Research education "agile\SimpleUniConverter\Convert.xaml.cs\

2012-05-05 20:22:31.800

861

$\Research education "agile\SimpleUniConverter\Convert.xaml\

2012-05-05 20:22:31.800

861

$\Research education "agile\SimpleUniConverter\MainWindow.xaml.cs\

2012-05-05 20:22:31.800

861

$\Research education "agile\SimpleUniConverter\MainWindow.xaml\

 

How many bytes changed?


As you may have noticed from the figure above, change-set file information contains a FileLength column. This suggests that we can obtain the number of changed bytes of a given file between two change-sets… through the following query results we can know that the file grew 696-551 = 145 bytes:

CreationDate

ChangeSetId

FullPath

FileLength

2012-04-30 22:33:10.040

843

$\Research education "agile\SimpleUniConverter\Convert.xaml.cs\

551

2012-05-05 20:22:31.800

861

$\Research education "agile\SimpleUniConverter\Convert.xaml.cs\

696

 

Who changed what?


The change-set table provides an CommitterId column through which we can know who made a given check-in. As this column contains an ID (integer refering tbl_identity.IdentityId), this is not straight forward information. We have to join other tables to obtain something useful.

In fact, the committer user name is stored in the Constants table. So, we have to join the Change-set's CommitterId column to tbl_identity.IdentityId, and then join tbl_identity.TeamFoundationId to Constants.TeamFoundationId… we can then get the user name in the Constants.DisplayPart column (or Constants.String column)

 

 

This can be presented by the following change in our initial view diagram:

 

CreationDate

ChangeSetId

FullPath

FileLength

DisplayPart

2012-04-30 22:33:10.040

843

$\Research education "agile\SimpleUniConverter\Convert.xaml.cs\

551

Taoffi

2012-05-05 20:22:31.800

861

$\Research education "agile\SimpleUniConverter\Convert.xaml.cs\

696

Taoffi

 

Changed bytes tracking generalization


It would, ideally, be beneficial to integrate such information into the TFS analytical database.

We still can do some turnaround by directly querying the database tables.

The idea is to list all change-set files in the order of their respective check-in date. To handle the probability of identical dates / files, we can use a row number to distinguish changes. This may be done like in the following view:


CREATE VIEW [dbo].[xtfs_changeset_files_qry]
AS
SELECT
TOP (2147483647)
    ROW_NUMBER() over( ORDER BY chg_set.CreationDate) as row_number,
    chg_set.CreationDate,

    const.DisplayPart        as user_name,
    chg_set.ChangeSetId,

    v.FullPath,
    f.FileLength

FROM dbo.tbl_ChangeSet (nolock)    AS chg_set INNER JOIN
dbo.tbl_Identity (nolock)    AS login ON chg_set.CommitterId = login.IdentityId INNER JOIN
dbo.tbl_Version (nolock)    AS v ON chg_set.ChangeSetId = v.VersionFrom LEFT OUTER JOIN
dbo.tbl_File (nolock)        AS f ON v.FileId = f.FileId inner join
Constants (nolock)        AS const ON const.TeamFoundationId=login.TeamFoundationId

ORDER
BY chg_set.
CreationDate, v.FullPath


GO

 

We can then refer the view to itself (current version / previous version) obtain row-relative quantitative changes:

SELECT v1.row_number,
       v1.CreationDate,
       v1.user_name,
       v1.FullPath AS v1FilePath,
       v1.FileLength AS v1_length,
       v_next.FileLength AS v2_length,
       v_next.FullPath AS v2FilePath,
       ISNULL(v_next.FileLength, 0)
          - ISNULL(v1.FileLength, 0) AS changed_bytes

FROM dbo.xtfs_changeset_files_qry AS v_next RIGHT OUTER JOIN
     dbo.xtfs_changeset_files_qry AS v1 ON v_next.FullPath = v1.FullPath
        AND v_next.row_number = v1.row_number + 1
        OR v_next.FullPath IS NULL
        AND v_next.row_number IS NULL