18 October 2011

Dimension Relationship in a Recursive Hierarchy

I'm now working on a data warehouse project with SSAS 2008 R2 (Sql Server Analysis Services). It is very refreshing to do something different after doing MS-Access for so long.

While designing the dimensions in Analysis Services I had the following problem: working in a Financial Company, every client has a client group. Sometimes it is the same client group as the client, and sometimes it isn't. Pretty classic material:
John & Jane Smith (account id: 1) and John Smith (account id 2) both belong to the same "client group" of John & Jane Smith. So far nothing special: the dimClient table has a self referencing attribute to dimClient with the id of dimClientGroupId.  The problem arises when I need to group by properties of the client group, and not by the client itself. Sometimes the adviser for "dimClient" is different then the adviser of "dimClientGroup", and the business needs to view the data by the Client Group, and not by the Client.
So I have quariable attributes that are different in different parent-child levels and I need to present the data properly. I tried using the "client group adviser" as an attribute to the client but it didn't work. It also didn't group the data correctly.

So, how do I use the attributes of the parent dimension in a self-referencing dimension?(Please note: using "self referencing" is better than "parent-child", because then you get to all sort of parenting material; useful, but not what I'm looking for right now!)

Generally, it is advised against using too many parent-child hierarchies in Analysis Services, unless "there's a genuine business requirement" (which happens to be the case).

So, after designing all the dimensions I had to link the "adviser" dimension to the "client" dimension. That was easy (and maybe even done automatically) as every client has an adviser. Now I had to link the "adviser" dimension to the "client group" dimension and this is where I got stuck (after all, "client group" is a self-referencing dimension and is treated by SSAS differently!).
My solution has two levels. I might change between them in the future, as there are no guarantees that my first level will continue to work.

Using "referenced dimensions" I linked the "adviser" dimension to "client".
Now, I've linked the "client" dimension to "client" again and - voila! - I have the "client group" dimension and I can view in the cube the "client group.adviser" dimension and filter by it. A warning: trying to deploy the cube generates an error: "Errors in the metadata manager. The 'Dim Client Group Id' intermediate granularity attribute of the 'Dim Client' measure group dimension does not have an attribute hierarchy enabled.". Nevertheless the cube can still be processed and view from the SSMS - Analysis services, and the numbers are correct.
There is a different solutions to the same problem that is suggested in here (under the "parent-child Hierarchies" short section):
As the numbers of Hierarchies in this business model is fixed (every client has one and only one client group), Creating a bridge table of "portfolio client" which includes only the client and the client group (both are foreign keys to the "dim client" table) and again, using it as a reference dimension as follwos:
[fact]portfoio -> [dim]portfolio client ->[dim] Client Group ->[dim]Adviser sorts this problem in an even clearer manner and without ugly error messages. A good explanation about bridge table could be found in here (they bought me with the term "Graph Theory").

10 October 2011

Change a dimension to a fact

Life don't always follow the "best practicing" methodologies of Life. "Star Schema" is the best and most preferred way to design the data warehouse; unfortunately, it's not always possible. In the industry I'm at (financial planning) you would expect the measures to be some financial product valuable (i.e. money); so the major feed is actually clients and their financial products. The portfolio is the "fact" table (easy to decide by the size of it) while the clients and their attributes are the "dimension" table. Sounds OK, until they asked for the number of clients. Easy enough, you'd say, every client must have a representation in the "portfolio" fact table, so all we have to do is to use "distinct count" on the "client id" in the fact table, right? Wrong. Some clients have financial products, but not in the feed of the portfolio but in some other file which hasn't been created yet. Again, before you scream "bad design!" may I add that the original intention was to create a "source of truth" which lays in an outsourced system, and only later to create the date warehouse itself?
So here I am, trying to create count of clients per month per adviser, or in DW/BI terms: trying to use a dimension table of a slowly moving dimension (that has the "date" only as time attributes) as a fact table (which would have a link to the date as a dimension). Or, as my Business Analyst put it: Before, the "portfolio" was in the center of what we want. Now, the center of attention is the client.
The solution?

Using indexed view from dimClient, created a view called "factClient". The view takes only the necessary fields from the dimClient table (no "date inserted" and no "valid till" - which is defaulted to 31 of December, 9999) and joins them to the dimDate table.
The view is created like this:


SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

SET ANSI_PADDING ON
GO

SET CONCAT_NULL_YIELDS_NULL ON
GO

SET ANSI_WARNINGS ON
GO




CREATE VIEW [dbo].[factClient]
WITH SCHEMABINDING
AS
SELECT d.[id] AS dimDateId
      ,1 AS ClientGroupCount
        ,C.[id]

      ,C.dimAdviserId
      -- Other dimension fields and other necessary fields
      --
      --
  FROM [dbo].[dimDate] d INNER JOIN
    [dbo].[dimClient] C ON c.DateInserted <= d.DateFormat AND c.ValidTill > d.DateFormat
  WHERE d.HasPast <> 0






As you can see I deleted (for the sake of this view) the columns that are particular to my database, but what I've kept are the followings:
First, I've added a field in the Date dimension table that relates weather this data has past or not. This fields get updated after the ETL process completes. The reason for that is that I didn't want to inner join with the (large) fact Portfolio table, and the other is that any other way of querying that client would result with an error of "Cannot create index on view 'dbo.factClient'. The function 'getdate' yields nondeterministic results. Use a deterministic system function, or modify the user-defined function to return deterministic results."
The second thing that happens is that it gives you the user id, and a dimension, like this:
Member Id   dimDateID      dimAdviser Id    Count
1                  1 Dec 2010     1                            1
1                  2 Dec 2010     2                            1
......
etc,
Which gives me the ability to query by adviser on a different date dimension. Giving every line the value of "1" enables me to "sum" without using a "distinct count" (or any count, for that matter) as distinct count requires a unique measure group.
Please note that I'm using SQL Server 2008 R2 which enables me to use the "date" data type instead of "datetime" datatype.
Please note that the "dateFormat" field in the "dimDate" table is in order to give the "date" version of the date id: so when the key is 20101130 (data type: int) the "date format" is '20101130' (data type: date).
(More about "Indexed Views" in here)

Now there's a new fact table the following stages should be made in the Analysis Services project:
Add the new fact table to the DSV (data source view) file
Create the necessary relationships between the fact table and the dimensions surrounding it
Create a new measure group in the cube. Make sure that the measures are defined correctly (the [Member id] field  is an INT, but I don't want it to be summed!)
Hand it over to your Business Analyst and hope that s/he approves...

18 September 2011

MdxScript(DB) (8, 5) Parser: The syntax for ',' is incorrect

This problem appeared all of a sudden out of the blue in my Cube, while trying to calculate a new member. I usually create a calculated new member using the "Calculate" sub-tab in the toolbar of the "Cube" Tab in SSAS (2008 R2).
The Solution is to go to "Script View" in the toolbar and this is what I saw there:


CREATE MEMBER CURRENTCUBE.[Measures].[Calculated Member]
 AS ,
VISIBLE = 1  ;   





With a red squiggly line under the "AS ,", as it did need a name.
Deleting those messy rows helped, as well as re creating the Calculated Members that I've needed.
Apparently, I was trying to create a new "calculated member" more than once and one of them wasn't edited. As I had quite a few (more than 10) I didn't even notice.
So while using the SSAS MDX features knowing actually how to write MDX is a useful feature even for beginners such as myself.

15 August 2011

How to find out how many decimals are in a column

This is the script to finding out the maximum number for a decimal in a column (how many decimals are there after the decimal point)

SELECT
MAX(LEN(CAST([aFloatColumn]AS VARCHAR(255))) - CHARINDEX('.',CAST([aFloatColumn]AS VARCHAR(255))))

FROM[dbo].[table_1]

27 May 2011

COM Error on PublishForm for Outlook (and you wouldn't believe the solution!)

The problem: an oft file for deployment.
Here is some code in C# which I've copied & pasted into my C# code: (source is in here)

string filename; string location;
_Application olApp = new Microsoft.Office.Interop.Outlook.ApplicationClass();
_NameSpace olNs = olApp.GetNamespace("MAPI");
MAPIFolder oInbox = olNs.GetDefaultFolder(Microsoft.Office.Interop.Outlook.OlDefaultFolders.olFolderInbox);
_MailItem oMailItem = (Microsoft.Office.Interop.Outlook._MailItem)olApp.CreateItemFromTemplate(location, oInbox);
Microsoft.Office.Interop.Outlook.FormDescription oFormDesc = (Microsoft.Office.Interop.Outlook.FormDescription)oMailItem.FormDescription;


oFormDesc.Name = filename;
oFormDesc.PublishForm(OlFormRegistry.olFolderRegistry, oInbox);
oMailItem.Close(OlInspectorClose.olDiscard);


And this code would fail on the line

oFormDesc.PublishForm(OlFormRegistry.olFolderRegistry, oInbox);

With an error "COMException was Unhandled"

The reason:
The filename (the parameter given) included the string " - ".
So for filename "Broker forms" - pass,
"Broker - forms" -fail.

06 January 2011

Update on a linked table failed

The problem:
"Update or insert of view or function 'function_name' failed because it contains a derived or constant field (#4406)"
in my access database which is linked to SQL server

The reason:
The underlying view is of type "union"

The solution:
change to a single source (not a union select)

17 December 2010

Where did all my values go to?

Or: Mess with values that you enter in a datasheet view.
This is the scenario:
There is a form which contains a sub form which is linked to values in a text box on the form (or to values in a different sub form). The form is displayed as datasheet.
You enter the values you need, all of them are from different linked tables,
and when you go the the next record your values had disappeared and instead you get totally different values!
For example:
the value should be "style" in the outside form
so for the style "myStyle" on the Parent Form the values in the sub form should be:

Bold 5%
Italic 10%
Normal 30%

But when you insert them they're seems to be taking from an existing style somewhere else. When you look in the table in the database, the values had entered correctly, it's just that the display is wrong.


The solution:
Add the id of the linked value to the subform, just hide it. That sort the mess out!

Environment:
Ms-Access 2003, Linked SQL tables.