ETL to handle deleted reference data

How to Handle Data that serves as both a dimension and attributes of another dimension

I have a invoicing warehouse with two fact tables. First is an invoice header fact table which reports on invoice totals and includes a surrogate key for a stat code that was assigned to the invoice. The grain is one row per invoice. Next is a part

Data Cleaning - replacing nulls and addresses

Hi, I need to do some data cleaning as part of a data warehouse project that I'm about to start. Some of the tasks will include some basic data cleaning (e.g. handling of nulls & replacing with a default value). Then there will be more complex

Best method to track deleted records on source in Oracle

We need a way to track deleted records from the source tables in Oracle. We have one source Oracle database and our data warehouse is also in Oracle. I'm somewhat familiar with CDC (asynchronous would do the job) to tap into the logs, but is there anythin

A dozen systems with many dimension codes

Hi, I have more than a dozen systems to integrate, and each has it's own version of dimension codes. The source systems need to use dummy codes, and multiple codes for one entity. This for operational reasons. So, to conform a dimension I have a

How long should -1 dummy records exist in fact tables?

Dear Kimball Group, we are using the Kimball BUS Architecture method with SQL Server 2005, SSIS, SSAS, ProClarity, PPS, and MOSS. I am at a clients site that is using Full Refresh process for building 3 data marts so far every night. We are trying to

How to handle a Flame War situation.

Flame wars have been popping up a few times, while rare, if it does happen, this will teach you what and what not to do when in/discovering a Flame War. What to do on Pivetworld: 1. Try not to get involved. 2. Always log out if you sense a Flame

Sparse dimensions, how to handle?

I've purchased the book "The Microsoft Data Warehouse Toolkit" and have reviewed it in order to build out a data warehouse solution I am currently designing. I am stuck in a spot that I don't seem to find any examples of , or am not abl

Data Warehouse with high data availability

When a data warehouse is being updated by periodical ETL load, there could be significant performance impact on the end users' reports while the underlying tables are touched. The data becomes volatile and the query results could be different between

Determining a users dynamic id

Hi all I've read quite a few forum topics and I have always managed to stumble across something that will get my problem solved. But this time I'm going to register and sign in and write up a query. I'm a bit of a newbie and to testing in general,

Data Warehousing Vendors (Manufacturing)

Dear Sir or Madam We are a local company located in Syria, specialized in Industrial Confectionary Development (Food; Biscuit, Chocolate and Juice) The company seeks to find consultants/ vendors in order to initiate a data warehouseing project to be

Reasons to use a 3NF design over a Dimensional Model design for an EDW - Discussion

I've been dimensional modelling for well over 10 years, and strongly held the opinon that only a dimensional model would work for an EDW, however i'm now begining to think that there are certain requirements that lead to a 3NF design for an 'Enterprise

Data Mining

I'm requesting that folks pitch in and give me their results from bosses. What I'm hoping to accomplish is first of all a verification of preexisting drop rates already determined for the facebook version. Second I'd like to build a proper gear/unit/fp

deleted account should mean posts go bye bye

if someone or u deletes someones account it should delete all the posts and or topics they made. not say annonymous